Skip to main content

Client for turnkey community prediction

Project description

microprediction tests deploy

Looking for the microprediction client or hello world or stuff I've written?

Get in the game and win $125 every day

Running this script makes a heck of a lot more economic sense than bitcoin mining - but few know that. There's an even gentler introduction taking the form of a colab notebook you can open and run now. See also the examples of how to predict 979 stocks and 1000 portfolio intra-day changes in submission_examples_xray.

Me, my slack and I

This was supposed to be the microprediction client page! Github made this a "user page". Grumble. Well, hi this is my dog. This is my blog. I've always worked in the private sector, though very occasionally publish. My book is out soon. I'm dynamic and fascinating. Not really but I run a slack channel for those interested in collective open-source time-series, covariance prediction, and optimization (see the slack invite above). Join the google meet Fri noon EST - see the slack. If the Slack invite requires an email domain, it has expired so DM me on Linked-In for a new one.

The TimeMachines, Precise, and HumpDay packages

I maintain three benchmarking packages to help me, and maybe you, surf the open-source wave.

Topic Package Elo ratings Methods Data sources
Univariate time-series timemachines Timeseries Elo ratings Most popular packages (list) microprediction streams
Global derivative-free optimization humpday Optimizer Elo ratings Most popular packages (list) A mix of classic and new objectives
Covariance, precision, correlation precise See notebooks See notebook Stocks, timeseries residuals etc

These packages aspire to advance online autonomous prediction in a small way, but also help me notice if anyone else does!

The streams ($50,000 in prediction prizes)

I also maintain a live exchange where distributional time-series prediction algorithms (Python, R, Julia mostly) duke it out.

  • I publish live data repeatedly, like this say, and it creates a stream (like this stream or this one or any from the listing).
  • Algorithms like this guy compete to make distributional predictions 1 min ahead, 5 min ahead, 15 min ahead and 1 hr ahead.

In this way I can:

  • Get live prediction of public data for free (sometimes I make the "public" data obscure)
  • Indirectly benefit from hundreds of packages from Github of uncertain quality, and not just Python.

I then sip pina coladas while the accuracy magically improves over time. Read about this example.

Wanna use it?

Knock yourself out. My company funds the platform because algorithms that help solve your problem might also help solve mine. The marginal cost of their doing so is tiny. There are even video instructions.

Wanna make it better?

On the flip side you can:

  • Contribute predictive intelligence, win beer money (see competitions) and then trash talk people in our slack channel. About $50,000 a year in prizes complements the scientific motivations.
  • Automatically benchmark your work or find unexpected uses to include in your next paper. You just need to know how to predict data streams, which amounts to modifying a Python script and running it.

It's up to you if you choose to reveal your code.

Microprediction Python client

pip install --upgrade pip
pip install microprediction

The client assists use of the microprediction api. If you don't know about the live algorithm frenzy at making this possible because you skipped down this page, did I mention that an extremely simple way to grok it is to open this colab notebook and run it on Google's dime? Why yes I did mention that. This will create an identity for you and enter your algorithm in an ongoing contest to predict the next roll of a die. You'll also be exactly half way to understanding the four critical pieces of functionality.

Task Method or function Full code example Video tutorial
A Create a write_key new_key python-1: Your first submission
B Publish one scalar value at a time, usually representing a live measurement. MicroWriter.set() python-4: Creating a stream
C Send 225 guesses of the next value of a stream, after a fixed quarantine period. MicroWriter.submit() python-2: Creating your first crawler
D Retrieve community predictions (PDF) 1min, 5min, 15min or 1hr ahead. MicroWriter.get_own_predictions() colab example

Someone wanting something predicted performs A, B and D. Someone providing predictions performs A and C (mindful of the reward mechanism explained in Collective Distributional Prediction).

Memorable Unique Identifiers and other platform repos

  • The muid identifier package is explained in this video.
  • microconventions captures things common to client and server, and may answer many of your more specific questions about prediction horizons, et cetera.
  • rediz contains server side code. For the brave.
  • There are other rats and mice like getjson, runthis and momentum.

Some of my other packages:

  • winning - A recently published fast algorithm for inferring relative ability from win probability.
  • embarrassingly - A speculative approach to robust optimization that sends impure objective functions to optimizers.
  • pandemic - Ornstein-Uhlenbeck epidemic simulation (related paper)

For (American) football fans

  • firstdown - The repo that aspires to ruin the great game of football. See Wilmott paper.

For M6 competitors:

  • m6 - Illustrates fast numerical rank probability calculations, using winning. However since the rules changed, this isn't that useful for M6 anymore. There are some other resources there although you might be better served by precise. Good luck.

Microprediction versus TimeMachines

The TimeMachines package is traditional open-source software for point-estimates and confidence, whereas the Microprediction client offers live crowd based distributional prediction. Hundreds of algorithms compete at Microprediction and quite a few of the TimeMachines algorithms (see /skaters) are involved, drawn from packages like river, pydlm, tbats, pmdarima, statsmodels.tsa, neuralprophet, Facebook Prophet, Uber's orbit, Facebook's greykite and more. Some are open source (look for CODE badges on leaderboards) but others are private to their author.


One bridge between the /skaters and the microprediction leaderboards is provided by the StreamSkater class in the microprediction package, illustrated in the StreamSkater examples folder. This makes it trivial to use any skater from the TimeMachines package in a MicroCrawler (a live algorithm).

More about the Microprediction Python Client

See also or

Class Hierarchy

Use MicroReader if you just need to get data and don't care to use a key..

MicroWriter ----------------------------
   |                                   |
MicroPoll                         MicroCrawler
(feed creator)               (self-navigating algorithm)

You can pull most data directly, by the way, without a key.

Scheduled submissions versus "crawling"

The MicroWriter class can publish data or submit predictions. However if you intend to run a continuous process you might consider the MicroCrawler class or its derivatives.

Type Suggestion Example More examples
Scheduled submission MicroWriter Ambassy Fox submission_examples_transition
Running process MicroCrawler Malaxable Fox crawler_examples
Running process using timemachines StreamSkater Shole Gazelle crawler_skater_examples

A more complete picture would include SimpleCrawler, RegularCrawler, OnlineHorizonCrawler, OnlineStreamCrawler and ReportingCrawler.

Publishing absolute quantities versus changes

It is often better to publish changes in values than actual values of live quantities, to avoid race conditions or latency issues. There is a discussion in the

Certainly it is easy to publish live quantities using only the MicroWriter as shown in However you might consider:

Microprediction.Com versus

The former contains the blog, a knowledge center with video tutorials, details of competitions and prizemoney, and so forth. The latter is browser for humans looking to see how their algorithms are are performing, or whether their streams are updating.

Slack & Google Meets Tue 8pm/ Fri noon EST

Most people looking to contribute to this open initiative (and win beer money) join the microprediction slack. If that invite fails there might be one in the knowledge center that hasn't expired. There you will find Google Meet invite details for our regular informal chats.

Microprediction bookmarks

Data: stream list | stream explanations | csv Client: client | reader | writer | crawler | crawler examples | notebook examples Resources: popular timeseries packages | knowledge center | faq | linked-in | (dashboard) | (resources) | what | blog | contact | competitions | make-predictions | get-predictions | applications | collective epidemiology Video tutorials : 1: non-registration | 2: first crawler |3: retrieving historical data | 4: creating a data stream | 5: modifying your crawler's algorithm | 6: modifying crawler navigation Colab notebooks creating a new key | listing current prizes | submitting a prediction | choosing streams | retrieving historical data Related humpday | timemachines | timemachines-testing | microconventions | muid | causality graphs | embarrassingly | key maker | real data| chess ratings prediction Eye candy copula plots | causality plots | electricity case study

Probably best to start in the knowledge center and remember Dorothy, You're Not in Kaggle Anymore.





Video tutorials

See the Knowledge Center

Hey, where did the old README go?

Project details

Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

microprediction-1.1.5.tar.gz (67.9 kB view hashes)

Uploaded source

Built Distribution

microprediction-1.1.5-py3-none-any.whl (80.6 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page