Competition Rules


The choice prediction competition is open to everybody. Interested participants are required to register for the competition no later than April 10, 2018.

Please indicate in your registration for which track you plan to submit a model, and include a list of all co-authors who plan to make a submission. A team can register for only one or for both tracks in parallel (a single model competing in both tracks is also acceptable).

The size of participating teams is unlimited. However, only the first three co-authors of each winning submission (i.e. of each track) will be invited to co-author the paper summarizing the competition.

Each person may be registered as the (co‑)author of no more than two submissions per track, and be the first author of no more than one submission per track. In addition, each person may also make one additional early-bird submission (for one of the tracks), sent to the organizers by the end of January 2018 (see “important dates”). Therefore, including an early-bird submission, a person may make up to 5 submissions, up to 3 of which are to the same track.


The rules specifying what constitutes as a valid submission differ between the tracks.

Track I (Aggregate Behavior, Unfamiliar Problems) – A valid submission in this track must include a complete, functional, documented source code, written in Python, Matlab, R, or SAS. The source code must be able to read as input the parameters of the new (unfamiliar) problems on which it will be tested and provide as output predictions for the choice rates (of Option B, in five blocks of five trials each, see paradigm”). The submission of the source code must be completed on or before May 8, 2018.

If running the source code requires special resources (e.g. unusually strong computational power or specialized software packages), participants are required to notify the organizers at least 14 days before the Code Submission Deadline and make efforts to allow the organizers to test the submission code independently.

Because the organizers will run the best source codes of the best submissions, the submitted codes are expected to include a “main” file that is highly similar to the main files included in the baseline example codes. Example of acceptable submission codes, including the expected format for the main file, are given here. 

In addition to the source code, a valid submission should include also a csv file with a list of predictions (i.e. numbers corresponding to the mean aggregate choice rates in the unfamiliar problems). The submission of this list must be completed on or before May 12, 2018.

Track II (Individual Behavior, Familiar Problems) A valid submission in this track includes only a list of predictions (i.e. numbers corresponding to choice rates) for the missing data (the choice rates in the target games for each of the target individuals) in a csv file. There is no need to submit any source codes. The submission of this list must be completed on or before May 12, 2018.

An example for an acceptable submission is given here.

After the competition data is published (a day after the predictions submission deadline), all teams will be required to independently evaluate their submissions’ MSE scores, and submit the results to us.