Forecasting Workshop
▪️

Forecasting Workshop

Last updated: 4th October, 2023

1. Key Information

Length: 1-2 hours, or can be longer over multiple days/weeks

Cost: Minimal cost involved, mainly for materials (paper, pens, white board) and prizes (if any).

Target Audience: Participants interested in improving forecasting skills, decision-making, and having a fun and engaging activity.

2. Important considerations

Pros:

  • Enhances forecasting abilities through practical experience.
  • Promotes clear thinking and decision-making.
  • Fosters community engagement and friendly competition.
  • Provides a unique and enjoyable activity for retreats.

Things to consider:

  • Requires careful question design to avoid manipulative strategies.
  • May need manual scoring that you create yourself and can be done by participants without difficult computations.

3. Preparation

  • Question Design: Prepare a list of prediction questions that are engaging, tangible, and cover a range of difficulty levels.
  • Scoring System: Design a user-friendly scoring system that provides intuitive feedback and encourages calibrated predictions.
  • Materials: Gather + create necessary materials, including making prediction sheets, printing them out (or just having blank paper sheets) and pens.
  • Prizes: Consider offering a donation prize to the winner's chosen charity (optional).
  • Instructions: Create clear instructions for participants on how to make predictions and calculate scores either written on white board or posted/printed somewhere for everyone.

Event Logistics

  • Allocate time for both the prediction session and the evaluation session in the retreat schedule.

4. Event structure

  • Introduction (Day 1): Explain the purpose and rules of the prediction tournament. Present the list of prediction questions.
  • Prediction Session (Day 1): Allow participants to make their predictions and assign confidence scores to each question.
  • Tournament Progress (Throughout Retreat): Keep track of participants' predictions and scores in a central location.
  • Evaluation Session (Last Day): Revisit each prediction question, reveal the outcomes, and allow participants to calculate their scores.
  • Winner Announcement (Last Day): Determine the participant with the highest score and award the donation prize.

5. Further resources

For a more in-depth case study of this event – check out this forum post: Report on Running a Forecasting Tournament at an EA Retreat

Prediction Questions: Prepare a list of yes/no questions with clear and unambiguous outcomes, that participants can’t meddle in, has a range of difficulty, and required no background knowledge. Example:

  • On Saturday, will active COVID cases in New Zealand exceed 100?
  • Will at least two people have gotten visibly sunburnt by the time we do scoring?
  • Will at least three people attend morning yoga on Saturday?
  • Will anyone end up kayaking to the island? (There was a specific island which some people had already planned to kayak to.)
  • When we do scoring, I will pick two random people and ask if they have talked to each other. Will they both say “yes”?

Scoring System: Design a user-friendly scoring system that encourages accurate and well-calibrated predictions. Higher confidence yields more points for correct predictions but with a limit, preventing infinite points for absolute certainty. Incorrect predictions result in losses but are bounded to avoid extreme negative scores. Only correct predictions yield points, discouraging random guessing. Assigning 50% confidence gives zero points for either a win or a loss, promoting thoughtful decision-making. Example:

image

Key People to Reach Out:

Workshop Organizer: Collaborate with retreat organizers to integrate the prediction tournament into the event schedule.

Participants: Engage with potential participants who are attending the retreat or gathering and let them know that event is happening.