Last updated: 4th October, 2023
1. Why is measurement and evaluation important?
Groups have and do play a pivotal role in introducing individuals to Effective Altruism (EA) and impactful cause areas. Exchanging strategies and best practices among these groups significantly enhances their effectiveness and fosters a culture of continuous improvement. By conducting self-assessments, groups can better understand their members' needs and desires, pinpoint and address potential issues, and plan more strategic events.Such assessments also pave the way for a collective learning experience, enabling groups to share best practices, innovations, and lessons learned rather than repeatedly starting from scratch. This not only promotes a culture of continuous improvement but also helps ensure time and resources are allocated to proven, impactful strategies. See an example below:
2. Possible broad metrics to monitor
Measurement and evaluation for university groups should be tailored to the unique characteristics and goals of each group. A group's size, event nature, and objectives play a decisive role in determining the metrics they adopt. Collaborating with core members is essential to determine which measurements will be most valuable, ensuring that evaluation efforts produce actionable insights.
Some example goals to shoot for and then track with metrics and evals might be:
- High engaged EA’s: Finding people in the community who become highly engaged with the content and are very invested in the groups (this will look differently depending on the group).
- Raising Awareness: Introduce the local community to EA, ensuring they gain an accurate understanding of its essence.
- Initial Engagement: Encourage community members to participate in EA events or activities, aiming for them to leave with a positive impression and the inclination to further engage.
- Building Community Connections: Foster a deeper understanding of EA's core principles within community members, motivating them to act or commit to these principles.
However, it's essential for each group to assess its unique context and determine which goals and evaluation methods align best with its mission and capacities.
3. Things to track!
Metrics play a crucial role in understanding the effectiveness of a group's initiatives. To gauge impact and guide future strategies, it's advisable to adopt multiple metrics, being adaptive to feedback and evolving circumstances.
Engagement Duration:
- How many people have engaged with your group for x hours in the past year?
- Number of individuals who've spent at least 10, 50, or 100 hours with effective altruism content.
Application of EA Principles:
- Count of people using effective altruism principles to:
- Decide on charitable donations.
- Formulate career objectives.
- Participate in EA-related projects.
- Apply to specific job positions or internships.
- Select academic programs.
Commitment Metrics:
- Number of group members who've committed to the Giving What We Can pledge.
- Number of people who completed a fellowship in full
- Attrition rate from fellowships
Event Attendance:
- How many unique attendees were at each event?
- Ratio of returning attendees to new ones.
Content Interaction:
- How many people interacted with online content or webinars?
- Number of downloads or shares of group resources.
4. Guidelines for gathering data
Feedback Forms & Surveys:
- For self-reported evaluations, use feedback forms and surveys post-events or periodically.
- Craft clear and jargon-free questions, ensuring they're understandable to a wider audience.
- Gauge frequency; while feedback after events is beneficial, frequent independent surveys might lead to "survey fatigue," reducing participation.
- Keep forms concise. In digital formats, make fewer mandatory questions, encouraging participants to complete rather than abandon the form.
- You can use airtable or google forms to track forms and surveys. Google forms can be made into a spreadsheet through sheets which can be helpful for seeing responses and following up with certain people.
- Examples feedback forms: Event feedback form
Examples of how to keep track of other metrics:
- You can track interest in things through links that give you the number of people who opened them like bitly
- Using spreadsheets to track fellowship data or people who attended events can be useful!
- Digital platforms like google meets, zoom, or eventbrite can offer insights into online event durations and attendance, which can help calculate individual engagement hours.
- For tracking content engagement, analytics platforms like google analytics can be instrumental.
Examples of impact writeups:
5. Final thoughts
Utilizing metrics provides valuable insights into a group's impact and the efficacy of its strategies. However, it's crucial to understand the assumptions and limitations of the metrics chosen. Over-reliance on, or optimizing for, an unsuitable metric can lead to undesirable outcomes. While quantitative data is great, qualitative assessments remain invaluable. Sometimes, a group might not meet all quantitative benchmarks, yet its quality and the subsequent impact of its core members can be significant. Therefore, while metrics are helpful, it's essential to balance them with a holistic understanding of a group's value and contributions.