Home Crowd Metrics Understanding unmoderated test metrics

Understanding unmoderated test metrics

Last updated on Jan 31, 2024

Key points;

  • What are metrics

  • Why Metrics matter

  • Key Unmoderated Test Metrics

What Are Metrics

Metrics are the lifeblood of user research and product development. In the context of unmoderated tests, they become invaluable tools for understanding participant behavior, evaluating test success, and making data-driven improvements.

In this guide, we'll delve into the various metrics Crowd collects during unmoderated tests, exploring their significance and providing best practices for utilizing them effectively.

Why Metrics Matter

Metrics serve as the bridge between user research and actionable insights. They offer quantifiable data that empowers you to:

  1. Evaluate Test Success

    Metrics help you assess the overall success of your unmoderated tests. They reveal whether your tests achieve their intended objectives and where improvements are needed.

  2. Understand Participant Behavior

    By tracking participant sessions and behaviors, metrics shed light on how users interact with your tests. This insight helps identify pain points and areas for enhancement.

  3. Optimize User Experience

    With a deep understanding of your participants' experiences, you can refine your user interfaces, content, and features to create a more engaging and user-friendly product.

Key Unmoderated Test Metrics

1. Participant Sessions

  • Definition: Individual user interactions with your unmoderated tests.

  • Significance: Gauges user interest and engagement. Reveals patterns in how users interact with your content.

2. Number of Sessions

  • Definition: A count of how many users have engaged with your unmoderated tests.

  • Significance: Provides an overview of test reach and visibility.

3. Completion Rate

  • Definition: Reflects the percentage of participants who complete your unmoderated tests.

  • Significance: High rates indicate user-friendliness and engagement, while low rates signal issues needing attention.

4. Average Duration

  • Definition: Measures the average time participants spend engaging with your unmoderated tests.

  • Significance: Helps identify if participants are spending an appropriate amount of time or rushing through.

5. Completion vs. Abandonment Rate

  • Definition: Compare the percentage of participants who complete your tests to those who abandon them.

  • Significance: Reveals usability issues or unclear instructions.

6. Completion Time Graph

  • Definition: Visually represents how long participants take to complete your tests.

  • Significance: Pinpoints where users might struggle or encounter bottlenecks.

7. Response by Country

  • Definition: Reveals the geographic distribution of participants.

  • Significance: This metric is crucial if your tests are targeted at specific regions or if you want to ensure a globally inclusive user experience.

8. Response by Source

  • Definition: Response by source tracks where participants originate from, whether it's a specific website, referral link, or other sources.

  • Significance: Understanding the sources of your participants can help you assess the effectiveness of your test promotion strategies.

9. Response by Device

  • Definition: Response by device categorizes participants based on the devices they use, such as desktop, mobile, or tablet.

  • Significance: Vital for optimizing the user experience for different devices.

10. Response by Operating System

  • Definition: Segments participants by the operating systems they use.

  • Significance: Helps tailor tests to specific OSs, ensuring a seamless experience.