Scoring System Specification

Overview

Impact Pulse uses a points-based scoring system to quantify survey responses. Every question type has a defined scoring mechanism that converts qualitative responses into measurable impact metrics. This enables before/after comparisons, trend tracking, and organizational benchmarking.

Related Flows: Take Survey | Survey Responses

Core Concepts

Question Points

• Every scored question has a max_points value (default: 10)
• The survey creator sets max_points per question
• Points represent the maximum achievable score for that question

Scoring Types

Each question has a scoring_type that determines how responses convert to points:

1. direct_scale (for Rating questions) ✅
2. binary (for Yes/No questions) ✅
3. weighted_options (for Single Select / Radio / Dropdown) ✅
4. multi_select_sum (for Multi-Select / Checkboxes) 🔲 Not yet
5. qualitative (for Text / Textarea, excluded from scoring) ✅

Survey Score Calculation

survey_score = (sum of points earned across all scored questions) / (sum of max_points for all scored questions) * 100

Result is a percentage (0-100%). Qualitative questions are excluded from both numerator and denominator.

Scoring Type Details

1. Direct Scale (Rating 0-5)

Used for: Rating questions with a numeric scale

Formula: points = (rating / scale_max) * max_points

Example: Question with max_points = 10, scale 0-5

Configuration:

{
  "scoring_type": "direct_scale",
  "max_points": 10,
  "scale_min": 0,
  "scale_max": 5
}

2. Binary (Yes/No)

Used for: Boolean questions

Formula: positive_answer = max_points, negative_answer = 0

The survey creator specifies which answer is "positive" (scores full points). By default, "Yes" is positive.

Example: "Do you have access to clean drinking water?" max_points = 10

Example with inverted polarity: "Do you experience food insecurity?" max_points = 10

Configuration:

{
  "scoring_type": "binary",
  "max_points": 10,
  "positive_answer": "yes"
}

3. Weighted Options (Single Select)

Used for: Radio buttons, dropdowns where user selects ONE option

Each option has an explicitly assigned point value set by the survey creator. Options are ordered from highest impact to lowest.

Example: "How often do you access program services?" max_points = 10

Configuration:

{
  "scoring_type": "weighted_options",
  "max_points": 10,
  "option_weights": {
    "Weekly": 10,
    "Bi-weekly": 8,
    "Monthly": 6,
    "Quarterly": 3,
    "Rarely": 1,
    "Never": 0
  }
}

Auto-distribute option: If the creator doesn't want to manually assign weights, the system can auto-distribute points evenly across options in order:

• 5 options with max_points = 10: first option = 10, second = 7.5, third = 5, fourth = 2.5, fifth = 0
• Formula: weight = max_points * (n - index) / n where n = number of options, index is 0-based from worst to best

4. Multi-Select Sum (Checkboxes)

Used for: Questions where user selects MULTIPLE options

Each selected option contributes points. Total is capped at max_points.

Two modes:

a) Equal weight mode: Each option worth max_points / total_options. More selections = higher score.

Example: "Which services do you use?" (5 options, max_points = 10) Each option = 2 points. Select 3 = 6 points. Select all 5 = 10 points.

b) Custom weight mode: Each option has a specific point value. Sum is capped at max_points.

Example: "Which health services have you accessed?" max_points = 10

Select primary care + mental health + dental = 8 points.

Configuration (equal weight):

{
  "scoring_type": "multi_select_sum",
  "max_points": 10,
  "mode": "equal_weight",
  "options": ["Primary care", "Mental health", "Dental", "Vision", "Nutrition counseling"]
}

Configuration (custom weight):

{
  "scoring_type": "multi_select_sum",
  "max_points": 10,
  "mode": "custom_weight",
  "option_weights": {
    "Primary care": 3,
    "Mental health": 3,
    "Dental": 2,
    "Vision": 1,
    "Nutrition counseling": 1
  }
}

5. Qualitative (Text / Textarea)

Used for: Free-form text responses

Not scored. Excluded from the survey score calculation entirely. These questions capture context, stories, and qualitative feedback.

Future enhancement: LLM-based sentiment/quality analysis to assign a qualitative score (0-10) based on response depth, sentiment, and relevance.

Configuration:

{
  "scoring_type": "qualitative",
  "max_points": 0
}

Survey Score Calculation Example

A survey with 5 questions:

Score = (8 + 10 + 10 + 6) / (10 + 10 + 10 + 10) * 100 = 34/40 * 100 = 85%

Question 5 is excluded from both numerator and denominator.

Impact Comparison (Pre/Post)

The primary use case for scoring is before/after measurement within a single survey:

1. Baseline: Respondent's first attempt at the survey (e.g., score: 45%)
2. Program intervention happens
3. Latest: Same respondent takes the same survey again (e.g., score: 72%)
4. Impact delta = latest - baseline = +27 percentage points

This works because the questions, order, and scoring rules are identical across attempts. Cross-survey comparison is not supported (different questions, order, and context make it statistically invalid).

See Impact Comparison for the full specification.

Maslow Assessment Scoring

The Maslow Needs Assessment uses the direct_scale scoring type exclusively:

• 5 categories (Physiological, Safety, Love & Belonging, Esteem, Self-Actualization)
• 4 questions per category, each rated 0-5
• Category score = average of question ratings, expressed as percentage
• Overall score = average of all 5 category scores

Question Weight (Advanced)

For surveys where some questions matter more than others, an optional weight_multiplier (default: 1.0) can be applied: ✅

weighted_points = points_earned * weight_multiplier
weighted_max = max_points * weight_multiplier

Example: A critical question about food security might have weight_multiplier = 2.0, doubling its influence on the overall score.

Data Model Changes

Question Model (additions) ✅

scoring_config: {
  scoring_type: "direct_scale" | "binary" | "weighted_options" | "multi_select_sum" | "qualitative"
  max_points: number (default: 10)
  scale_min: number (default: 0, for direct_scale)
  scale_max: number (default: 5, for direct_scale)
  positive_answer: string (for binary, default: "yes")
  option_weights: Record<string, number> (for weighted_options and multi_select_sum)
  mode: "equal_weight" | "custom_weight" (for multi_select_sum)
  weight_multiplier: number (default: 1.0)
}

Survey Response Model (additions)

score: {
  total_points: number
  max_possible_points: number
  percentage: number (0-100)
  question_scores: [
    {
      question_id: string
      points_earned: number
      max_points: number
      scoring_type: string
    }
  ]
}

Survey Model (additions)

scoring_summary: {
  max_possible_score: number (sum of all max_points for scored questions)
  scored_question_count: number
  qualitative_question_count: number
}

Implementation Phases

Phase 1: Core Scoring Engine

• Add scoring_config to question model
• Implement score calculation for all 5 types
• Calculate and store scores on survey submission ✅
• Display score on results page

Phase 2: Pre/Post Comparison ✅

• Link baseline and follow-up surveys
• Calculate impact deltas per question and overall
• Comparison visualization

Phase 3: Organization Analytics 🔲 Not yet

• Aggregate scores across respondents
• Average score per question, per survey, per organization
• Score distributions and percentiles

Phase 4: LLM Qualitative Analysis (Future) 🔲 Not yet

• Send text responses to LLM for sentiment analysis
• Generate qualitative scores (0-10)
• Extract themes and insights from free-text responses