WeHappers Happiness Index Migration Spec

Status (June 2026): The migration script is complete and validated against dev (scripts/migrate_happiness_index.py, idempotent, in-memory + dry-run modes). The production cutover has NOT been run. The data committed to the repo is anonymized (user_<id>@migrate.local, passwords/tokens cleared), so the dev run never touches real users. To retire happiness.wehappers.org we still need to: export the real data, run the migration against production, send a "your account moved" email, and switch DNS. See Production Cutover (Remaining Work) at the end of this doc.

Overview

The original Happiness Index application (deployed at happiness.wehappers.org via Vercel) is the predecessor of Impact Pulse. It contains real user data (257 users, 177 Maslow survey results across 43 countries) collected since August 2024. This spec defines how to migrate that data into Impact Pulse so it becomes a superset of the old system.

Related Specs: Structured Assessments | Maslow Assessment | Database Schema

Goals

Import all historical Maslow survey results into Impact Pulse
Import user accounts so historical data is linked to real users
Preserve geo-location data as survey question responses
Ensure the old happiness.wehappers.org dataset is fully represented inside Impact Pulse
Allow WeHappers to build their own aggregation and visualization on top of the Impact Pulse API

Non-Goals

Replicate the old frontend or UI
Build country-level aggregation endpoints inside Impact Pulse (WeHappers does this on their side via the API)
Migrate the donation/wallet/HITs system (WeHappers-specific, not part of Impact Pulse)
Migrate translations/i18n (separate initiative, coming later)
Migrate Goal2050 or LiveKit features (unrelated)

Source Data

MongoDB Atlas Connection

Cluster: hapiness-index.dptq5gr.mongodb.net
Database: hapiness-index
Credentials: Stored in Vercel project happiness-deploy environment variables

Collections and Counts

Collection	Documents	Migration Action
`users`	257	Import into Impact Pulse `users`
`happiness_index_results`	177	Import into Impact Pulse `survey_responses`
`happers`	9	Skip (WeHappers-specific profile data)
`organizations`	1	Skip (different org model in Impact Pulse)
`campaigns`	1	Skip (donation system, not in scope)
`hits_donations`	5	Skip
`hits_transactions`	2	Skip

Data Dump (Pre-Downloaded)

A complete JSON dump of all collections is saved locally at:

impact-pulse/scripts/migration-data/
├── users.json                    # 257 users (120 KB)
├── happiness_index_results.json  # 177 survey results (1.1 MB)
├── happers.json                  # 9 happers (2.6 KB)
├── organizations.json            # 1 organization (1.9 KB)
├── campaigns.json                # 1 campaign (891 B)
├── hits_donations.json           # 5 donations (2.5 KB)
└── hits_transactions.json        # 2 transactions (1 KB)

The migration script reads from these local files. No MongoDB connection is needed for development/testing.

Email Safety: All emails in the committed data are obfuscated as user_<id>@migrate.local (a non-routable domain). Password hashes and reset tokens are cleared. This ensures no real emails are ever sent during development or testing.

Production migration: For the final production run, the script must be pointed at the live MongoDB to read real emails. The connection string is stored in the Vercel project happiness-deploy environment variables (only Chip has access). The data was originally exported on 2026-04-09.

Source Data Structures

Old User Document

{
  "_id": "ObjectId",
  "email": "string",
  "hashedEmail": "string (SHA-256)",
  "displayName": "string (auto-generated, e.g. 'Xenodochial-Pondering-Gaur-3058')",
  "verified": true,
  "geoLocation": {
    "country": "Canada",
    "region": "Quebec"
  },
  "isHapper": false,
  "providerId": "string (Google OAuth ID, optional)",
  "providers": ["google"],
  "lastLogin": "Date",
  "createdAt": "Date (implicit)"
}

Old Survey Result Document

{
  "_id": "ObjectId",
  "userId": "ObjectId (ref to users._id)",
  "answers": {
    "WATER": 5,
    "FOOD": 4,
    "AIR": 3,
    "SLEEP": 4,
    "SHELTER": 5,
    "CLOTHING": 4,
    "SANITARY": 3,
    "HEALTH": 4,
    "PERSONAL_SECURITY": 5,
    "SOCIAL_STABILITY": 4,
    "EDUCATION": 3,
    "FINANCIAL_STABILITY": 4,
    "FRIENDSHIP": 5,
    "INTIMACY": 4,
    "FAMILY": 5,
    "COMMUNITY": 3,
    "SELF_RESPECT": 4,
    "RECOGNITION": 3,
    "STATUS": 3,
    "ACHIEVEMENT": 4,
    "CREATIVITY": 3,
    "PERSONAL_GROWTH": 4,
    "MEANING_AND_PURPOSE": 5,
    "SELF_FULFILLMENT": 4
  },
  "score": 10000,
  "results": { "levels": [...], "totalScore": ..., "maxPossibleScore": ..., "percentage": ... },
  "userGeoLocation": { "country": "France", "region": "Auvergne-Rh\u00f4ne-Alpes" },
  "info": { "event": "My first HIT", "description": "This is my first Happiness Index Test" },
  "createdAt": "Date"
}

Data Quality Issues

Inconsistent Category Names

The old data has two naming conventions mixed across results:

Old Format (11 results)	New Format (166 results)
`"Water"`	`"WATER"`
`"Personal Security"`	`"PERSONAL_SECURITY"`
`"Self-Fulfillment"`	`"SELF_FULFILLMENT"`
`"Meaning and Purpose"`	`"MEANING_AND_PURPOSE"`
`"Financial Stability"`	`"FINANCIAL_STABILITY"`

Resolution: Normalize all category names to UPPER_SNAKE_CASE before mapping to Impact Pulse question IDs.

Incomplete Answers

Answer Count	Number of Results
24 (complete)	166
0-23 (partial)	11

Resolution: Import partial results as-is. Impact Pulse supports partial responses. Unanswered categories get no score (score 0 for that question).

Target Data Structures

Impact Pulse User

{
  "id": "cuid",
  "email": "string",
  "full_name": "string",          # From old displayName
  "password": "string (hashed)",   # Migrated bcrypt hash or null for Google-only
  "google_id": "string",           # From old providerId
  "email_verified": true,          # From old verified field
  "created_at": "datetime"
}

Impact Pulse Survey Response

{
  "id": "cuid",
  "survey_id": "cuid (the migrated Maslow survey)",
  "user_id": "cuid (mapped from old userId)",
  "respondent_email": "string",
  "responses": {
    "<water_question_id>": 5,
    "<food_question_id>": 4,
    "<country_question_id>": "France",
    "<region_question_id>": "Auvergne-Rh\u00f4ne-Alpes"
  },
  "score": 85.5,
  "score_breakdown": { "<question_id>": 5.0, ... },
  "group_results": [
    {
      "group_id": "vital",
      "group_name": "Vital",
      "color": "red",
      "tint": "700",
      "total_score": 16.0,
      "max_possible_score": 20.0,
      "percentage": 80.0,
      "question_scores": [...]
    }
  ],
  "overall_percentage": 85.5,
  "submitted_at": "datetime (from old createdAt)",
  "completed_at": "datetime (from old createdAt)"
}

Migration Steps

Step 1: Create the WeHappers Organization in Impact Pulse

Create an organization for WeHappers in Impact Pulse using the API:

POST /api/v1/organizations
{
  "name": "WeHappers",
  "description": "Global well-being measurement through the Happiness Index"
}

Step 2: Create the Maslow Survey from Template

Use the existing maslow-needs-v1 template:

POST /api/v1/templates/maslow-needs-v1/create-survey
{
  "title": "Happiness Index Test (HIT)",
  "description": "Maslow Needs Assessment - migrated from happiness.wehappers.org",
  "organization_id": "<wehappers_org_id>"
}

This creates the survey with all 24 questions and 6 groups already configured.

Step 3: Add Country and Region Questions

Add two additional MULTIPLE_CHOICE questions to the survey for geo-location:

POST /api/v1/surveys/<survey_id>/questions
{
  "text": "What country are you in?",
  "type": "MULTIPLE_CHOICE",
  "required": true,
  "options": ["Canada", "France", "Uganda", ...all 43 countries from old data + more],
  "order": 25,
  "scoring_config": null
}

POST /api/v1/surveys/<survey_id>/questions
{
  "text": "What region/province are you in?",
  "type": "TEXT",
  "required": false,
  "order": 26,
  "scoring_config": null
}

Note: The country question options can be populated via CSV upload (see separate spec: CSV Option Upload).

Step 4: Build the Category-to-Question Mapping

After the survey is created, fetch all questions and build a mapping from old category names to new question IDs:

CATEGORY_TO_QUESTION_TEXT = {
    "WATER": "How would you rate your access to water?",
    "FOOD": "How would you rate your access to nutritious food?",
    "AIR": "How would you rate the quality of air you breathe daily?",
    "SLEEP": "How would you rate the quality and quantity of your sleep?",
    "SHELTER": "How would you rate your living conditions and housing stability?",
    "CLOTHING": "How would you rate your access to appropriate clothing?",
    "SANITARY": "How would you rate your access to sanitation and hygiene facilities?",
    "HEALTH": "How would you rate your overall physical health and access to healthcare?",
    "PERSONAL_SECURITY": "How safe do you feel in your daily life?",
    "SOCIAL_STABILITY": "How stable and supportive do you find your social environment?",
    "EDUCATION": "How would you rate your access to and quality of education?",
    "FINANCIAL_STABILITY": "How would you rate your financial security and stability?",
    "FRIENDSHIP": "How satisfied are you with the quality of your friendships?",
    "INTIMACY": "How satisfied are you with the level of intimacy in your life?",
    "FAMILY": "How would you rate your family relationships and support?",
    "COMMUNITY": "How connected do you feel to your community?",
    "SELF_RESPECT": "How would you rate your level of self-respect and self-esteem?",
    "RECOGNITION": "How satisfied are you with the recognition you receive from others?",
    "STATUS": "How satisfied are you with your social or professional status?",
    "ACHIEVEMENT": "How satisfied are you with your personal and professional achievements?",
    "CREATIVITY": "How satisfied are you with your ability to express and apply your creativity?",
    "PERSONAL_GROWTH": "How satisfied are you with your personal growth and development?",
    "MEANING_AND_PURPOSE": "How strong is your sense of meaning and purpose in life?",
    "SELF_FULFILLMENT": "How fulfilled do you feel in reaching your potential?"
}

Also normalize old-format names:

def normalize_category(name: str) -> str:
    """Convert both 'Water' and 'WATER' formats to 'WATER'."""
    return name.upper().replace(" ", "_").replace("-", "_")

Step 5: Import Users

For each user in the old users collection:

Check if user already exists in Impact Pulse (by email)
If not, create with:
- email = old email
- full_name = old displayName
- google_id = old providerId (if present)
- email_verified = old verified field
- password = old password hash (if present, bcrypt is compatible)
Store a mapping of old_id -> new_id for the results import

Step 6: Import Survey Results

For each document in old happiness_index_results:

Look up the new user_id from the old userId using the mapping from Step 5
Normalize all category names in answers to UPPER_SNAKE_CASE
Map each answer to the corresponding Impact Pulse question_id
Add country and region from userGeoLocation as responses to the geo questions
Submit via the Impact Pulse API:

POST /api/v1/surveys/<survey_id>/respond
{
  "respondent_email": "<user_email>",
  "responses": {
    "<water_question_id>": 5,
    "<food_question_id>": 4,
    ...
    "<country_question_id>": "France",
    "<region_question_id>": "Auvergne-Rh\u00f4ne-Alpes"
  }
}

Impact Pulse's scoring engine will automatically compute score, score_breakdown, group_results, and overall_percentage.

After import, update the submitted_at timestamp to match the old createdAt date (to preserve historical ordering).

Step 7: Publish the Survey

After all historical data is imported:

POST /api/v1/surveys/<survey_id>/publish

The survey is now live for new respondents.

Step 8: Validate

Run validation checks:

Total survey_responses count matches (177)
Each response has correct score computation
Users can be looked up by email
Country distribution matches old data (43 countries)
Date range preserved (Aug 2024 to Mar 2026)

Migration Script Location

The migration script should live at:

impact-pulse/scripts/migrate_happiness_index.py

It should:

Read from the local JSON files in scripts/migration-data/ (pre-downloaded, no MongoDB access needed)
Write to Impact Pulse via its REST API (so all validation and scoring runs through the normal path)
Be idempotent (can be re-run without creating duplicates)
Log progress and any errors
Produce a summary report at the end

Countries in the Old Data

43 countries with survey responses:

Algeria, Australia, Austria, Belgium, Bhutan, Brazil, Cameroon, Canada, China, Colombia, Costa Rica, Cote d'Ivoire, Egypt, El Salvador, Finland, France, French Polynesia, Germany, Ghana, India, Japan, Kenya, Luxembourg, Madagascar, Mexico, Morocco, Netherlands, New Zealand, Nigeria, Norway, Peru, Portugal, Senegal, Slovenia, South Africa, Spain, Switzerland, Tunisia, Uganda, United Kingdom, United States, Venezuela, Vietnam

Sequence Diagram

Migration Script                Impact Pulse API              Local JSON Files
      |                               |                           |
      |-------- read users.json ----------------------------------->|
      |<------- 257 users ------------------------------------------|
      |                               |                           |
      |-- POST /organizations ------->|                           |
      |<-- org_id --------------------|                           |
      |                               |                           |
      |-- POST /templates/maslow-needs-v1/create-survey -->|      |
      |<-- survey_id, question_ids ---|                           |
      |                               |                           |
      |-- POST /questions (country) ->|                           |
      |-- POST /questions (region) -->|                           |
      |                               |                           |
      |-- POST /auth/register ------->|  (for each user)         |
      |<-- user_id --------------------|                           |
      |                               |                           |
      |-------- read happiness_index_results.json ----------------->|
      |<------- 177 results ----------------------------------------|
      |                               |                           |
      |-- POST /surveys/{id}/respond ->| (for each result)        |
      |<-- response_id + scores ------|                           |
      |                               |                           |
      |-- POST /surveys/{id}/publish ->|                           |
      |<-- published ------------------|                           |

Open Questions (Resolved in the Script)

These were resolved during implementation. Recorded here as the decisions of record.

User conflicts: Resolved. The script matches by lowercased email and links to the existing user when one is found; otherwise it creates a new user. Re-running is idempotent.
Old event metadata: Resolved. The old info.event / info.description are not carried into survey_responses; the source linkage is preserved instead via _migration_source / _migration_source_id on the migrated user.
Partial results: Resolved. All 177 results are imported, including the 11 partial ones. Unanswered categories simply score 0 for that question.
Password migration: Resolved. Users are created with password: None (no hash carried over). At cutover, email users set a password via the existing reset-password token flow; Google users sign in with Google. See the cutover section below.

Production Cutover (Remaining Work)

Everything below is what still has to happen to actually move happiness.wehappers.org onto Impact Pulse and turn off the old Vercel app. None of it has run yet.

C1. De-anonymized data export (data prep)

Status: Not done. Owner: Chip (only he holds the live Mongo credentials).

The committed scripts/migration-data/*.json is the anonymized dev copy. The production run needs the real export: real email, providerId (Google), displayName, and geoLocation, with password hashes and reset tokens still stripped (Impact Pulse does not reuse old hashes).

Re-export users (257) and happiness_index_results (177) from the live cluster hapiness-index.dptq5gr.mongodb.net (creds in the Vercel happiness-deploy project).
Run the existing copy step with anonymization disabled (the current tooling copies-and-anonymizes; cutover needs a copy-only / no-anonymize mode), or a one-off mongoexport.
Write the real JSON to a gitignored directory (never commit real emails). The migration script reads its data dir from a path/env, so point it at the real export for the production run only.

C2. Account state on import (small code change)

Status: Needs a one-line decision in the script.

Migrated users currently land with email_verified: None and password: None. For cutover, set email_verified: True (the old verified flag was true for these accounts) so users are not bounced into the verification flow. They authenticate by either setting a password (C3) or Google sign-in.

C3. "Your account moved" cutover email (not implemented)

Status: Not done. No welcome/migration email template exists, and the migration script sends no email today.

Template: add a migration/welcome template to backend/app/services/email.py ("The WeHappers Happiness Index has moved to Impact Pulse").
Set-password link: reuse the existing reset-password token mechanism (POST /auth/forgot-password → /reset-password?token=) to generate a per-user set-password link for email accounts. Google-only accounts get a "Sign in with Google" link instead (no token).
Sender: a separate idempotent notify step (a --send-cutover-email flag on the migration script, or a small companion script) that iterates migrated users and sends once. Must support --dry-run and a "send to a short allow-list of test addresses first" guard before the full 257-user blast. Throttle to respect Resend rate limits.
Verify Resend prod config (RESEND_API_KEY, noreply@catalyzeup.ai domain verified) before sending. The dev run logs [DEV EMAIL] instead of sending.

C4. Production migration run + validation

Status: Not done.

Run USE_INMEMORY_DB=false MONGODB_URL=<prod-cosmos> python scripts/migrate_happiness_index.py --dry-run first, read the summary, then run without --dry-run.
Validate against Step 8: 257 users, 177 responses, 43 countries, Aug 2024 to Mar 2026 date range preserved, scores recomputed by the live scoring engine.
Publish the survey.

C5. Domain + auth cutover (infra)