CatalyzeUp
sondely/migration

Data Model: Mongo to Postgres

This page maps every collection the Python backend reads or writes to the Prisma models of the TypeScript app, field by field, and gives the draft Prisma schema. It is the input to the migration script described in the Cutover Runbook and to the tenant isolation work in Framework Copy. The overview is the migration plan.

Sources read: backend/app/models/*.py, every insert_one and update_one under backend/app/api, backend/app/services, backend/app/utils, backend/app/core/database.py, backend/app/core/migrations.py, backend/app/main.py and scripts/migrate_happiness_index.py in CatalyzeUp/sondely (branch develop, commit 06debbf). The collections that actually exist were listed read-only from Azure (az cosmosdb mongodb collection list, account cosmos-catalyzeup) on 5 November 2026. The older Database Schema page is stale (it lists maslow_scores, which no code uses, and embeds questions in surveys, while questions are their own collection); this page replaces it for the migration.

What exists in Cosmos today

Both databases, impactpulse (dev) and impactpulse_prod (prod), hold the same 14 collections. Mongo creates a collection on its first write, so a collection the code names but never writes in that environment does not exist.

Collection In Cosmos Written by Read by Target
users yes api/auth/routes.py, utils/auth.py, api/organizations/routes.py (dev invite path), core/seed.py, WeHappers import everything User
organizations yes api/organizations/routes.py, migrations 0001 everything Organization
organization_members yes organizations, invitations, auth/routes.py (auto-enroll), services/enrollment.py permissions, dashboard, every org route OrganizationMember
organization_invitations no api/organizations/routes.py (production branch of invite) api/invitations/routes.py Invitation (starts empty)
surveys yes api/surveys/routes.py, api/templates/routes.py, migrations 0001 and 0002 everything Survey, QuestionGroup, SurveyTranslation
questions yes surveys, templates, translation routes surveys, scoring, sessions, reminders Question, SurveyTranslation
survey_sessions yes api/sessions/routes.py, services/reminder_engine.py sessions, reminders SurveySession
survey_responses yes api/surveys/routes.py, api/sessions/routes.py, reminders, migrations dashboard, impact, exports, WeHappers map SurveyResponse
survey_assignments yes api/assignments/routes.py, services/enrollment.py, reminders, response submit dashboard, reminders SurveyAssignment
api_keys yes api/api_keys/routes.py, utils/auth.py (usage counters) utils/auth.py ApiKey
otp_codes yes auth/routes.py request-otp verify-otp OtpCode (not migrated)
email_verifications yes register, resend-verification verify-email VerificationToken
oauth_states yes Google and Apple start callbacks OAuthState (not migrated)
schema_migrations yes core/migrations.py core/migrations.py not migrated, checked
sessions yes nobody: only create_index in core/database.py touches it nobody not migrated, asserted empty
notification_preferences no api/notifications/routes.py (created lazily on first GET) reminders, notifications NotificationPreference (starts empty)
password_resets no forgot-password reset-password VerificationToken (nothing to move)
email_reminders no nobody: indexes only, in the dead core/database_enhanced.py nobody EmailReminder, new purpose (below)

Redis (redis-catalyzeup, DB 0 dev and DB 1 prod) holds one thing: the session token minted at sign-in (session:<token>, 24 hour TTL, core/redis.py, auth/routes.py _create_tokens_and_session). It is not migrated; see Session.

Conventions for every model

Rule How
Ids kept The Mongo id field (a cuid, utils/cuid.py) becomes the Postgres primary key, unchanged. Two exceptions: question groups (their ids are slugs reused across surveys, see QuestionGroup) and the collections that never had an id field (email_verifications, notification_preferences, responses written by /sessions/{id}/complete), described per model
Tenant column Every row that belongs to an organization carries organizationId (non-null, indexed first in its compound indexes) and is listed in the copied TENANT_MODELS set, so the tenant extension and the row level security policy cover it. Python stores organization_id on only some of these (members, assignments, api keys, surveys); for questions, sessions, responses and translations the migration derives it from the survey
Not tenant-scoped Organization (it is the tenant), User (global), NotificationPreference (belongs to the person), Session, OtpCode, VerificationToken, OAuthState (resolved before a tenant is bound, as TourCockpit's Session and MagicLinkToken), MigrationQuarantine, MigrationRun (admin only, no grant to the app role)
Timestamps Every DateTime is @db.Timestamptz(3). Python writes datetime.utcnow(), a naive value; BSON stores it as a UTC instant with millisecond precision, so the conversion is exact. TourCockpit's own schema uses plain DateTime (timestamp without zone, 0 uses of Timestamptz in prisma/schema.prisma); Sondely deliberately diverges here
Strings stored as ISO text Older imported documents may hold dates as strings (core/migrations.py _ts_key handles both). The script parses Z, +00:00 and naive ISO strings as UTC; an unparsable value is quarantined
Email Stored lower-cased and trimmed, @unique, plus a SQL CHECK (email = lower(btrim(email))) added in the migration file. No citext: it would need the extension allow-listed on the flexible server, and every write path in Python already lower-cases (auth/routes.py lines 133, 312, 541, 690, 779)
Column names camelCase columns, snake_case table names through @@map, as TourCockpit. The API layer maps back to today's snake_case JSON so the bearer contract does not change at cutover
Enums Python stores upper-case strings for roles, survey status, question type, assignment status; lower-case for result_type, session status and translation status. Prisma enums are upper-case; the API serializer emits the original case
Legacy fields _migration_source_id, _migration_source, _migration_metadata (WeHappers import, scripts/migrate_happiness_index.py) go to a legacy Json? column, kept for traceability, never read by the app

JSON columns and why

A column is JSON only when its shape is a document the product reads whole and never filters on, or when it is keyed by ids that live elsewhere. Each one has a Zod schema at the write boundary; rows migrated from Mongo are stored verbatim and validated as a report (see the runbook).

Model.column Shape Why JSON
Organization.settings {allow_member_invites, require_email_verification, default_reminder_schedule, timezone, locale} Read whole on the settings screen, never queried
Organization.profileConfig {email, geo, name}: {enabled, required} Read whole by the take page; keyed by profile field
Survey.settings free-form dict today (Dict[str, Any]) No known reader filters on it; typed later
Question.ratingLabels {"1": "Poor", ...} Keyed by rating value
Question.scoringConfig {type, min_value, max_value, true_value, false_value, option_weights} A discriminated union by type; read whole by the scoring engine
SurveyTranslation.content {survey: {title, description}, groups: {<groupKey>: {name, description}}, questions: {<questionId>: {text, options, rating_labels}}} Keyed by group key and question id; read whole per language
SurveySession.answers {<questionId>: value} Keyed by question id; parity with today (decision 5 on the overview)
SurveyResponse.answers {<questionId>: value} Same
SurveyResponse.scoreBreakdown {<questionId>: number} Keyed by question id, written by the scoring engine
SurveyResponse.groupResults [{group_id, group_name, color, tint, total_score, max_possible_score, percentage, question_scores[]}] A computed snapshot at submit time; kept verbatim because groups and colours can change after submission
Audit.beforeSummary, afterSummary as TourCockpit Copied framework
MigrationQuarantine.document, detail the raw source document in canonical JSON Must hold anything, including what failed

Points are columns, not JSON: score, overallPercentage, geoCountry, geoRegion, contextEvent, contextDescription, so the impact screens, the map and later insight tools can filter and aggregate on them.

Collection by collection

User

From users. Insert sites: auth/routes.py register (133), Google callback (565), Apple callback (715), auto-enroll (776), utils/auth.py create_user, the dev-only invite branch of organizations/routes.py (436), core/seed.py, the WeHappers import.

Mongo field Prisma Notes
id id kept
email email lower-cased; collisions handled in the runbook
password passwordHash bcrypt; the Auth page rehashes to argon2 on first sign-in
password_hash, is_active, is_verified, _id dropped, reported written only by the ENVIRONMENT == "development" invite branch (organizations/routes.py 436 to 448). dev.sondely.com runs with ENVIRONMENT=development (azure/main.tf), so the dev database has these documents; prod does not (inferred from the env, to be confirmed by rehearsal counts)
full_name fullName
locale locale null means no preference (models/user.py comment)
google_id googleId @unique; a duplicate keeps the oldest user and the other row is quarantined as a warning
apple_id appleId @unique, same rule. Apple is not configured in production today: no APPLE_* variable on impactpulse-prod (read from Azure)
email_only emailOnly absent means false
email_verified emailVerifiedAt a datetime in Python, despite the name. Imported accounts store it as a boolean in production (223 of 267 users, found by the production dry run on 8 November 2026): true maps to the account's creation time, false to null
image image
created_at, updated_at same
_migration_* legacy WeHappers import

Organization

From organizations. Model: models/organization_enhanced.py (the only organization model imported by routes). Insert: organizations/routes.py 88.

Mongo field Prisma Notes
id id kept
name name Python refuses a duplicate name on create only (organizations/routes.py 61); indexed, not unique, duplicates reported
description, website, industry, size same
custom_domain customDomain @unique, nullable (Mongo has a sparse unique index). Production holds hit.wehappers.org as a bound domain on impactpulse-prod
settings settings (JSON)
profile_config profileConfig (JSON) default from default_profile_config() when absent
stats.member_count, survey_count, response_count dropped Denormalized counters. survey_count and response_count are written once at creation (0) and never incremented by the API (only the WeHappers startup branch sets them), so they are wrong today. The TS API computes them with counts; the read-by-read diff normalizes this field and lists every organization whose stored counter differed
created_by_id createdById no foreign key: the creator can leave
created_at, updated_at same

OrganizationMember

From organization_members. Inserts in five places (organizations/routes.py 100, 422, 458; invitations/routes.py 106; auth/routes.py 798; services/enrollment.py 27; main.py WeHappers branch).

Mongo field Prisma Notes
id id kept
organization_id organizationId orphan (org deleted) is quarantined: organizations/routes.py 310 deletes the org, its members and invitations, but not its surveys, keys or responses
user_id userId orphan quarantined
role role (MemberRole) values in use: OWNER, ADMIN, RESPONDENT. The lower-case owner/admin/member in organization_enhanced.py is never stored. A stored MEMBER, from that unused model, migrates as RESPONDENT, since Python grants it nothing more (built 8 November 2026). Any other value is quarantined
invited_by_id invitedById
joined_at joinedAt

Unique (organizationId, userId). Python has no such index, and five code paths check then insert, so duplicates are possible. The script keeps one row per pair (highest role, then earliest joinedAt) and quarantines the others with the kept id in detail.

Invitation

From organization_invitations (absent in both databases today). Insert: organizations/routes.py 479: id, organization_id, email, role, token, invited_by_id, created_at, expires_at, status. Accept and decline set accepted_at, declined_at.

token is a raw cuid today; the new row stores tokenHash (sha-256), so a migrated invitation link keeps working. The stored status: "PENDING" is never updated (acceptance writes accepted_at only, invitations/routes.py), so it is dropped and the status is derived, as OrganizationInvitation.compute_status does.

Survey

From surveys. Inserts: surveys/routes.py 76, templates/routes.py 101, seed, WeHappers import. Updates: survey update, translation routes, migrations 0001 and 0002, the WeHappers branch (status).

Mongo field Prisma Notes
id, public_id id, publicId both kept; publicId @unique (Mongo unique index). Survey links (hit.wehappers.org/survey/c19ea72df1datmhjhs1i, in the WeHappers site) use it
organization_id organizationId Optional in SurveyInDB. A survey with none, or with a deleted organization, is quarantined
title, description same
status status DRAFT, PUBLISHED, CLOSED
is_active isActive
settings settings (JSON)
min_days_between_attempts, auto_remind, require_email same, camelCase migration 0002 normalized these on orgs with a profile config
result_type resultType stored lower-case (flat, pyramid, radar, breakdown)
question_groups[] QuestionGroup rows see below
template_id templateId a template slug such as maslow-needs-v1, not a foreign key
recurrence_enabled, recurrence_interval_days, reminder_intervals same; reminderIntervals Int[] default [0, 3, 7]
translations, translation_metadata SurveyTranslation rows see below
available_languages, default_language availableLanguages String[], defaultLanguage
created_by_id, created_at, updated_at same

QuestionGroup

Embedded in surveys.question_groups as {id, name, order, color, tint, description} (models/survey.py QuestionGroup). The id is not unique across surveys: template groups use fixed slugs (vital, physiological, safety_development, ... in data/survey_templates.py), copied into every survey built from the template. So:

  • QuestionGroup.id is new, derived from (surveyId, legacy group id) by a hash (for example qg_ plus the first 24 hex characters of a sha-256), so a re-run produces the same id and the upsert stays idempotent.
  • QuestionGroup.key holds the legacy group id; unique (surveyId, key).
  • The API keeps exposing key as group_id, and the JSON keyed by group (groupResults[].group_id, SurveyTranslation.content.groups) keeps the key. Nothing a client holds changes.
  • color is stored as written (a CSS name or hex, normalize_group_color); tint, description, order as is.

A survey whose groups contain the same id twice, or a question whose group_id names no group of its survey, is quarantined as a warning; the question is migrated with groupId null.

Question

From questions (their own collection, indexed on survey_id and (survey_id, order)). Inserts: surveys/routes.py 98 and 446, templates/routes.py 121.

Mongo field Prisma Notes
id id kept
survey_id surveyId, plus organizationId from the survey a question whose survey is gone, or whose survey belongs to a deleted organization, is quarantined
text, required, order, weight same weight default 1.0
type type includes COUNTRY, which migration 0001 deletes from surveys but the WeHappers map still reads as a fallback
options options String[] always a list of strings; the overview listed it as JSON, a text array is the tighter type
rating_labels ratingLabels (JSON)
scoring_config scoringConfig (JSON)
group_id groupId (foreign key to QuestionGroup.id, resolved through the key)
translations SurveyTranslation.content.questions
created_at, updated_at same

SurveyTranslation

Built from three places for each language L: surveys.translations[L] ({title, description, groups}), surveys.translation_metadata[L] ({status, translated_at, approved_at, approved_by}) and every questions.translations[L] ({text, options, rating_labels}) of the survey. One row per (surveyId, language), unique. A question translation for a language the survey has no entry for still creates the row (status from metadata, else DRAFT) and is reported.

SurveySession

From survey_sessions. Model: models/survey_session.py (the live one). Insert: sessions/routes.py 111, which writes both id and _id with the same cuid.

Mongo field Prisma Notes
id (or _id when id is missing) id kept
survey_id, user_id surveyId, userId organizationId derived from the survey: the insert never sets organization_id (absent from SurveySessionCreate)
attempt_number, progress_percentage, time_spent_seconds, current_question_index same
status status stored in_progress, completed, abandoned
responses answers (JSON)
started_at, last_activity_at, completed_at same
incomplete_reminders_sent, last_incomplete_reminder_at same written by reminder_engine.py 317
(new) channel WEB for every migrated row

Python treats one in_progress session per (survey, user) as the rule (sessions/routes.py 64) without an index. The partial unique index enforcing it is created only if the rehearsal finds no duplicates; otherwise the duplicates are listed for a decision before cutover.

SurveyResponse

From survey_responses. Two document shapes exist in the code:

Shape Written by Fields
A, the real response surveys/routes.py 612 (public submit), seed, WeHappers import id, survey_id, respondent_email, respondent_name, geo, context, user_id, responses, score, score_breakdown, group_results, overall_percentage, completed_at, submitted_at, next_eligible_date, recurrence_reminders_sent, last_recurrence_reminder_at, later recurrence_status (reminder_engine.py 147)
B, the session copy sessions/routes.py 222 complete_session, "for backward compatibility" _id (a cuid), no id, survey_id, user_id, session_id, attempt_number, responses, submitted_at, organization_id (always null: the session never had one), no score

Shape B documents are not readable by the Python API as responses (SafeCollection strips _id and SurveyResponse requires id; inferred, confirmed in rehearsal 1). The script migrates them with id = _id, scores null, sessionId and attemptNumber set, and records each in quarantine with severity NOTE, so their count is visible before anyone decides whether they are kept.

Mongo field Prisma Notes
id (_id for shape B) id kept
survey_id surveyId, organizationId derived
user_id userId nullable: anonymous respondents
respondent_email respondentEmail not lower-cased: Python stores it as typed and the cooldown check compares it exactly (surveys/routes.py 553). Kept verbatim for parity
respondent_name respondentName
geo.country, geo.region geoCountry, geoRegion any other key in geo goes to legacy and is reported
context.event, context.description contextEvent, contextDescription restored by migration 0004 for WeHappers
responses answers (JSON) stored verbatim; see "answers that fail the new schemas" in the runbook
score, overall_percentage score, overallPercentage (Float)
score_breakdown, group_results JSON
completed_at, submitted_at, next_eligible_date same
recurrence_reminders_sent, last_recurrence_reminder_at, recurrence_status same reminder state lives on the response
session_id, attempt_number sessionId, attemptNumber shape B only
_migration_* legacy WeHappers

SurveyAssignment

From survey_assignments. Inserts: assignments/routes.py 92, services/enrollment.py 79, seed.

member_id is a user id, not a membership id (assignments/routes.py checks organization_members with user_id: member_id), so it becomes userId. assigned_by is a user id or the literal "system" (enrollment); it becomes assignedById (null for system), and the serializer writes "system" back. Status values: PENDING, COMPLETED (surveys/routes.py 622), OVERDUE (reminder_engine.py 92). Unique (surveyId, userId), which Python checks before insert but does not index; duplicates handled as for members.

EmailReminder

The email_reminders collection was designed (models/reminder.py, core/database_enhanced.py) but never written; both are dead code and the collection does not exist. Reminder state today lives on assignments, responses and sessions (counters and last-sent timestamps). The EmailReminder table is kept with a new, narrower job: one row per reminder email sent or attempted, unique on (kind, targetId, number). The reminder job, which replaces the 60 second reminder_loop that runs on every replica, inserts the row before sending, so two replicas or a retried tick cannot send the same reminder twice. That is the record of intent the idempotency rule asks for, and it gives support a history. The counters on assignments, responses and sessions stay, because the API returns them.

NotificationPreference

From notification_preferences (absent in both databases: created on first GET /notifications/preferences, never called in production so far). Fields: user_id, unsubscribe_token, email_reminders_enabled, reminder_frequency, survey_reminders, organization_updates, quiet_hours_enabled, quiet_hours_start, quiet_hours_end, timezone, muted_survey_ids, created_at, updated_at (api/notifications/routes.py). No id field: the row gets a new cuid and is unique on userId. Per person, not per organization. unsubscribeToken becomes unsubscribeTokenHash so links already emailed keep working. mutedSurveyIds String[].

ApiKey

From api_keys. Insert: api_keys/routes.py 74, seed, the WeHappers startup branch. Fields: id, organization_id, name, key_prefix (first 12 characters, ip_live_ plus 4), key_hash (bcrypt of the full key), last_four, created_by_id, created_at, last_used_at, request_count, expires_at, revoked_at.

Change Why
legacyBcryptHash kept, hashedKey (sha-256, @unique) added for new keys A bcrypt hash cannot be looked up; old keys resolve by keyPrefix then bcrypt, as Python does (utils/auth.py _resolve_api_key). Whether old keys keep working is decision 2 on the overview
scopes ApiKeyScope[] Migrated keys get FULL, the behaviour they have today, unless decision 2 says otherwise
organizationId enforced Today a key acts as its creator in every org; the new resolution binds it to its own

One key matters at cutover: the WeHappers site (wehappers.org/src/app/api/happiness-map/route.ts) calls https://impactpulse.catalyzeup.ai/api/v1/surveys and /surveys/{id}/responses with IMPACTPULSE_API_KEY, falling back to the hard-coded ip_live_wehappers_dev_key_00001. The wehappers container app sets no environment variable (read from Azure), so the hard-coded key is the one in use, and it authenticates against production today (HTTP 200 on 5 November 2026). See the WeHappers section of the Cutover Runbook.

Session, OtpCode, VerificationToken, OAuthState

Source Target Migrated
Redis session:<token> Session (tokenHash, userId, expiresAt, lastSeenAt, activeOrganizationId), as TourCockpit's No. 24 hour TTL, and Redis is not part of the snapshot. A person signed in with a session token signs in again; a 30 minute JWT may still be honoured, which is the Auth page's call
otp_codes (id, email, code_hash, expires_at, used_at, attempts, created_at) OtpCode No. Codes live 10 minutes and the write freeze stops new ones
email_verifications (user_id, token, expires_at, no id) VerificationToken kind EMAIL_VERIFY, tokenHash = sha-256 of the raw token Yes, unexpired rows only, so a verification link sent the day before still works
password_resets VerificationToken kind PASSWORD_RESET Nothing to move (collection absent)
oauth_states (state, expires_at, return_origin, no id) OAuthState No. 15 minute lifetime

Collections that are not migrated

Collection What the cutover does with it
schema_migrations Checked, not moved: the go/no-go requires all five ids (0001_decouple_geo to 0005_assign_tenant_surveys) applied in the source database, because the TS app has no equivalent of these one-off Python migrations
sessions Asserted empty; any document is quarantined

Dead and duplicate models

Found by grepping every import of app.models.<name> and app.core.<name> outside the module itself.

File Status Imported by Fate
models/survey_session.py live api/sessions/routes.py, tests/test_drip_campaigns.py Source for SurveySession
models/session.py dead duplicate of the above (different fields, SurveyAttempt, UserSessionHistory) tests/unit/test_session_model.py only Not ported. Deleted with backend/
models/organization_enhanced.py live (despite the name) api/organizations/routes.py Source for Organization, OrganizationMember, Invitation
models/organization.py dead (adds a Contact model never stored) nobody Not ported
models/survey.py live surveys, sessions routes Source for Survey, Question, SurveyResponse
models/survey_enhanced.py dead (QuestionValidation, QuestionLogic, SurveySettings, SurveyStats) nobody Not ported. Its validation and logic ideas belong to the agent phase if anywhere
models/user.py live 15 modules Source for User
models/user_enhanced.py dead (UserPreferences, refresh tokens) nobody Not ported
models/reminder.py dead survey_enhanced.py (dead), tests/unit/test_reminder_model.py Not ported; EmailReminder is redefined above
models/campaign.py dead (CampaignStatus enum in enums.py too) nobody Not ported
models/base.py, base_simple.py dead fix_pydantic_models.py (a one-off script) Not ported
models/responses.py live: API response shapes only, no storage nine route modules Read by API Routes for the contract, not by this page
models/api_key.py, assignment.py, enums.py live Sources above
core/database_enhanced.py dead (a second connection manager, reminder indexes) nobody Not ported
core/inmemory_db.py test and local only (USE_INMEMORY_DB, default true) core/database.py Replaced by the Postgres test database, as TourCockpit's CI
core/seed.py dev seed, runs when ENVIRONMENT is development or test and the hostname is known main.py Rewritten as a Prisma seed for local and CI only, never on dev.sondely.com

Draft Prisma schema

The framework models (Session, IdempotencyKey, Audit) follow TourCockpit at f952d0eea with companyId renamed organizationId and User moved out of the tenant set. Row level security policies (organizationId = current_setting('app.tenant_id'), as TourCockpit's setTenantGuc in src/lib/db.ts) and the email CHECK live in the migration SQL, not in the schema file. This is a draft: API Routes and Auth may add fields, and the rehearsal findings may change uniqueness rules. The schema as built, copied from TourCockpit at 74e9a2a53 on 7 November 2026, is prisma/schema.prisma in CatalyzeUp/sondely-platform, and where it differs from this draft it is the one the migration script and the app follow.

// Sondely: draft schema for the move from Cosmos (Mongo API) to Postgres.
// Docs: /sondely/migration/data-model
// Framework models copied from ideaplaces-tour-platform@f952d0eea, companyId -> organizationId.

generator client {
  provider      = "prisma-client-js"
  binaryTargets = ["native", "linux-musl-openssl-3.0.x"]
}

datasource db {
  provider = "postgresql"
  url      = env("DATABASE_URL")
}

// ---------------------------------------------------------------------------
// Enums. Upper-case here; the API serializer emits today's casing.
// ---------------------------------------------------------------------------

enum MemberRole {
  OWNER
  ADMIN
  RESPONDENT
}

enum SurveyStatus {
  DRAFT
  PUBLISHED
  CLOSED
}

enum ResultType {
  FLAT
  PYRAMID
  RADAR
  BREAKDOWN
}

enum QuestionType {
  BOOLEAN
  TEXT
  SINGLE_CHOICE
  MULTIPLE_CHOICE
  RATING
  COUNTRY // legacy WeHappers geo question, removed from surveys by Python migration 0001
}

enum TranslationStatus {
  PENDING
  DRAFT
  APPROVED
}

enum SessionStatus {
  IN_PROGRESS
  COMPLETED
  ABANDONED
}

enum SessionChannel {
  WEB
  API
  WHATSAPP
  VOICE
}

enum AssignmentStatus {
  PENDING
  COMPLETED
  OVERDUE
}

enum ReminderKind {
  ASSIGNMENT
  RECURRENCE
  INCOMPLETE_SESSION
}

enum ReminderState {
  SENDING
  SENT
  FAILED
}

enum ApiKeyScope {
  READ
  DRAFT
  FULL
}

enum TokenKind {
  EMAIL_VERIFY
  PASSWORD_RESET
}

enum AuditSource {
  SEMANTIC
  ROW
}

enum QuarantineSeverity {
  ERROR // row NOT migrated
  WARNING // row migrated, something about it needs a decision
  NOTE // row migrated, recorded for visibility
}

// ---------------------------------------------------------------------------
// Tenant root and people
// ---------------------------------------------------------------------------

model Organization {
  id            String   @id
  name          String
  description   String?
  website       String?
  industry      String?
  size          String?
  /// Unique when set (Mongo: sparse unique). Resolved by Next middleware.
  customDomain  String?  @unique
  settings      Json     @default("{}")
  profileConfig Json
  createdById   String?
  createdAt     DateTime @default(now()) @db.Timestamptz(3)
  updatedAt     DateTime @updatedAt @db.Timestamptz(3)

  members       OrganizationMember[]
  invitations   Invitation[]
  surveys       Survey[]
  questionGroups QuestionGroup[]
  questions     Question[]
  translations  SurveyTranslation[]
  sessions      SurveySession[]
  responses     SurveyResponse[]
  assignments   SurveyAssignment[]
  reminders     EmailReminder[]
  apiKeys       ApiKey[]
  idempotency   IdempotencyKey[]
  audits        Audit[]

  @@index([name])
  @@map("organizations")
}

/// Global: one row per person across every organization. NOT in TENANT_MODELS.
model User {
  id              String    @id
  /// Lower-cased and trimmed; CHECK (email = lower(btrim(email))) in the migration SQL.
  email           String    @unique
  passwordHash    String?
  fullName        String?
  locale          String?
  googleId        String?   @unique
  appleId         String?   @unique
  emailOnly       Boolean   @default(false)
  emailVerifiedAt DateTime? @db.Timestamptz(3)
  image           String?
  /// _migration_source_id, _migration_source, _migration_metadata (WeHappers import).
  legacy          Json?
  createdAt       DateTime  @default(now()) @db.Timestamptz(3)
  updatedAt       DateTime  @updatedAt @db.Timestamptz(3)

  memberships        OrganizationMember[]
  sessions           Session[]
  verificationTokens VerificationToken[]
  surveySessions     SurveySession[]
  responses          SurveyResponse[]
  assignments        SurveyAssignment[]
  notificationPrefs  NotificationPreference?

  @@map("users")
}

model OrganizationMember {
  id             String       @id
  organizationId String
  organization   Organization @relation(fields: [organizationId], references: [id], onDelete: Cascade)
  userId         String
  user           User         @relation(fields: [userId], references: [id], onDelete: Cascade)
  role           MemberRole
  invitedById    String?
  joinedAt       DateTime     @default(now()) @db.Timestamptz(3)
  updatedAt      DateTime     @updatedAt @db.Timestamptz(3)

  @@unique([organizationId, userId])
  @@index([userId])
  @@map("organization_members")
}

model Invitation {
  id             String       @id @default(cuid())
  organizationId String
  organization   Organization @relation(fields: [organizationId], references: [id], onDelete: Cascade)
  email          String
  role           MemberRole
  message        String?
  /// sha-256 of the token in the emailed link.
  tokenHash      String       @unique
  invitedById    String
  createdAt      DateTime     @default(now()) @db.Timestamptz(3)
  expiresAt      DateTime     @db.Timestamptz(3)
  acceptedAt     DateTime?    @db.Timestamptz(3)
  declinedAt     DateTime?    @db.Timestamptz(3)

  @@index([organizationId, email])
  @@map("invitations")
}

// ---------------------------------------------------------------------------
// Surveys
// ---------------------------------------------------------------------------

model Survey {
  id                     String       @id
  organizationId         String
  organization           Organization @relation(fields: [organizationId], references: [id], onDelete: Cascade)
  publicId               String       @unique
  title                  String
  description            String?
  status                 SurveyStatus @default(DRAFT)
  isActive               Boolean      @default(true)
  settings               Json         @default("{}")
  minDaysBetweenAttempts Int          @default(0)
  autoRemind             Boolean      @default(true)
  requireEmail           Boolean      @default(true)
  resultType             ResultType   @default(FLAT)
  /// A template slug such as "maslow-needs-v1"; not a foreign key.
  templateId             String?
  recurrenceEnabled      Boolean      @default(false)
  recurrenceIntervalDays Int?
  reminderIntervals      Int[]        @default([0, 3, 7])
  availableLanguages     String[]     @default(["en"])
  defaultLanguage        String       @default("en")
  createdById            String
  createdAt              DateTime     @default(now()) @db.Timestamptz(3)
  updatedAt              DateTime     @updatedAt @db.Timestamptz(3)

  groups       QuestionGroup[]
  questions    Question[]
  translations SurveyTranslation[]
  sessions     SurveySession[]
  responses    SurveyResponse[]
  assignments  SurveyAssignment[]

  @@index([organizationId, createdAt])
  @@index([createdById])
  @@map("surveys")
}

/// Was embedded in surveys.question_groups. `key` is the legacy group id
/// (a slug such as "vital", reused across surveys), unique per survey.
model QuestionGroup {
  id             String       @id
  organizationId String
  organization   Organization @relation(fields: [organizationId], references: [id], onDelete: Cascade)
  surveyId       String
  survey         Survey       @relation(fields: [surveyId], references: [id], onDelete: Cascade)
  key            String
  name           String
  order          Int
  color          String
  tint           String?
  description    String?

  questions Question[]

  @@unique([surveyId, key])
  @@index([organizationId, surveyId, order])
  @@map("question_groups")
}

model Question {
  id             String         @id
  organizationId String
  organization   Organization   @relation(fields: [organizationId], references: [id], onDelete: Cascade)
  surveyId       String
  survey         Survey         @relation(fields: [surveyId], references: [id], onDelete: Cascade)
  groupId        String?
  group          QuestionGroup? @relation(fields: [groupId], references: [id], onDelete: SetNull)
  text           String
  type           QuestionType
  required       Boolean        @default(false)
  order          Int
  weight         Float          @default(1.0)
  options        String[]       @default([])
  ratingLabels   Json?
  scoringConfig  Json?
  createdAt      DateTime       @default(now()) @db.Timestamptz(3)
  updatedAt      DateTime       @updatedAt @db.Timestamptz(3)

  @@index([organizationId, surveyId, order])
  @@map("questions")
}

/// One row per survey and language. content = {survey, groups, questions}.
model SurveyTranslation {
  id             String            @id @default(cuid())
  organizationId String
  organization   Organization      @relation(fields: [organizationId], references: [id], onDelete: Cascade)
  surveyId       String
  survey         Survey            @relation(fields: [surveyId], references: [id], onDelete: Cascade)
  language       String
  status         TranslationStatus @default(DRAFT)
  content        Json
  translatedAt   DateTime?         @db.Timestamptz(3)
  approvedAt     DateTime?         @db.Timestamptz(3)
  approvedById   String?
  createdAt      DateTime          @default(now()) @db.Timestamptz(3)
  updatedAt      DateTime          @updatedAt @db.Timestamptz(3)

  @@unique([surveyId, language])
  @@index([organizationId])
  @@map("survey_translations")
}

// ---------------------------------------------------------------------------
// Taking a survey
// ---------------------------------------------------------------------------

model SurveySession {
  id                       String         @id
  organizationId           String
  organization             Organization   @relation(fields: [organizationId], references: [id], onDelete: Cascade)
  surveyId                 String
  survey                   Survey         @relation(fields: [surveyId], references: [id], onDelete: Cascade)
  userId                   String
  user                     User           @relation(fields: [userId], references: [id], onDelete: Cascade)
  channel                  SessionChannel @default(WEB)
  attemptNumber            Int            @default(1)
  status                   SessionStatus  @default(IN_PROGRESS)
  progressPercentage       Float          @default(0)
  answers                  Json           @default("{}")
  timeSpentSeconds         Int            @default(0)
  currentQuestionIndex     Int            @default(0)
  startedAt                DateTime       @default(now()) @db.Timestamptz(3)
  lastActivityAt           DateTime       @default(now()) @db.Timestamptz(3)
  completedAt              DateTime?      @db.Timestamptz(3)
  incompleteRemindersSent  Int            @default(0)
  lastIncompleteReminderAt DateTime?      @db.Timestamptz(3)

  responses SurveyResponse[]

  @@index([organizationId, surveyId, userId, status])
  @@index([organizationId, lastActivityAt])
  @@map("survey_sessions")
}

model SurveyResponse {
  id                       String         @id
  organizationId           String
  organization             Organization   @relation(fields: [organizationId], references: [id], onDelete: Cascade)
  surveyId                 String
  survey                   Survey         @relation(fields: [surveyId], references: [id], onDelete: Cascade)
  userId                   String?
  user                     User?          @relation(fields: [userId], references: [id], onDelete: SetNull)
  sessionId                String?
  session                  SurveySession? @relation(fields: [sessionId], references: [id], onDelete: SetNull)
  attemptNumber            Int?
  /// As typed by the respondent; not normalized (parity with the cooldown check).
  respondentEmail          String?
  respondentName           String?
  geoCountry               String?
  geoRegion                String?
  contextEvent             String?
  contextDescription       String?
  answers                  Json           @default("{}")
  score                    Float?
  overallPercentage        Float?
  scoreBreakdown           Json?
  groupResults             Json?
  completedAt              DateTime?      @db.Timestamptz(3)
  submittedAt              DateTime       @db.Timestamptz(3)
  nextEligibleDate         DateTime?      @db.Timestamptz(3)
  recurrenceRemindersSent  Int            @default(0)
  lastRecurrenceReminderAt DateTime?      @db.Timestamptz(3)
  recurrenceStatus         String?
  legacy                   Json?

  @@index([organizationId, surveyId, submittedAt])
  @@index([organizationId, userId])
  @@index([surveyId, respondentEmail])
  @@index([recurrenceRemindersSent])
  @@map("survey_responses")
}

model SurveyAssignment {
  id                String           @id
  organizationId    String
  organization      Organization     @relation(fields: [organizationId], references: [id], onDelete: Cascade)
  surveyId          String
  survey            Survey           @relation(fields: [surveyId], references: [id], onDelete: Cascade)
  /// Python's member_id: a USER id.
  userId            String
  user              User             @relation(fields: [userId], references: [id], onDelete: Cascade)
  /// Null means "system" (tenant enrollment).
  assignedById      String?
  assignedAt        DateTime         @default(now()) @db.Timestamptz(3)
  status            AssignmentStatus @default(PENDING)
  lastCompletedAt   DateTime?        @db.Timestamptz(3)
  currentDueDate    DateTime?        @db.Timestamptz(3)
  nextDueDate       DateTime?        @db.Timestamptz(3)
  remindersSent     Int              @default(0)
  lastReminderAt    DateTime?        @db.Timestamptz(3)

  @@unique([surveyId, userId])
  @@index([organizationId, assignedAt])
  @@index([organizationId, userId])
  @@map("survey_assignments")
}

/// One row per reminder email, written BEFORE the send (record of intent),
/// so a second replica or a retried tick cannot send it twice.
model EmailReminder {
  id             String        @id @default(cuid())
  organizationId String
  organization   Organization  @relation(fields: [organizationId], references: [id], onDelete: Cascade)
  kind           ReminderKind
  /// The assignment, response or session the reminder is about.
  targetId       String
  number         Int
  recipientEmail String
  state          ReminderState @default(SENDING)
  providerId     String?
  lastError      String?
  createdAt      DateTime      @default(now()) @db.Timestamptz(3)
  sentAt         DateTime?     @db.Timestamptz(3)

  @@unique([kind, targetId, number])
  @@index([organizationId, createdAt])
  @@map("email_reminders")
}

/// Per person, not per organization. NOT in TENANT_MODELS.
model NotificationPreference {
  id                    String   @id @default(cuid())
  userId                String   @unique
  user                  User     @relation(fields: [userId], references: [id], onDelete: Cascade)
  unsubscribeTokenHash  String   @unique
  emailRemindersEnabled Boolean  @default(true)
  reminderFrequency     String   @default("default")
  surveyReminders       Boolean  @default(true)
  organizationUpdates   Boolean  @default(true)
  quietHoursEnabled     Boolean  @default(false)
  quietHoursStart       String?
  quietHoursEnd         String?
  timezone              String   @default("UTC")
  mutedSurveyIds        String[] @default([])
  createdAt             DateTime @default(now()) @db.Timestamptz(3)
  updatedAt             DateTime @updatedAt @db.Timestamptz(3)

  @@map("notification_preferences")
}

// ---------------------------------------------------------------------------
// Credentials
// ---------------------------------------------------------------------------

model ApiKey {
  id               String        @id
  organizationId   String
  organization     Organization  @relation(fields: [organizationId], references: [id], onDelete: Cascade)
  name             String
  /// "ip_live_" plus 4 characters: the lookup for legacy bcrypt keys.
  keyPrefix        String
  lastFour         String
  /// bcrypt of the full key, for keys minted by Python. Null for new keys.
  legacyBcryptHash String?
  /// sha-256 of the full key, for keys minted by the TS app.
  hashedKey        String?       @unique
  scopes           ApiKeyScope[] @default([FULL])
  createdById      String
  createdAt        DateTime      @default(now()) @db.Timestamptz(3)
  lastUsedAt       DateTime?     @db.Timestamptz(3)
  requestCount     Int           @default(0)
  expiresAt        DateTime?     @db.Timestamptz(3)
  revokedAt        DateTime?     @db.Timestamptz(3)

  @@index([organizationId])
  @@index([keyPrefix])
  @@map("api_keys")
}

/// Resolved before a tenant is bound. NOT in TENANT_MODELS (as TourCockpit).
model Session {
  id                   String    @id @default(cuid())
  userId               String
  user                 User      @relation(fields: [userId], references: [id], onDelete: Cascade)
  tokenHash            String    @unique
  expiresAt            DateTime  @db.Timestamptz(3)
  createdAt            DateTime  @default(now()) @db.Timestamptz(3)
  lastSeenAt           DateTime? @db.Timestamptz(3)
  /// Names an organization, grants nothing: re-checked against OrganizationMember.
  activeOrganizationId String?

  @@index([userId])
  @@map("sessions")
}

model OtpCode {
  id        String    @id @default(cuid())
  email     String
  codeHash  String
  expiresAt DateTime  @db.Timestamptz(3)
  usedAt    DateTime? @db.Timestamptz(3)
  attempts  Int       @default(0)
  createdAt DateTime  @default(now()) @db.Timestamptz(3)

  @@index([email, createdAt])
  @@map("otp_codes")
}

model VerificationToken {
  id         String    @id @default(cuid())
  userId     String
  user       User      @relation(fields: [userId], references: [id], onDelete: Cascade)
  kind       TokenKind
  tokenHash  String    @unique
  expiresAt  DateTime  @db.Timestamptz(3)
  consumedAt DateTime? @db.Timestamptz(3)
  createdAt  DateTime  @default(now()) @db.Timestamptz(3)

  @@index([userId])
  @@map("verification_tokens")
}

model OAuthState {
  id           String   @id @default(cuid())
  stateHash    String   @unique
  returnOrigin String?
  expiresAt    DateTime @db.Timestamptz(3)
  createdAt    DateTime @default(now()) @db.Timestamptz(3)

  @@map("oauth_states")
}

// ---------------------------------------------------------------------------
// Framework (copied from TourCockpit f952d0eea)
// ---------------------------------------------------------------------------

model IdempotencyKey {
  id             String       @id @default(cuid())
  organizationId String
  organization   Organization @relation(fields: [organizationId], references: [id], onDelete: Cascade)
  key            String
  requestHash    String
  status         Int
  contentType    String?
  body           String
  sending        Boolean      @default(false)
  createdAt      DateTime     @default(now()) @db.Timestamptz(3)

  @@unique([organizationId, key])
  @@index([organizationId, createdAt])
  @@map("idempotency_keys")
}

model Audit {
  id               String       @id @default(cuid())
  organizationId   String
  organization     Organization @relation(fields: [organizationId], references: [id], onDelete: Cascade)
  actorUserId      String?
  role             MemberRole?
  actorKind        String? // "USER" | "APIKEY" | "SYSTEM"
  apiKeyId         String?
  action           String
  entity           String
  entityId         String?
  beforeSummary    Json?
  afterSummary     Json?
  source           AuditSource  @default(SEMANTIC)
  requestId        String?
  revertsRequestId String?
  createdAt        DateTime     @default(now()) @db.Timestamptz(3)

  @@index([organizationId, createdAt])
  @@index([organizationId, entity, entityId])
  @@index([organizationId, requestId])
  @@map("audit")
}

// ---------------------------------------------------------------------------
// Migration bookkeeping (admin role only; no grant to the app role)
// ---------------------------------------------------------------------------

model MigrationRun {
  id          String    @id @default(cuid())
  /// "impactpulse" or "impactpulse_prod", and the snapshot it read.
  sourceDb    String
  snapshotRef String
  rehearsal   Boolean
  startedAt   DateTime  @default(now()) @db.Timestamptz(3)
  finishedAt  DateTime? @db.Timestamptz(3)
  /// Per collection: read, written, quarantined. Per table: rows, hash digest.
  counts      Json?
  verdict     String? // "go" | "no-go" with the failing checks

  quarantine MigrationQuarantine[]

  @@map("migration_runs")
}

model MigrationQuarantine {
  id             String             @id @default(cuid())
  runId          String
  run            MigrationRun       @relation(fields: [runId], references: [id], onDelete: Cascade)
  collection     String
  /// The Mongo id, or _id when the document has no id.
  sourceId       String
  organizationId String?
  /// A stable code: ORPHAN_SURVEY, NO_ORGANIZATION, EMAIL_CASE_COLLISION,
  /// DUPLICATE_MEMBERSHIP, ANSWER_SCHEMA, UNKNOWN_ENUM, BAD_DATE, SESSION_SHAPE_RESPONSE ...
  reason         String
  severity       QuarantineSeverity
  detail         Json?
  /// The source document as canonical JSON, so nothing is lost.
  document       Json
  createdAt      DateTime           @default(now()) @db.Timestamptz(3)
  resolvedAt     DateTime?          @db.Timestamptz(3)
  resolution     String?

  @@unique([collection, sourceId, reason])
  @@index([runId, severity])
  @@map("migration_quarantine")
}

Indexes and unique constraints, side by side

Mongo index today (core/database.py create_indexes) Postgres
users.email unique users.email unique, plus the lower-case CHECK
users.created_at dropped (no reader sorts users by it on a large set)
surveys.public_id unique surveys.publicId unique
surveys.created_by_id, organization_id, created_at (organizationId, createdAt), createdById
questions.survey_id, (survey_id, order) (organizationId, surveyId, order)
survey_responses.survey_id, user_id, submitted_at (organizationId, surveyId, submittedAt), (organizationId, userId), (surveyId, respondentEmail) for the cooldown, recurrenceRemindersSent for the reminder sweep
survey_sessions.survey_id, user_id, completed_at, last_activity_at, attempt_number (organizationId, surveyId, userId, status), (organizationId, lastActivityAt)
survey_assignments.assigned_at (organizationId, assignedAt), unique (surveyId, userId)
api_keys.organization_id, key_prefix same, plus hashedKey unique
otp_codes (email, created_at) same
organizations.custom_domain unique sparse customDomain unique (Postgres allows many nulls)
sessions.session_token unique, expires the new sessions.tokenHash unique
none new uniques: (organizationId, userId) on members, (surveyId, userId) on assignments, (surveyId, key) on groups, (surveyId, language) on translations, googleId, appleId, tokenHash everywhere

Every new unique constraint is one Python never enforced, so each is checked against the data in rehearsal before the cutover, and each violation has a quarantine rule (see the Cutover Runbook).

Open points on this page

Point Owner page
Whether shape B responses are kept, merged into the matching shape A response, or dropped Decision after rehearsal 1 counts
Whether old ip_live_ keys keep working, and with which scope Overview, decision 2
Whether answers stay JSON or become an answer table Overview, decision 5
Row level security for User reads across organizations (a person is visible to an org only through a membership) Framework Copy
Whether a legacy 30 minute JWT is honoured at cutover Auth
Survey soft delete (deletedAt) Phase 5, Agent Layer