Sondely as an Agent: an investigation

What it would take for a person, or another agent, to make and run a Sondely survey by talking or from code, measured against how TourCockpit built its assistant (Faro). Investigated on 3 November 2026 from the Sondely code on develop and main, the specs in this folder, and the TourCockpit code and docs.

The short answer

Sondely has a good domain model and a real REST API, but not yet the layer an agent needs: no dry run, no semantic validation before publish, free-text errors, no idempotency, no audit trail, API keys not limited to their organization, and one open security hole in production (step 0 below). TourCockpit shows the pattern that works, and almost all of it carries over as a pattern. As of 5 November 2026 Sondely's backend is moving to the TourCockpit stack, so much of it will also carry over as code: see the migration to the TourCockpit platform.

Built on 7 and 8 November 2026. From Chip's goal of 7 November 2026 ("a full agentic system with voice and charts"), steps 1 to 3 are built on the new platform, CatalyzeUp/sondely-platform, and the builder agent has a name: Sona, which echoes Sondely, sondage and sonar. Owners and admins reach Sona in the in-app drawer, from their own Claude or ChatGPT over MCP, and by voice in the browser, with survey charts in the conversation and during a call. Respondents by WhatsApp and by voice (step 4) are not built. It runs on platform-dev.sondely.com; sondely.com is still the Python app, so step 0 still applies there. What was built, door by door: Agent Layer.

The recommended order:

1. Close the security hole and scope the keys. Nothing agent-shaped ships before this.
2. Make the API legible to an agent: validate, dry run, error codes, idempotency, audit.
3. Open the outside door first (MCP). Claude or ChatGPT is the brain, Sondely is the tools. This is "make a survey from code" with no model cost to Sondely.
4. Then the in-app chat, using the Azure OpenAI deployment Sondely already has.
5. Respondents by WhatsApp text, then voice, only after a decision on consent and data processing. This is where Sondely would go further than TourCockpit, and where the real risk is.

Two different agents

"Sondely as an agent" covers two products that share an API and nothing else.

The builder agent is the TourCockpit pattern applied to a smaller domain. The respondent agent is new ground.

How TourCockpit does it

Every surface runs the same turn and the same tools ("many doors, one path"):

What makes it work, in order of importance:

• Tools are generated from the API registry. Every write route becomes a tool whose parameters are the route's own schema (87 write tools from about 125 routes). The model never writes a request body as free text.
• Every write is proposed, dry-run, shown, then committed. The route runs with ?dryRun=true and returns what would happen; the person sees a card; only an explicit yes from the person commits it, at most once, with an idempotency key. A proposal cannot be committed in the same turn that made it.
• Refusals are legible. About 65 error codes in one envelope, each classed as ask, explain, denied or retry, typed so a new code does not compile until it is classed. The rule that came out of an incident on 20 October: "A rule added in the place an agent cannot see is a rule every agent will break."
• The prompt holds principles, the tools hold specifics. Feature-specific text lives on the tool or the refusal, never in the always-loaded prompt. A parity test keeps the in-app and MCP rules identical.
• Voice only gets one tool. The realtime model can only ask the text assistant (ask_faro) and hang up, so voice never grows its own write path.
• Identity is per request. WhatsApp maps a number a manager registered; MCP uses OAuth or an API key, resolved on every call, never cached.
• It is tested like the product. Simulated WhatsApp calls in CI, nightly live-model runs graded on what happened rather than on wording, refusal probes, and a read-only agent that checks every API change for legibility.

What it cost: about 24,000 lines in the assistant and MCP code plus about 55,000 lines of tests for those areas, 588 commits over eight weeks (6 September to 30 October 2026), by two people working with agents. The most expensive parts were voice transport (recording, timing, interruptions, hang-up) and refusal legibility, not the chat itself. Model cost is still not metered per call; the nightly live-model suite alone was about USD 46 a night before it was cut to changed nights only.

Where Sondely stands

Size for comparison: Sondely is about 12,500 lines of backend, 18,600 of frontend and 15,000 of tests, 288 commits, quiet since 21 September. The builder agent's domain is much smaller than TourCockpit's: no money, no bookings, no rooms, no groups, six question types.

Step 0: the security hole, now

POST /api/v1/auth/auto-enroll needs no sign-in. Given an email and an organization id, it returns a four-hour token for that email's account if one exists. The token carries "scope": "respondent", but get_current_user in backend/app/utils/auth.py never reads the scope, so the token works as a full sign-in. Anyone who knows an owner's email can act as that owner; the organization id is public on every survey page. This is in production. Two smaller ones belong in the same pass: API keys are not limited to their organization, and CORS allows * with credentials.

Done means: an existing account can no longer get a token from this endpoint without proving the email (the one-time code already exists), respondent tokens are refused on every non-respondent route, keys only reach their own organization, and a test proves each.

Step 1: an API an agent can read

The work that makes every later door safe, and that also helps the web app:

• POST /api/v1/surveys/validate and ?dry_run=true on create, update, publish and question changes, returning what would happen and every problem found.
• A semantic validator: choice questions have options, scoring type matches question type, groups exist, translations keep ids and option counts (the translation validator already does this last part).
• One error envelope with codes, and a table that classes each code as ask, explain, denied or retry, enforced by a test the way TourCockpit types it.
• Idempotency keys on every write.
• Question update and reorder.
• Soft delete, or a two-step confirmation, on the destructive routes.
• API key scopes: read, draft (create and edit drafts, never publish or delete), full.
• An audit log: actor, key, route, before and after.
• The stale specs fixed (api-and-keys.md still says keys and rate limits are not built).

Done means: an outside script can create a Maslow pre/post survey with a draft-scoped key, get a dry-run report with a coded error for a missing option, fix it, and the retry does not create a duplicate.

Step 2: the outside door first (MCP)

The cheapest way to "make a survey from code" is to let the person's own agent be the brain. Sondely exposes an MCP server whose tools are generated from the OpenAPI schema FastAPI already produces, with the same propose, dry run, commit flow. Sondely pays no model cost; the customer's Claude or ChatGPT does the talking.

Done means: from one sentence in Claude ("a food security baseline in French and Arabic for 40 families, monthly"), Sondely ends with a valid draft, translated, and publishes only after a separate yes, proven by a scenario test that runs in CI.

This fits the zero-friction principle: meet the customer in the tools they already use, show value before asking for an account. It also gives a developer API story ("surveys from code") for free.

Step 3: the in-app chat

The same executor behind a chat drawer in Sondely, using the gpt-5.4 deployment already provisioned. Two jobs:

• Build: describe a survey, get a draft card, edit by talking.
• Read results: "how did the Maslow scores move for the March cohort". This should go through read-only insight tools on top of the existing scoring engine, never model-written queries. TourCockpit learned this the hard way on 6 October, when a hand-written summary reported 70 departures where 33 ran.

Done means: the same scenario tests pass through the drawer as through MCP, and a parity test keeps the two sets of rules identical.

Builder voice, added on 7 November 2026

Chip's goal of 7 November 2026 put voice in the builder agent too, ahead of respondents: an owner or admin calls Sona from the browser and talks through a survey or its results, and the charts appear beside the live transcript. The TourCockpit rule holds: the realtime model has two tools, ask_sona and end_call, so voice has no write path of its own, and a change is committed only on a spoken yes in a later turn. It runs on Azure AI Voice Live through a CatalyzeUp Speech resource, because the subscription's Azure OpenAI realtime quota is fully used by TourCockpit. Built on 8 November 2026 (Agent Layer).

Step 4: respondents by WhatsApp, then voice

This is where Sondely could do something TourCockpit does not: a beneficiary with a phone but no email answers a survey in their own language by WhatsApp, or by a call. For nonprofits that reach people who do not use web forms, this may be the most valuable part of the whole idea. It is also the part with the most risk.

What it needs beyond steps 1 to 3:

• Identity by phone number, invited by the org, never by caller ID alone (TourCockpit refused a plain phone line for this reason).
• One conversation is one session, mapped onto the existing start, save, complete and abandon routes, so a person can stop and resume.
• Answers validated against the question type before they are saved.
• The agent reads questions and options verbatim. It never rephrases, explains an option in its own words, or suggests an answer. A survey's value is that every respondent got the same question. A voice or chat mode can still shift answers compared with a form (a known effect in survey research), so results collected by voice should be marked with their channel, and the scoring for validated instruments like the Maslow assessment should not mix channels without saying so.
• Consent: captured at the start, recorded with the session, in the respondent's language.
• Data processing: answers about people's lives would pass through a language model. The Azure deployment is GlobalStandard, which can process in any region; a regional or data-zone deployment and a written data-processing term for each org are needed before real beneficiary data goes through it.
• Retention and anonymisation, which do not exist today.
• Voice follows the TourCockpit rule: the realtime model gets only "ask the survey agent" and "end the call", so it never has its own write path. Builder voice already runs this way in the browser (ask_sona, end_call); a respondent call adds the phone, recording and the survey runner. Expect the same transport work TourCockpit paid for: recording, silence, interruptions, hanging up.

Done means: a simulated respondent completes a 12-question Maslow survey by WhatsApp in Spanish, stops halfway and resumes the next day, every answer is stored with its channel and consent record, and a live-model eval checks that no question was paraphrased.

Cost and money

• Sondely runs in the same Azure sponsorship subscription as IdeaPlaces, so Azure OpenAI, Speech and Container Apps spend is covered by the credits until 28 July 2027.
• WhatsApp conversation fees are real money, per message window, and grow with every respondent.
• MCP costs Sondely nothing in model spend.
• Meter model use per call from the first day. TourCockpit specified per-call cost metering and has not built it, and is now flying blind on it.

Size by complexity

Steps 0 to 2 are the same kind of work Sondely already does. Step 4 is a product decision before it is an engineering one.

Decisions for Chip

1. Fix the auto-enroll hole now, ahead of everything else.
2. Whether the builder agent (steps 1 to 3) is worth doing for Sondely at all, and if so whether MCP first is the right door. Decided by Chip's goal of 7 November 2026: yes, all three doors at once (MCP, the drawer, and voice), built on 8 November 2026.
3. Whether respondent conversations (step 4) are a direction for Sondely, given the consent and data-processing work it needs, and which customer would be first.