One agent. Every inbox. Trained to sound like the artist.
A client DMs on Instagram, messages WhatsApp or Viber, or emails — it all lands in one normalized thread. The agent works out what they want, pulls the studio's rates, the artist's voice, availability and the client's history, then answers, quotes, books, reminds, or hands off — talking as the artist, not as a bot. It cold-starts from uploaded past conversations and gets sharper with every message the artist approves or edits.
The pricing engine, memory and deposit rails it stands on already ship in co-muse today — this is the concierge layer on top.
"Hey! A colour koi half-sleeve like that runs about €1,260 over 3 sessions — a €378 deposit locks your first date. I've got Thu 14:00 free. Want it?" — in William's voice
Three moving parts: fan-in, the loop, and two training loops that feed it.
① Fan-in
Every channel behind one adapter interface. Inbound normalizes to a single message shape; the rest of the system never knows which channel it came from.
② The loop
Per message: perceive → retrieve → decide → compose (in voice) → gate → act → learn. Idempotent and checkpointed, one run per inbound.
③ Training
Voice profiles (per artist/studio) and quote calibration (per artist) learn continuously — from uploads at cold start, from approvals + real outcomes live.
Design stance (foundation-first): data model and channel boundaries are locked before any adapter is written; every external call is wrapped, timed, retried with backoff; every agent run is idempotent and logged. Voice is learned as reviewable prompt material (style cards + few-shot), not fine-tuned weights — instant to update, safe with PII, and the artist can read exactly what the agent thinks their voice is.
1
Channels & fan-in
Four platforms, one normalized inbound shape. Each adapter verified against live docs, never assumed.
// every channel collapses to this — nothing downstream is channel-aware
NormalizedInbound {
channel_kind: "instagram" | "whatsapp" | "viber" | "gmail"
shop_id, external_thread_id, sender_ref
text, attachments[] // images drive the quote path
lang?, timestamp, provider_msg_id // provider_msg_id = idempotency key
}
Per-channel reality — verified, not assumed
WhatsApp · Business Cloud
Free-form replies only inside the 24-hour customer-service window; outside it, pre-approved template messages. Meta Business verification + registered number. Webhook verify via hub.challenge.
Verify before codingwindow rules, template approval flow, current webhook payload shape
Instagram · Messaging
Professional account linked to a Page, instagram_manage_messages + App Review. 24h window; the human-agent tag extends it (~7 days). Media + story-reply events differ.
Verify before codingpermission scopes, App Review, human-agent tag rules, event types
Viber · Public Account bot
Register via set_webhook; distinct event types (message, conversation_started, delivered, seen, failed). Own rate limits and sender/keyboard format.
Verify before codingevent schema, min API version, keyboard/quick-reply format, limits
Gmail · API
OAuth2 gmail.modify; push via Pub/Sub watch() (expires ~7 days → renew) or history polling. Threading via threadId / References; strip quoted history on parse.
Verify before codingwatch renewal, history vs push, threading + quote-stripping
Adapter rule (anti-assumptions): before writing each adapter, fetch that platform's current docs, produce an assumption inventory (auth, webhook verify, inbound shape, send limits, session window, media), verify each item, and flag anything unconfirmed. Ship one channel end-to-end (WhatsApp or Gmail first) before adding the rest.
2
The agent loop
One idempotent, checkpointed run per inbound. Watch it play — or click a step to pin it.
Invariants (agentic): every step is safe to re-run; a run that dies at step 6 restarts from its last checkpoint, never re-charges or double-sends; external calls have timeouts + exponential backoff with jitter; every run logs input, output, tools, decision, confidence and latency so a human can see exactly what happened.
3
Voice training — per artist / studio
Learn how each artist actually talks, then talk that way. Two loops.
Loop A · cold start — from uploaded conversations
Upload / pasteIG · WhatsApp · Gmail exports
Parse & segmentsplit by role: artist vs client
DistilClaude → style card
Select exemplarsrepresentative artist turns
voice_profile v1style card + few-shot
Loop B · online — from live approvals & edits
Agent draftsin current voice
Human approves / edits / rejectspreference signal
Store signalapproved + (draft→edited)
Re-distil at Nrefresh card + exemplars
voice_profile v++versioned · rollback-able
The style card what's learned
A structured, human-readable distillation the artist can read and correct: tone & warmth, formality, emoji habit, sentence length, signature phrases, greeting/closing patterns, how they explain price, what they never say, and languages used. Injected into the system prompt on every reply.
voice_profiles.style_card (JSON)
Dynamic few-shot how it stays situational
Exemplars are embedded; at compose time the agent retrieves the artist turns most similar to the current situation — a colour half-sleeve enquiry pulls past colour half-sleeve replies. Recent approved sends are up-weighted; rejected drafts down-weighted.
search_memory(query, scope=artist)
Deliberate decision · flagged for sign-off
Prompt-based voice, not per-artist fine-tuning.
Style card + retrieved few-shot is instant to update, cheap, fully inspectable, reversible, and never bakes client PII into model weights. Fine-tuning is slower, costlier, opaque to the artist, and a privacy liability. Recommendation: ship prompt-based; revisit fine-tuning only if scale demands it.
Guardrails on voice: the learned voice controls style, never facts or policy. Prices, availability, deposit terms and safety rules come from the guardrail layer and always override tone. A casual, emoji-happy artist voice still can't promise a date that isn't free or a price below the minimum.
4
Quote training — calibrated, not black-box
The pricing engine is deterministic; learning means calibrating its parameters to reality.
Quote issuedpredicted hours + price
Outcomesactual chair-time · booked?
Residualsactual ÷ predicted, by ink
Update factorsEWMA · bounded
k · by_inkversioned · artist-visible
Signals
Logged actual chair-hours vs predicted; quote→book conversion (did the price land?); the artist's manual overrides on drafts.
Update rule
A bounded EWMA of actual÷predicted per artist, split by ink (colour vs b&grey); optional light residual regression on size/complexity/fill. Interpretable so artists trust it.
Guards
Adjustments capped per cycle (no wild swings); never below studio minimum; the artist can pin or override any factor. Shown live in the console Training view.
Why not learn the price directly: artists must trust the number and be able to explain it. A calibrated, transparent multiplier on a known formula beats an opaque model that quotes a figure nobody can defend at the chair.
5
Autonomy & escalation
One dial per studio (or per artist). Try it — start supervised, earn autonomy.
Routine, high-confidence replies send themselves; quotes, negotiation and anything sensitive wait for the artist. The recommended cruising mode.
Always escalate to the human when
Low confidence
The classifier or compose step isn't sure — never guess at the client.
Negotiation
Price push-back beyond the studio's allowed flex. The artist decides discounts.
Complaint
Anything unhappy, medical, or reputational. Humans handle feelings.
Unknown fact
A question whose answer isn't in retrieved context. Ask or hand off — never hallucinate.
Safety / policy
Age, consent, medical suitability, or anything the safety layer flags. Hard stop to human.
Explicit request
"Can I talk to a person?" → immediate handoff, no friction.
Handoff is graceful: on escalation the agent posts an internal summary (intent, what it gathered, its draft) to the studio inbox and, if useful, tells the client a human will follow up — then goes quiet on that thread until released.
6
Data · tools · infra
The foundation the loop stands on. Full schemas + build order live in the build spec.
shop, artist, client, policies, voice profile for a conversation.
search_memory
top-k situational exemplars & past resolved threads.
read_design
vision read of an attached image → factors.
estimate_quote
calibrated price / hours / deposit for an artist.
check_availability
real open slots for an artist + window.
create_checkout
Stripe deposit link for a quote.
send_message
outbound via adapter; window + template aware.
escalate
hand the thread to the human queue with a summary.
Tools now, MCP later: expose these as Anthropic tool-use functions inside the agent worker (the product already speaks the Messages API). Wrapping them as an MCP server is an optional later step for portability — not required to ship.
Infra shape
Webhook ingressverify · 200 fast
Durable queueper-conversation order
Agent workerthe loop · idempotent
Send workerlimits · templates
Secrets
Per-shop channel tokens in a secret manager; OAuth per studio. Never in code.
Cost control
Cheap intent model before full compose; vision only when images present; cache retrieval; batch embeddings; token caps.
Observability
Every run traced; dashboards on escalation rate, send failures, latency, spend. Alert on spikes.
Safety & GDPR
Reuse child-safety + scope guardrails; PII minimized; voice learned as reviewable text, never weights; export/delete.
Build order — seven layers
L0 data model + normalized types
L1 channel adapters behind one interface ← verify each API live; ONE channel firstL2 agent core logic (pure, fixture-tested): intent · policy · guardrails · voice compose
L3 state + durable queue (per-conversation serialization, checkpoints)
L4 webhook transport + send workers (windows, templates, backoff)
L5 console wiring (inbox, suggestions, voice/quote training views)
L6 autonomy dial + escalation + training loops online
L7 hardening: observability, rate limits, cost caps, safety, GDPR ← before production traffic
Prove it manually first: run one channel end-to-end with a human approving every send before turning the autonomy dial up. Automating an unclear process just makes the mistakes faster.
co-muse · the agent · omni-channel conciergeOpen the app →