Meeting Notetaker Agent

Nova-3 leads on streaming latency (sub-300ms), accented-English WER, and built-in diarization quality. AssemblyAI is competitive. Self-hosted Whisper is viable at scale but requires GPU ops.

Alternatives: AssemblyAI Universal-2, Whisper v3 Large, AWS Transcribe

Sonnet 4 produces cleaner structured output and fabricates action items much less often than GPT-4o on meeting transcripts — verified on real-world eval sets.

Alternatives: GPT-4o, Gemini 2.5 Pro

Runs every 60s on rolling window. Speed and cost matter more than peak quality.

Alternatives: GPT-4o-mini, Gemini 2.0 Flash

Recall.ai abstracts Zoom/Meet/Teams/Webex bot joining with one API. Building this yourself takes months of platform-specific edge cases.

Alternatives: Fireflies API, Custom via Zoom SDK

Meeting search is the killer feature at scale ('what did we decide about pricing last month'). Embed each meeting's summary + quotable moments, not every turn.

Alternatives: Supabase, Turbopuffer

Action items are useless unless they land where the owner already works. Support 2–3 destinations deeply rather than 10 shallowly.

Alternatives: Asana, Jira, Notion

You must eval against real meeting transcripts with known action items — synthetic data misses the 'X said they'd think about it' ambiguity.

Alternatives: Langfuse, Humanloop

Streaming vs batch STT

Streaming enables live captions and live summaries but costs 2–3x batch. For most workflows (post-meeting summary), batch after the call is fine and cheaper. Use streaming only when you're actively surfacing context during the call.

Deepgram vs self-hosted Whisper

At <10k hrs/month, Deepgram is cheaper and better. Above 50k hrs/month, self-hosted Whisper v3 on L40S or H100 GPUs hits lower unit cost but requires ops, GPU capacity planning, and gives up some WER on accents without fine-tuning.

One-shot summary vs extract-then-summarize

Feeding the full transcript and asking for everything at once is simpler but more lossy. Two-pass (first extract decisions + action items as bullets, then summarize) yields measurably better action item recall and fewer hallucinations, at ~1.5x cost.

Hallucinated action items no one actually committed to

Mitigation: Require each action item to include a verbatim quote from the transcript as source. Reject extractions where the quote doesn't contain explicit commitment language ('I'll', 'we should', 'let me', 'by Friday'). Run a secondary Sonnet pass as LLM-as-judge.

Wrong speaker attribution on crosstalk

Mitigation: Use Deepgram diarization with per-speaker channels where the platform provides them (Zoom does, Google Meet partially). For single-channel audio, voice-print match against attendees' past meetings; flag low-confidence attributions in the summary.

Summary misses the one key decision

Mitigation: Add a dedicated 'decisions' extraction pass with a tight rubric (a decision = statement + agreement from >1 speaker). Run against a golden eval set of 200 meetings with human-labeled decisions before any prompt change ships.

Privacy breach — summary delivered to wrong channel

Mitigation: Never auto-deliver across org boundaries — internal summaries never DM external attendees. Per-meeting ACL derived from calendar invite. Default to private (invitee-only) delivery; user explicitly opts in to broader channels.

Audio with thick accent or technical jargon drops WER

Mitigation: Enable Deepgram's custom vocabulary with product/person/company names from the meeting invite and past transcripts. Prompt the summarizer with 'likely typos: X→Y' hints derived from STT confidence scores below 0.7.