Real-Time AI Chief of Staff for Product Teams

earmark · Tech · 2026 · source Agentic: yes · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Real-time meeting intelligence for product teams

Earmark needed to turn product-team meetings into usable specs, tickets, summaries, questions, and follow-up artifacts during the meeting, without overwhelming users with transcript noise, hallucinations, privacy risk, or unsustainable token costs.

Constraints:

• The product operates live, so transcript deltas and artifacts must update while the meeting is still happening.
• One-hour meeting transcripts are large enough that repeatedly sending full histories would make API costs untenable.
• Previous agent outputs can bias later summaries if included in the model history.
• Enterprise buyers care deeply about privacy, storage, and whether customer conversations are retained.
• Long-horizon search across many meetings exceeds naive context-window and vector-search approaches.

Architecture

Earmark uses real-time AssemblyAI transcription, OpenAI prompt caching, and multiple task/template agents to transform meeting transcripts into artifacts for product teams. A shared transcript history is kept as the source of truth for caching, while individual templates request fresh outputs without inserting prior agent outputs into history. The roadmap extends this into multi-strategy agentic search over transcripts, metadata, BM25, vector search, and generated summary layers.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  A["Meeting audio"] --> B["AssemblyAI transcription"]
  B --> C["Cached transcript history"]
  C --> D["Shared history agent"]
  C --> F["OpenAI models"]
  D --> E["Template task agents"]
  D --> F
  E --> G["Artifact workspace"]
  G --> H["Cursor and v0 export"]
  I["Search and retrieval agents"] --> J["Transcript and summary pyramid"]
  I --> F
  K["Temporary privacy mode"] --> G

Components

Data flow

Audio is transcribed in real time by AssemblyAI. Every roughly 30 seconds, transcript deltas are appended to a cached OpenAI message history. A single responsible agent sends the shared transcript history, then task/template prompts request specific artifacts such as engineering translations, specs, tickets, meeting minutes, persona questions, or summaries. For longer-term retrieval, Earmark plans a data-pyramid approach where role-specific summaries sit above raw transcripts and agents choose between vector, keyword, metadata, database, and summary tools.

Why this architecture

Earmark's architecture is optimized for live artifact generation at tolerable cost. Prompt caching makes repeated transcript use economical, keeping agent outputs out of history avoids bias and preserves cache hits, and a single shared history path prevents every card/tool from paying for the full transcript independently.

Models used: GPT-4.1 · GPT-5.1 · GPT-5.2 · OpenAI models · Reasoning models

Scale

• A one-hour meeting transcript is about 16,000 tokens.
  “for comparison an hour meeting transcript is about 16,000 tokens compounded that could be a lot within the hour.”transcript, stated
• Prompt caching reduced an hour-long meeting from about $70 in API cost to under a dollar.
  “it was around $70 for an hour meeting in terms of API cost. We were able to get that down obviously to like sub like a dollar.”transcript, stated
• The initial customer-discovery pivot was based on about 60 interviews.
  “conducted like 60 interviews of product managers communication coaches and all these adjacent personas”transcript, stated
• The PMF survey reported 78% of respondents would be super bummed if the product disappeared.
  “78% of our respondents in our PMF survey say they'd be super bummed if it went away tomorrow.”transcript, stated
• The search problem changes once teams have used the product for a year or across many people.
  “if your team is using it or you've been using this for a year, suddenly your answers are going to degrade.”transcript, stated

Alternatives considered (roads not taken)

Use an Apple Vision Pro rehearsal product

Rejected because: Customer discovery showed few people prepare for presentations, and the addressable market was too small.

Show the full transcript prominently

Rejected because: Users fixated on transcript mistakes instead of the generated insights, so Earmark minimized the transcript UI.

Pass previous agent outputs into the model history

Rejected because: Earlier summaries biased later outputs and also would have broken prompt caching.

Rely on vector search alone for cross-meeting analysis

Rejected because: Analytical questions often require synthesis across meetings rather than retrieving an answer that exists verbatim in one transcript.

Patterns

multi-agent-orchestration (orchestration)

Multiple live task/template agents run in parallel to generate engineering translations, persona questions, specs, tickets, compliments, and other artifacts from the meeting.

context-engineering (context)

The system keeps transcript history as the source of truth, excludes previous agent outputs from history, uses prompt caching, and plans summary layers to manage context selection.

cost-aware-routing (operations)

Prompt caching and a single shared history path keep repeated transcript processing economically viable.

rag-retrieval (retrieval)

The planned search layer combines vector search, BM25, metadata, database queries, and generated summaries rather than relying on one retrieval method.

privacy-preserving-processing (safety)

Temporary mode can bypass Earmark servers entirely, and the early no-storage posture became an enterprise security advantage.

eval-driven-development (quality)

The team uses customer feedback, copy-out analytics, manual artifact review, and targeted future evals rather than comprehensive production-data evals.

Tradeoffs

Use GPT-4.1 as the default writing model despite newer models being available.

Chose GPT-4.1 for final prose style over Newer models that produced bullet-heavy output.

Rationale: Customers preferred GPT-4.1's prose for real-time summaries/artifacts even though newer models extracted more nuance.

Prioritize privacy/no storage before a fuller persistent product.

Chose Ephemeral/no-database early product and temporary mode over Persisting every meeting by default.

Rationale: The no-storage limitation became a security advantage for enterprise evaluations and supported privacy commitments.

Design asynchronous search for high-quality answers instead of forcing instant responses.

Chose Longer-running asynchronous tasks over Sub-second chatbot expectations.

Rationale: Deep cross-meeting analysis may require slower multi-step retrieval and reasoning, and users will wait when the output is drafted work rather than chat.

Use targeted customer/usage evaluation before comprehensive eval coverage.

Chose Customer feedback and copy-out analytics first over Broad production-data evaluation from day one.

Rationale: Privacy constraints and team bandwidth made comprehensive evals impractical; persistent issues will drive targeted evals.

Gotchas (what bit them)

Prompt caching can be defeated by putting agent outputs into the shared history.

Root cause: OpenAI caching depends on identical message history, and extra tool outputs both change the history and bias later generations.

Resolution: Earmark keeps prior artifact outputs out of the history and sends only transcript plus the current card/template request.

Incorrect speaker attribution is worse than no speaker attribution.

Root cause: The local/internal-audio setup cannot reliably diarize speakers, and wrong names seed misinformation into the LLM.

Resolution: Earmark avoids speaker diarization and relies on context/calendar truth where available.

Prominent raw transcripts shifted user attention to transcription errors.

Root cause: Exposing intermediate system artifacts invited scrutiny even when the errors did not materially hurt downstream LLM outputs.

Resolution: The UI minimized transcript display to subtitle-like feedback while emphasizing generated insights.

Vector search alone fails for analytic questions whose answers are not verbatim in transcripts.

Root cause: Questions like improving discovery calls require reasoning across patterns, not retrieving a sentence that already exists.

Resolution: Earmark is building multi-tool agentic retrieval with summaries, metadata, keyword search, vector search, and reasoning models.

Evaluation

Earmark evaluates through customer feedback, manual review of artifacts on updates, usage analytics such as copying artifacts out of the system, privacy-preserving future dev evals, and potential proof-of-work/citation UX for grounding.

Metrics: Copy-out rate for generated artifacts · Customer feedback after releases · Manual artifact review for style/prose quality · Whether outputs can cite transcript evidence · PMF survey response · Cost per meeting after prompt caching

Outcomes

• Prompt caching turned previously untenable meeting economics into sub-dollar LLM costs for an hour-long meeting.
• Users adopted the product even when it lacked storage and headphone support, signaling strong value.
• Enterprise prospects treated no-storage/temporary behavior as a privacy and security advantage.
• The product pivoted from Vision Pro rehearsal into a web-based AI chief-of-staff direction grounded in product-team workflows.
• The retrieval roadmap is moving from simple transcript search toward agentic multi-strategy search and summary pyramids.

Earmark is a small team serving product teams; cited operating numbers include 60 discovery interviews, 78% PMF-survey disappointment if removed, one-hour transcripts around 16,000 tokens, API cost reduction from about $70/hour to under $1/hour, and first-month template-heavy user behavior.