Building an AI-Powered Email Writing Assistant with Personalized Style Matching
ghostwriter · Tech · 2024 · source Agentic: no · promotional bias: med · source-backed · extracted with prompt 0.5.4 (schema v2)
Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.
Problem
Domain: Personalized email writing assistance
Shortwave needed its Ghost Rider writing assistant to generate email drafts and autocomplete suggestions that sounded like the individual user and included correct contextual facts instead of generic or hallucinated boilerplate.
Constraints:
- Generic LLM email drafts sounded unlike the user and were too long-winded.
- The assistant needed access to factual context such as promo codes, Wi-Fi passwords, or office addresses.
- Autocomplete required strict formatting behavior around cursor position, whitespace, and partial drafts.
- Email is messy, and relevant examples may depend on style, content, thread context, and fact lookup needs.
Architecture
Ghost Rider indexes each user's sent emails with embeddings, stores them in a vector database, and retrieves the top similar prior emails for the current thread or draft. Those examples are included in the prompt so the LLM can copy writing style and content. For autocomplete, Shortwave fine-tuned a model with synthetic training examples derived from real emails and handcrafted fact-lookup cases, because autocomplete needs more precise cursor-aware output than full-draft generation.
Diagram confidence: reconstructed (reconstructed from prose).
flowchart LR email-client[Email Client] --> embedding-indexer[Embedding Indexer] embedding-indexer --> vector-database[Vector Database] email-client --> retriever[Retriever] retriever --> vector-database[Vector Database] retriever --> draft-assistant[Draft Assistant] retriever --> autocomplete-model[Autocomplete Model] training-data-generator[Training Data Generator] --> autocomplete-model[Autocomplete Model] draft-assistant --> email-client[Email Client] autocomplete-model --> email-client[Email Client]
Components
| Component | Role | Interacts with |
|---|---|---|
Email Client |
Shortwave inbox and editor where users read, search, draft, and autocomplete emails. | Embedding Indexer, Retriever, Draft Assistant, Autocomplete Model |
Embedding Indexer |
Offline pipeline that embeds user emails for semantic retrieval. | Email Client, Vector Database |
Vector Database |
Stores email embeddings and supports semantic similarity search over prior sent emails. | Embedding Indexer, Retriever |
Retriever |
Embeds the current thread or draft and selects relevant prior emails as examples. | Vector Database, Draft Assistant, Autocomplete Model |
Draft Assistant |
LLM prompt path for full email drafting using retrieved examples. | Retriever, Email Client |
Autocomplete Model |
Fine-tuned model for cursor-aware inline completion in the editor. | Retriever, Email Client |
Training Data Generator |
Creates synthetic autocomplete examples from real sent emails and handcrafted fact-lookup cases. | Autocomplete Model |
Data flow
Incoming and sent emails are embedded offline and stored in a vector database. At write time, the current thread or draft is embedded, similar prior emails are retrieved, the top examples are added to the prompt, and either GPT-4 produces a draft or a fine-tuned autocomplete model completes the current editor text.
Why this architecture
Retrieval solves the two failures of generic email generation: examples give the model the user's style and factual content from prior messages. Fine-tuning is reserved for autocomplete because inline completion has stricter formatting and cursor-position requirements than full drafts.
Models used: GPT-4 · embedding model · fine-tuned autocomplete model
Scale
- Ghost Rider launched earlier in 2024 and became one of Shortwave's most popular features.“like you we launched this uh earlier last year and writing has been one of our most popular features and we built this Tech to basically make the email sound like you um and we then use that Tech to uh ship a new feature early this year which”transcript, stated
- The retriever uses roughly the top five to ten similar emails, depending on email length.“we get a bunch of uh results and we sort of take the top five to 10 this depends on the the exact uh length of the emails”transcript, stated
- The team saw large autocomplete gains with four or five hundred examples.“with four or 500 examples and the improvements were massive and it turns out you can actually synthesize training data from real emails so you can take an email that you actually sent and now pretend like you're actually writing”transcript, stated
- The style-profile experiment used ten representative emails.“we took 10 emails we thought we were representative of how you write and we asked the llm please describe the style”transcript, stated
Alternatives considered (roads not taken)
Generic GPT-4 email drafting without personalization.
Rejected because: It produced boilerplate text that did not sound like the user.
Similarity retrieval alone forever.
Rejected because: The team expected ranking, cross-encoding, snippets, and multiple embeddings to improve relevance.
Only full-email examples for fact lookup.
Rejected because: Fine-grained snippets may be better when the model needs a specific fact such as a password or address.
Patterns
rag-retrieval (retrieval)
Ghost Rider retrieves semantically similar prior sent emails and uses them as examples in the writing prompt.
fine-tuning-adaptation (model-adaptation)
Autocomplete uses a fine-tuned model trained on synthetic examples from real emails and cursor positions.
synthetic-data-generation (training-data)
Shortwave turns sent emails into synthetic draft-completion examples by randomizing cursor position and withholding the rest of the text.
Tradeoffs
Personalization architecture
Chose Retrieve similar prior emails into the prompt over Ask a generic LLM to infer style from the current request.
Rationale: Retrieved examples supply both personal style and content evidence from the user's own history.
Model adaptation scope
Chose Fine-tune autocomplete over Use only off-the-shelf GPT-4 for every writing surface.
Rationale: Full draft generation worked with GPT-4, but autocomplete needed task-specific output behavior.
Gotchas (what bit them)
Style without facts is not enough for useful email drafting.
Root cause: Users need contextual details such as promo codes or office information, not just prose.
Resolution: Retrieve prior emails and add handcrafted fact-lookup examples to training.
Autocomplete can hallucinate missing facts if not trained on absence cases.
Root cause: The model needs to learn when examples do not contain the requested fact.
Resolution: Include no-fact-lookup examples alongside handcrafted fact examples.
Email retrieval relevance is more nuanced than whole-email semantic similarity.
Root cause: Facts, style, topic, and snippets may require different retrieval signals.
Resolution: Explore cross-encoding, snippets, and multiple embeddings per email.
Evaluation
Shortwave evaluates autocomplete by user acceptance of suggestions and by training examples designed to cover cursor position, completion boundaries, fact lookup, and missing-fact behavior.
Metrics: Suggestion acceptance · Autocomplete formatting correctness · Fact inclusion · No-hallucination cases · Retrieval similarity quality
Outcomes
- Ghost Rider launched as a popular writing feature in Shortwave.
- The autocomplete feature shipped after fine-tuning improved performance.
- The system could include specific facts such as a real promo code from prior email context.
The source reports launch timing, popularity, top-five-to-ten retrieval, ten-email style profiling, and large fine-tuning gains from roughly 400-500 examples.