Collaborative Prompt Engineering Platform for Production LLM Development

linkedin · Tech · 2025 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.5 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Collaborative prompt engineering for production Sales Navigator LLM features

LinkedIn needed a way for engineers, product teams, domain experts, and sales users to iterate on production-like prompts together without brittle local setup, manual template filling, stale test data, or slow feedback loops.

Constraints:

• LLM outputs are non-deterministic, making quality control at scale difficult.
• Prompt development requires the same baseline model configuration for comparable results.
• Prompt templates depend on dynamic context, chains, external tools, and production-like orchestration.
• Non-technical domain experts needed direct access to prompt iteration without managing dependencies.

Architecture

LinkedIn built collaborative prompt-engineering playgrounds around Jupyter Notebooks launched from the Sales Navigator Python backend repository. The backend uses LangChain chains, Jinja prompt templates, gRPC use-case routing, Trino-backed test-data collection, Bing Search APIs for live external data, privacy-enhanced test datasets, and remote development containers so cross-functional teams can run production-equivalent chains, edit prompts, review changes, and deploy through the normal repository pipeline.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  cross-functional-prompt-users["cross-functional prompt users"] --> remote-build-container["remote build container"]
  cross-functional-prompt-users --> jupyter-prompt-notebook["Jupyter prompt notebook"]
  remote-build-container --> jupyter-prompt-notebook
  remote-build-container --> python-backend-service["Python backend service"]
  jupyter-prompt-notebook --> python-backend-service
  jupyter-prompt-notebook --> test-data-notebooks["test data notebooks"]
  jupyter-prompt-notebook --> jinja-prompt-templates["Jinja prompt templates"]
  jupyter-prompt-notebook --> internal-github-repository["internal GitHub repository"]
  python-backend-service --> grpc-endpoint["gRPC endpoint"]
  python-backend-service --> langchain-chain-runtime["LangChain chain runtime"]
  grpc-endpoint --> langchain-chain-runtime
  langchain-chain-runtime --> jinja-prompt-templates
  langchain-chain-runtime --> bing-search-api["Bing Search API"]
  langchain-chain-runtime --> llm-provider-configuration["LLM provider configuration"]
  test-data-notebooks --> trino-data-lake["Trino data lake"]
  test-data-notebooks --> privacy-enhancing-data-handling["privacy-enhancing data handling"]
  jinja-prompt-templates --> internal-github-repository
  cross-functional-prompt-users --> internal-github-repository

Components

Data flow

A team member launches a containerized notebook from the Python repository, selects a use case and test company, invokes the same LangChain chain used by the Sales Navigator backend, and the chain fetches/transforms data, populates Jinja placeholders, optionally calls Bing Search for real-time external data, invokes the LLM, and returns outputs in the notebook. Prompt, notebook, and test-data changes stay in the repository for code review and deployment, while tracked production usage refreshes the test datasets.

Why this architecture

LinkedIn chose notebooks inside the real code repository because prompt engineering for production features requires production-equivalent chains, reusable test data, shared model configuration, domain-expert collaboration, and code-reviewed changes rather than isolated provider playgrounds.

Models used: OpenAI models · LinkedIn internal models · Llama · LLMs behind Sales Navigator features

Scale

• AccountIQ reduced company research from two hours to five minutes.
  “AccountIQ automates company research, reducing what used to take two hours to just five minutes.”article, stated
• The platform handles multiple LLM configurations including OpenAI, internal models, and Llama.
  “There are different LLMs available, such as OpenAI models, internal models, and publicly available models, like Llama.”article, stated
• Setup is designed to launch with a single command.
  “launch the notebooks in browser or IDE with a single command.”article, stated
• Remote containers can be launched with a few commands.
  “With just a few commands, anyone in the team can launch their remote build containers with code repositories fetched and pre-built.”article, stated
• The collaboration loop can move prompt iteration within days.
  “we’re able to directly involve the cross-functional team to collaborate on prompt iteration within days”article, stated

Alternatives considered (roads not taken)

LLM provider online playgrounds

Rejected because: They are useful for exploration but do not integrate with production feature code, model configuration, chains, and environment requirements.

Manual placeholder replacement in prompt templates

Rejected because: It is not feasible or scalable when prompts need company-specific context and external data.

Testing prompts in isolation

Rejected because: Production behavior may involve multiple chained prompts and orchestration frameworks such as LangChain.

Pre-caching external Bing results for prompt tests

Rejected because: LLM-generated search queries are dynamic, so external results need to be fetched in real time.

One-off static test data collection

Rejected because: LinkedIn says test data must be sampled regularly to stay fresh and representative.

Patterns

prompt-versioning (prompt-management)

Prompt files live in the code repository, can be committed, versioned, reviewed, and deployed through the standard pipeline.

prompt-chaining (orchestration)

The platform tests entire LangChain workflows with multiple prompts rather than single prompts alone.

human-in-the-loop-gate (collaboration)

Non-technical domain experts and Sales users participate directly in live prompt sessions and feedback loops.

eval-driven-development (evaluation)

Representative, diverse, regularly refreshed test datasets are used to evaluate prompt changes quickly before production.

tool-use-orchestration (orchestration)

AccountIQ chains invoke external tools such as Bing Search APIs when LLM-generated queries require live data.

privacy-preserving-processing (security)

Test-data handling includes privacy-enhancing processes to avoid exposing personal information in prompt datasets.

structured-output-contract (interface)

Each chain receives dictionary-style input parameters and use-case metadata through a gRPC endpoint.

Tradeoffs

Use Jupyter Notebooks as prompt playgrounds.

Chose Notebook-based prompt engineering launched from the Python repository over Standalone online prompt tools.

Rationale: Notebooks can combine executable production code, test data, chain invocation, prompt editing, and cross-functional visibility in one environment.

Store notebooks and test data in the code repository.

Chose Repository-managed notebooks and test datasets over Unreviewed ad hoc notebook copies.

Rationale: Repository storage forces code review before prompt playground or test-data changes are merged.

Use remote containers for setup.

Chose LinkedIn remote development containers with prebuilt repositories over Each user resolving local dependencies manually.

Rationale: Containers make setup accessible to non-technical contributors and keep dependencies consistent.

Fetch external data in real time during tests.

Chose Live Bing Search API calls generated by the LLM over Saved Bing results.

Rationale: Dynamic search queries cannot be known or cached in advance, so the framework automates live access.

Use production-equivalent chains for prompt experimentation.

Chose LangChain chains from the Python backend over Single-prompt isolated experiments.

Rationale: The team needed tests to reflect actual multi-prompt, tool-using production behavior.

Gotchas (what bit them)

Provider playgrounds are insufficient for customer-ready prompt development.

Root cause: Production prompts depend on internal code, chain orchestration, model configuration, dynamic data, and environment requirements.

Resolution: Launch notebooks from the production Python repository so they access the same code, libraries, dependencies, and chains.

Prompt comparisons become unreliable if users do not share a baseline model configuration.

Root cause: Different OpenAI, internal, and public model configurations can produce output differences unrelated to prompt changes.

Resolution: Maintain configurations in code and make them quick to launch in the prompt engineering setup.

Dynamic prompt placeholders make manual testing unscalable.

Root cause: Prompts need company-specific values fetched and injected for each test company.

Resolution: Use Jinja templates and chains that automatically load context and replace placeholders.

External search results cannot always be cached for prompt testing.

Root cause: The LLM dynamically creates web-search queries, so the required external source data is only known at runtime.

Resolution: Automate live Bing Search API access inside the prompt-engineering framework.

Prompt work can drift from production behavior.

Root cause: If playground chains differ from the deployed backend, prompt improvements may not transfer cleanly.

Resolution: Run notebooks against the same repository code and chains so local behavior matches production behavior.

Evaluation

LinkedIn evaluates prompt changes with representative and regularly refreshed test datasets, production-equivalent chain execution in notebooks, live end-user prompt sessions, tracked real usage data, and code review for notebook/test-data changes.

Metrics: representative real-world test data · diverse company coverage by industry, size, country, and ownership type · real usage data sampled through tracking · prompt iteration within days · interactive and instant prompt-run feedback · two-hour to five-minute AccountIQ research reduction

Outcomes

• The notebook playground enabled seamless collaboration across engineering and domain teams.
• AccountIQ automated company research and reduced a two-hour workflow to five minutes.
• Prompt iteration could involve cross-functional users within days with interactive feedback.
• Engineers could focus on prompt structure, multi-prompt orchestration, and product integration while domain experts iterated on behavior.

The strongest numeric outcome is AccountIQ's reduction of company research from two hours to five minutes; the source also reports prompt-iteration collaboration within days and setup in a single or few commands.