Building and Implementing a Company-Wide GenAI Strategy

thumbtack · Tech · 2024 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.4 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Company-wide GenAI adoption strategy

Thumbtack needed a company-wide GenAI strategy that could improve consumer products, transform operations, and boost employee productivity without duplicating effort or exposing sensitive data through unmanaged model usage.

Constraints:

Teams needed optionality across hosted open-source models and external vendors.
GenAI adoption required privacy-preserving infrastructure and policy updates.
Early hallucination risks had to be managed through human-in-the-loop use cases.
The strategy had to fit an 18-month planning horizon in a rapidly changing technology space.

Architecture

Thumbtack framed GenAI strategy around consumer products, internal operations, and employee productivity. It invested in ML infrastructure for LLaMA 2 and external APIs, added a PII-scrubbing layer to its inference framework, used policy-violation review as an early human-in-the-loop proof point, piloted GitHub Copilot, and drove adoption through leadership alignment and planning memos.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  genai-strategy[GenAI Strategy] --> leadership-alignment[Leadership Alignment]
  genai-strategy --> ai-usage-policies[AI Usage Policies]
  genai-strategy --> ml-infrastructure[ML Infrastructure]
  ai-usage-policies --> secure-inference-layer[Secure Inference Layer]
  ml-infrastructure --> secure-inference-layer[Secure Inference Layer]
  secure-inference-layer --> external-genai-apis[External GenAI APIs]
  secure-inference-layer --> product-teams[Product Teams]
  product-teams --> human-reviewers[Human Reviewers]

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`GenAI Strategy`	Sets company priorities across products, operations, and productivity.	`Leadership Alignment`, `AI Usage Policies`, `ML Infrastructure`
`Leadership Alignment`	Sponsors adoption through demos, planning memos, and periodic check-ins.	`GenAI Strategy`
`AI Usage Policies`	Track and approve LLM usage with IT, Platform, and Legal leaders.	`Secure Inference Layer`, `GenAI Strategy`
`ML Infrastructure`	Provides internal open-source model clusters and LLM deployment support.	`Secure Inference Layer`
`Secure Inference Layer`	Scrubs PII and securely accesses external GenAI APIs.	`ML Infrastructure`, `External GenAI APIs`, `Product Teams`
`External GenAI APIs`	Vendor models such as OpenAI used when appropriate.	`Secure Inference Layer`
`Product Teams`	Use GenAI for search, matching, policy review, support, and productivity pilots.	`Secure Inference Layer`, `Human Reviewers`
`Human Reviewers`	Review policy-violation cases where hallucination risk is mitigated by human judgment.	`Product Teams`

Data flow

A team identifies a GenAI use case, routes it through updated AI/ML policies, uses the shared inference layer for PII scrubbing and secure API access, chooses between internal open-source clusters and external vendors, then uses proof points and leadership check-ins to expand adoption.

“we built a new layer around our existing inference framework [4], that could scrub personally identifiable information (PII)”article, stated

“Our ML infrastructure team also created shared clusters for open source LLMs that data scientists could leverage”article, stated

Why this architecture

Thumbtack treated GenAI as an organizational capability rather than a single feature. Shared policy and inference infrastructure reduced bottlenecks, while early human-reviewed operational wins built momentum before broader product bets.

“we wanted to identify bottlenecks to gen AI adoption, and identify use cases of gen AI that could act as proof points for further investment.”article, stated

“picking some relatively straightforward problems which were less sensitive to gen AI issues like hallucination helped build momentum.”article, stated

Models used: GPT-4 · LLaMA 2 · OpenAI · GitHub Copilot · open-source LLMs

Scale

Thumbtack planned the roadmap over an 18-month horizon.
“we identified a set of user problems, a strategy for how we would invest in that space, and an illustrative roadmap of our investments for the next 18 months. We gathered feedback from stakeholders across different functions as we built alignment around the relative importance of different user”article, stated
The strategy began in summer 2023 and was assessed near the end of 2024.
“By the summer 2023, as OpenAI enabled access to GPT-4 and Meta released the [LLama 2”article, stated

Alternatives considered (roads not taken)

Rely only on external vendors.

Rejected because: Thumbtack wanted optionality across internal open-source models and external APIs.

“internally hosted open source models as well as the ability to work with external vendors (like OpenAI) in a secure, privacy preserving way.”article, stated

Let teams explore GenAI without new policies or infrastructure.

Rejected because: Exploration and productization would be limited without privacy, approval, and infrastructure frameworks.

“In order to drive gen AI adoption, we needed to create new policies and infrastructure frameworks around LLM usage (deployed internally and accessed via APIs), otherwise exploration and productization of gen AI would be limited.”article, stated

Start only with biggest product unlocks.

Rejected because: A straightforward human-reviewed policy-violation use case created early proof and momentum.

“this use case didn’t represent the biggest capability unlock on our roadmap. Thus picking some relatively straightforward problems”article, stated

Patterns

`privacy-preserving-processing` (governance)

Thumbtack added PII scrubbing and secure access to external GenAI APIs in its inference layer.

“that could scrub personally identifiable information (PII), and securely access external gen AI APIs.”article, stated

`human-in-the-loop-gate` (operations)

Policy-violation review remained human-in-the-loop, reducing risk from false positives and hallucinations.

“Since processes that review potential policy violations had humans in the loop, false positives and hallucinations were less of a concern”article, stated

`model-routing` (platform)

The strategy deliberately preserved optionality between hosted open-source models and external vendors.

“enable product teams to explore trade-offs in terms of performance, scalability and price”article, stated

`canary-rollout` (adoption)

Thumbtack used pilots and proof points such as GitHub Copilot and policy-violation review to demonstrate value.

“We decided to launch a pilot program for GitHub Copilot usage.”article, stated

Tradeoffs

Model access strategy

Chose Internal open-source clusters plus external APIs over A single vendor or single hosting model.

Rationale: Optionality let product teams explore performance, scalability, and price tradeoffs.

“working with internally hosted open source models as well as the ability to work with external vendors (like OpenAI)”article, stated

Adoption sequencing

Chose Early proof points in human-reviewed operations over Waiting for the largest roadmap unlock.

Rationale: Quick operational wins helped create organizational momentum.

“using gen AI to detect Thumbtack policy violations resulted in significant efficiency gains.”article, stated

Employee productivity rollout

Chose GitHub Copilot pilot over Unmeasured broad deployment.

Rationale: The pilot was intended to measure adoption and productivity signals before broader rollout.

“We wanted to measure both adoption and get signals around productivity improvements.”article, stated

Gotchas (what bit them)

GenAI adoption stalls without policy and infrastructure support.

Root cause: Teams need secure data access, approvals, and vendor access before productizing LLM use cases.

Resolution: Create AI/ML policies and a secure inference layer before pushing broader adoption.

“otherwise exploration and productization of gen AI would be limited.”article, stated

Hallucination risk can block early wins.

Root cause: Stakeholders may overestimate or underestimate adoption risks in a new technology cycle.

Resolution: Start with less sensitive, human-reviewed operational simulations to build confidence.

“some folks were underestimating just how much this would transform the company, and some were overestimating either what could be achieved”article, stated

Strategic planning can overfit a fast-moving model landscape.

Root cause: The technology was rapidly evolving and assumptions could become stale.

Resolution: Use a modest 18-month horizon and periodic leadership check-ins.

“We chose a time frame of 18 months because we wanted to be forward looking in terms of how we plan”article, stated

Evaluation

Thumbtack evaluated strategy through proof points, GitHub Copilot adoption and productivity signals, leadership check-ins, policy-violation efficiency gains, and iterative investment-level adjustments.

Metrics: Adoption · Productivity signals · Efficiency gains · Leadership progress · Investment-level changes

“measure both adoption and get signals around productivity improvements.”article, stated

“periodic check-ins with the leadership team, so that we can track ongoing progress and modify investment levels if necessary.”article, stated

Outcomes

Thumbtack created a company-wide GenAI strategy with three investment pillars.
The company built policy and infrastructure frameworks for secure LLM use.
Early human-in-the-loop policy-violation reviews produced efficiency gains and adoption momentum.

The strategy used an 18-month roadmap and reflected approximately summer 2023 through late 2024 adoption work.

“Having these proof points built confidence around the capability unlocks gen AI could achieve.”article, stated

“Having a relatively modest time horizon of 18 months also enabled us to make strategic investments”article, stated