Enhancing Workplace Assessment Tools with RAG and Vector Search

thomas · HR · 2024 · source Agentic: no · promotional bias: high · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Workplace assessment content retrieval

Thomas needed to modernize psychometric assessment delivery and content access because its paper-era, manually interpreted content model did not scale to broader customer access or modern workflows.

Constraints:

Thomas' legacy assessment model was paper-based and designed for one-on-one interpretation.
The legacy platform contained millions to billions of words of personalized content variants.
Users needed easier access to the right content instead of long static reports.
Sensitive people-science data required ethical handling, security, and transparency.
New tools needed to integrate into modern workplace platforms such as Microsoft Teams.

Architecture

Thomas adopted Databricks to improve data workflows from ingestion through analysis and to add GenAI capabilities through Databricks AI Search, Vector Search, and RAG. The system lets users ask natural-language questions over a large unstructured content base, retrieve relevant assessment material, and generate tailored insights that can be delivered through products such as Perform and integrations such as Microsoft Teams.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  legacy-psychometric-content["Legacy psychometric content"] --> databricks-data-workflows["Databricks data workflows"]
  databricks-data-workflows --> databricks-ai-search["Databricks AI Search"]
  databricks-ai-search --> rag-insight-generator["RAG insight generator"]
  rag-insight-generator --> perform-product["Perform product"]
  perform-product --> microsoft-teams-integration["Microsoft Teams integration"]
  microsoft-teams-integration --> customer-users["Customer users"]
  customer-users --> legacy-psychometric-content

Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component	Role	Interacts with
`Legacy psychometric content`	Large corpus of assessment interpretation content and personalized variants.	`Databricks data workflows`
`Databricks data workflows`	Ingest, transform, analyze, and secure Thomas data and content.	`Databricks AI Search`
`Databricks AI Search`	Vector/RAG retrieval layer for finding relevant assessment content.	`RAG insight generator`
`RAG insight generator`	Prompts LLMs with retrieved content to generate tailored insights from unstructured data.	`Perform product`
`Perform product`	New AI-enabled product foundation for personalized assessment insights.	`Microsoft Teams integration`
`Microsoft Teams integration`	Makes Thomas services available inside customer workflows.	`Customer users`
`Customer users`	Employees, HR leaders, and managers querying for people-science insights.	`Legacy psychometric content`

Data flow

Legacy psychometric content and assessment data are ingested and transformed in Databricks. AI Search and RAG retrieve relevant unstructured content, LLM prompting generates tailored insights, and the resulting content is delivered through Thomas products and workflow integrations.

“improving their data workflows — from ingestion and transformation to analysis and solutions for customers and employees.”article, stated

“gives Thomas the ability to rapidly get insights from their extensive database.”article, stated

Why this architecture

Thomas chose a Databricks-backed RAG and AI Search architecture because it needed to turn a massive static content base into interactive, personalized, explainable insights without compromising security or ethical commitments.

“Databricks AI Search sparked a breakthrough for us. We could give our clients the ability to find the answers they need, rather than throwing a 40- to 50-page report at them.”article, stated

“With Databricks, GenAI is not a black box. We can talk to our customers and take them step-by-step through how we got the insights”article, stated

Models used: Databricks AI Search · Databricks Vector Search · RAG · GenAI · NLP

Scale

Thomas has worked in people science for over 40 years.
“For over 40 years, Thomas’ central ethos has been that companies can elevate job satisfaction and productivity”article, stated
A user profile is completed with Thomas every 90 seconds.
“Because a user profile is completed with Thomas every 90 seconds”article, stated
The legacy platform contained millions to billions of words of content.
“Our legacy platform had millions — to the point of billions — of words of content”article, stated
The legacy output could be a 40- to 50-page report.
“rather than throwing a 40- to 50-page report at them.”article, stated
Thomas went from proof of concept to MVP in weeks.
“we were able to go from proof of concept to minimum viable product in weeks.”article, stated
Thomas integrated GenAI into three platforms within three months.
“Thomas has been able to integrate GenAI into three different platforms within the last three months.”article, stated

Alternatives considered (roads not taken)

Continue paper-based, one-on-one interpretation.

Rejected because: The traditional model did not scale as Thomas grew its customer base.

“Thomas’ foundational psychometric assessment model — rooted in paper-based processes and designed for one-on-one interpretations — became increasingly limiting”article, stated

Give customers static long reports.

Rejected because: Databricks AI Search let customers find answers interactively instead of reading 40- to 50-page reports.

“rather than throwing a 40- to 50-page report at them.”article, stated

Treat GenAI as a black box.

Rejected because: Thomas needed to explain how insights were produced and protect sensitive data.

“GenAI is not a black box. We can talk to our customers and take them step-by-step through how we got the insights”article, stated

Patterns

`rag-retrieval` (retrieval)

Thomas uses RAG through Databricks AI Search to retrieve relevant assessment content for generated insights.

“integration with generative AI tools like retrieval augmented generation (RAG) and natural language processing (NLP)”article, stated

`privacy-preserving-processing` (governance)

The Databricks platform provides secure data access and integration with existing security protocols.

“sensitive information is always protected, and data integrity is never compromised.”article, stated

`tool-use-orchestration` (workflow)

Thomas integrates AI-driven assessment processes into workflow tools such as Microsoft Teams.

“integrate their new, AI-driven processes into popular platforms like Microsoft Teams.”article, stated

Tradeoffs

Content access model

Chose Interactive AI Search and generated insights. over Manual navigation through long static reports..

Rationale: Interactive retrieval made content more dynamic and useful for customer-specific questions.

“This made our content incredibly more dynamic and interactive.”article, stated

Deployment environment

Chose Databricks secure data and AI platform. over Disconnected legacy data workflows..

Rationale: The platform supported scale, security, integration, and data integrity for sensitive assessment data.

“The platform’s built-in features for managing data access and its ability to seamlessly integrate with existing security protocols”article, stated

Gotchas (what bit them)

Static content variants became unmanageable.

Root cause: The platform contained every possible personalized wording variant, creating millions to billions of words.

Resolution: Use RAG and AI Search to retrieve and generate tailored insights dynamically.

“translating evergreen content into useful responses for each client became an absolute nightmare.”article, stated

Paper-era interpretation did not fit modern workflows.

Root cause: The assessment model was designed for one-on-one interpretation and disconnected from modern work applications.

Resolution: Digitize data workflows and integrate insights into customer platforms.

“they weren’t connected to the typical applications necessary within a modern work environment.”article, stated

Customers needed explainable GenAI over sensitive people data.

Root cause: Psychometric insights require trust, ethical commitments, and careful data handling.

Resolution: Use a secure Databricks environment and explain how insights are derived.

“ensuring their data is handled with the utmost care.”article, stated

Evaluation

The source reports product and operational evaluation through time-to-insight, faster MVP delivery, user satisfaction, engagement, and deployment across multiple platforms rather than publishing benchmark scores.

Metrics: User profile completed every 90 seconds · POC to MVP in weeks · Three platforms in three months · Increased user satisfaction and engagement

“This new iteration has increased user satisfaction and led to deeper engagement.”article, stated

“we were able to go from proof of concept to minimum viable product in weeks.”article, stated

Outcomes

Moved from paper-based interpretation toward interactive AI-powered assessment insights.
Made a massive content base searchable and generative through RAG and AI Search.
Reduced static report burden by letting customers query for needed answers.
Integrated AI-driven processes into Microsoft Teams and three platforms in three months.

Thomas cites over 40 years in people science, a profile completed every 90 seconds, millions to billions of words of content, POC-to-MVP in weeks, and three GenAI platform integrations in three months.

“GenAI through AI Search and retrieval augmented generation (RAG) further improved content searchability and automated content generation.”article, stated