Rapid Development of AI-Powered Video Interview Analysis System

vericant · Education · 2023 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Admissions video interview summarization and review

Vericant had scaled video interview collection for schools, but admissions teams still had to watch 15-minute interviews to understand each applicant, creating a processing bottleneck across large applicant volumes.

Constraints:

• The company needed to reduce review time without losing useful qualitative interview signal.
• The initial prototype was built by a non-engineer CEO using low-code tools.
• Outputs needed human evaluation before production integration.
• The feature needed transparent AI positioning and beta disclaimers in the admissions portal.

Architecture

Vericant prototyped AI Insights with OpenAI Playground, GPT prompts, interview transcripts, and a Google Sheets evaluation workflow. The CEO iterated prompts in the Playground, pasted transcripts as user inputs, copied generated summaries, key points, and topics into spreadsheets, and had team members watch videos, read transcripts, and score multiple outputs. Once prompt quality improved, engineering integrated the prompt into the admissions portal as AI Insights.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  A["Interview videos"] --> F["Human evaluators"]
  A --> B["Interview transcripts"]
  B --> C["OpenAI Playground"]
  C --> D["GPT model"]
  C --> E["Google Sheets evaluation workbook"]
  F --> E
  E --> G["Prompt iteration loop"]
  G --> C
  G --> H["Engineering integration"]
  H --> I["AI Insights"]
  D --> I
  I --> J["Admissions teams"]

Components

Data flow

Interview transcripts are pasted into the OpenAI Playground with a system prompt for admissions-officer assistance. GPT generates summaries, key points, and topics. Outputs are copied into Google Sheets, evaluated by humans against the original video and transcript, and prompt changes are made based on scored feedback. The finalized prompt is handed to engineering and embedded in the admissions portal as AI Insights.

Why this architecture

The problem was well matched to prompt-based summarization: interview transcripts already existed, the desired outputs were summaries/key points/topics, and quality could be evaluated by admissions-domain humans. Using Playground plus Sheets let the CEO validate the prompt before spending engineering effort, then hand off a concrete artifact for production integration.

Models used: GPT · OpenAI Playground

Scale

• Vericant conducted tens of thousands of interviews.
  “we can do tens of thousands of interviews for you you don't have to do them”transcript, stated
• The target review time dropped from 15 minutes to 20-30 seconds.
  “understand what the contents of an interview in maybe 20 to 30 seconds instead of 15 minutes”transcript, stated
• The feature was deployed in approximately 30 days.
  “Approximately 30 days from initial concept discussion to deployed feature”summary, stated

Alternatives considered (roads not taken)

Require admissions officers to watch full videos

Rejected because: Watching each 15-minute interview did not scale for large applicant pools.

Wait for a full engineering build before validating the feature

Rejected because: The CEO could prototype and evaluate prompts without programming using Playground and Sheets.

Rely on one generated output per transcript

Rejected because: The evaluation loop ran each transcript three times to compare output variability.

Patterns

eval-driven-development (quality)

The team scored multiple outputs per transcript in Google Sheets before handing prompts to engineering.

prompt-versioning (prompt-operations)

Prompt revisions were driven by scored feedback and kept in Google Sheets during the prototype.

human-in-the-loop-gate (workflow)

Human team members watched videos and judged generated outputs before production integration.

structured-output-contract (output)

AI Insights produces named sections: summary, key points, and key topics.

Tradeoffs

Prototype with Playground and Sheets.

Chose OpenAI Playground plus Google Sheets over A custom engineering prototype from day one.

Rationale: The low-code path let the CEO validate prompts and quality before engineering integration.

Use human judgment rather than automated metrics for early quality.

Chose Team members scoring outputs after watching videos and reading transcripts over Purely automated summarization metrics.

Rationale: Admissions usefulness depended on human judgments about whether outputs were amazing, good, passable, or unusable.

Be explicit that the feature is AI-generated.

Chose AI Insights name and beta disclosure over A neutral assistant label that hides AI provenance.

Rationale: The product team believed transparency set user expectations and increased forgiveness for AI-generated content.

Gotchas (what bit them)

Initial prompt quality had visible failures.

Root cause: A prompt that seemed good still produced greens, yellows, and reds on real transcript evaluations.

Resolution: Collect detailed feedback, revise the prompt, and rerun evaluation until the second round had no red outputs.

The feature needed expectation management.

Root cause: Users may judge AI-generated admissions summaries differently if the AI nature is hidden.

Resolution: Name the feature AI Insights and include beta disclaimers and feedback requests.

Strategic direction was hard to define before hands-on experimentation.

Root cause: The team did not fully understand the product category until they built a concrete summarization example.

Resolution: Use low-overhead prototyping to make the opportunity concrete and clarify the business vision.

Evaluation

Vericant ran a human evaluation loop in Google Sheets: each transcript went through the same prompt three times; evaluators watched the video, read the transcript, scored outputs, added comments, and used the feedback to revise prompts until quality improved.

Metrics: Three outputs per transcript · Human video/transcript review · Amazing/Good/Passable/Unsatisfactory scores · Color-coded Google Sheets results · No red outputs in second round

Outcomes

• The feature reduced interview understanding from 15 minutes to about 20-30 seconds.
• The initial solution was built without programming and deployed in about 30 days.
• The project shifted Vericant's vision from scaling interviews to scaling admissions teams' ability to process interviews.
• The production feature became AI Insights in the admissions portal.

The source reports tens of thousands of interviews, 15-minute videos, 20-30-second review targets, a roughly 30-day deployment, and only a few hours of CEO time across weeks for initial prototyping.