Building Fair Housing Guardrails for Real Estate LLMs: Zillow's Multi-Strategy Approach to Preventing Discrimination

zillow · Other · 2024 · source Agentic: no · promotional bias: low · source-backed · extracted with prompt 0.5.5 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Real estate conversational AI, fair housing compliance, and responsible AI

Zillow wanted to explore LLM-powered conversational real-estate experiences, but generic LLMs can produce discriminatory or steering responses when user prompts include protected-class characteristics. In real-estate search, lending, marketing, and transaction contexts, this can create Fair Housing Act, ECOA, and state/local anti-discrimination risk. Zillow therefore needed guardrails that could block both user inputs and model outputs that might facilitate steering while still allowing legitimate queries such as accessible-housing searches.

Constraints:

• Real-estate LLMs must comply with FHA, ECOA, and state/local anti-discrimination requirements.
• Protected classes include race/color, national origin, sex, familial status, religion, disability, age, marital status, income assistance, criminal background, and military status.
• The system must catch steering and other non-compliant content without over-blocking legitimate housing needs.
• Guardrail decisions must be fast enough to run inside the LLM application flow.
• The classifier must support tunable precision/recall because high recall protects compliance while false positives harm product experience.

Architecture

Zillow designed an LLM Fair Housing Guardrails system that combines three approaches: FHA compliance instructions in the LLM prompt, a syntactic stop list for unambiguous offensive terms, and a fine-tuned BERT-based sequence classifier for contextual fair-housing classification. The guardrail is exposed as a standalone service/API that validates both user inputs and system responses. It runs before the LLM as a pre-processing filter and after the LLM as a post-processing validator; flagged content is replaced by a predefined fair-housing message. The classifier was trained from real-estate query data, LLM-generated response data, legal/domain expert examples, sentence-level labels, weak labels reviewed by humans, and augmentation techniques.

Diagram confidence: explicit (topology described by the source).

flowchart LR
  real-estate-conversational-interface[Real-estate conversational interface] --> fair-housing-guardrails-service[Fair Housing Guardrails service]
  fair-housing-guardrails-service --> stop-list-matcher[Stop list matcher]
  fair-housing-guardrails-service --> fair-housing-compliance-classifier[Fair Housing Compliance classifier]
  fair-housing-guardrails-service --> guardrails-api[Guardrails API]
  guardrails-api --> llm-application[LLM application]
  guardrails-api --> pre-processing-validator[Pre-processing validator]
  guardrails-api --> post-processing-validator[Post-processing validator]
  pre-processing-validator --> llm-application
  pre-processing-validator --> predefined-compliance-response[Predefined compliance response]
  llm-application --> fha-compliance-prompt-instructions[FHA compliance prompt instructions]
  llm-application --> post-processing-validator
  post-processing-validator --> predefined-compliance-response
  predefined-compliance-response --> real-estate-conversational-interface
  fair-housing-compliance-classifier --> classifier-training-data[Classifier training data]
  classifier-training-data --> legal-and-domain-experts[Legal and domain experts]
  legal-and-domain-experts --> classifier-training-data
  data-augmentation[Data augmentation] --> classifier-training-data
  user-feedback-loop[User feedback loop] --> stop-list-matcher
  user-feedback-loop --> classifier-training-data
  fair-housing-guardrails-service --> user-feedback-loop

Components

Data flow

A user sends a real-estate conversational query. The Fair Housing Guardrails service first checks the input using stop-list matching and the Fair Housing Compliance classifier. The LLM prompt also includes FHA compliance instructions to improve the likelihood of compliant generation. After the LLM produces a response, the same service validates the output before display. If the input or output is flagged, a predefined fair-housing response is shown instead. User feedback and sampled data for human review update the stop list and add boundary examples for classifier retraining.

Why this architecture

Zillow combines prompt instructions, stop lists, and a classifier because each alone has a failure mode. Prompting is simple but over-cautious and non-deterministic; stop lists are deterministic but lack context; the classifier adds nuanced semantic decision-making and tunable precision/recall. Putting the guardrail both before and after the LLM creates a safety net for user requests and generated responses.

Models used: LLMs · Generative AI · BERT · Bidirectional Encoder Representations from Transformers · Sequence classification model · Binary cross entropy loss · Few-shot prompting

Scale

• Zillow frames the guardrail problem against hundreds of millions of online housing searchers across web and app experiences.
  “Given the hundreds of millions of people searching for housing online across our Web and apps experiences”article, stated
• The classifier requires fast inference because it runs as a step in the LLM reasoning flow.
  “the model must be fast to execute as we expect its decision making to be a step in the LLM reasoning flow.”article, stated
• The dataset included both query and response data, including 820 unique queries and 16,800 responses according to the ZenML summary.
  “The final dataset included 820 unique queries and 16,800 responses”summary, stated
• Sentence-level response data produced maximum precision lift around 0.6 recall.
  “precision lift is maximized when recall is around 0.6.”article, stated

Alternatives considered (roads not taken)

Prompt-only fair housing compliance.

Rejected because: Prompting was simple and high-recall, but became over-cautious and refused permissible queries that only resembled protected-class language.

Stop list as the sole compliance mechanism.

Rejected because: Strict lexical matching cannot distinguish discriminatory protected-class usage from legitimate terms like accessible housing or place names.

Classifier-only compliance.

Rejected because: Even though the classifier produced expected results on sample cases, Zillow says some scenarios may still be better handled by prompt guidance or stop lists.

Full-response labeling for generated responses.

Rejected because: Longer responses may contain only one non-compliant sentence, making full-response labels too coarse for the model to learn the offending text.

Static one-time training data.

Rejected because: Zillow expects feedback and sampled human review to expose new phrasings, contexts, false positives, false negatives, and decision-boundary examples.

Patterns

guardrails-io-filtering (safety)

Zillow runs fair-housing checks both before the LLM sees user input and after it generates output, replacing flagged content with a predefined compliance message.

safety-policy-enforcement (safety)

The system intentionally combines prompt instructions, stop-list lexical controls, and a trained classifier rather than relying on a single guardrail technique.

fine-tuning-adaptation (model-operations)

Zillow fine-tuned a BERT-based sequence classifier on domain-labeled real-estate examples to identify possible fair-housing violations with contextual nuance.

human-in-the-loop-gate (governance)

Legal and domain experts supply hand-crafted examples, sentence-level labeling guidelines, and review weak labels generated from few-shot prompting.

data-quality-loop (data)

User feedback and periodic human review identify false positives, false negatives, and decision-boundary examples used to improve the stop list and classifier over time.

synthetic-data-generation (data)

Because naturally collected data was mostly compliant, Zillow created non-compliant and long-tail examples by modifying compliant queries, generating responses, weak labeling, and applying augmentation.

eval-driven-development (evaluation)

Zillow evaluates prompt, stop-list, and classifier strategies with sample queries, dataset labels, precision/recall tradeoffs, and response-level training data before combining them.

Tradeoffs

Fair housing detection architecture

Chose combined prompt, stop-list, and classifier guardrails over choosing a single strategy.

Rationale: The combined design gets prompt expressiveness, stop-list explicit control, and classifier precision/recall tuning.

Classifier model requirements

Chose fast BERT sequence classifier over slower or less tunable compliance reasoning inside the main LLM alone.

Rationale: The classifier has to run as part of the LLM reasoning flow and allow precision/recall sensitivity tuning.

Response labeling granularity

Chose sentence-level labels over full-response labels.

Rationale: Sentence-level labeling tells the classifier which exact sentence is non-compliant in longer responses.

Stop list scope

Chose sparing use for worst offending terms over broad protected-class lexical blocking.

Rationale: Broad lexical blocking creates false positives for legitimate terms and place names, so nuanced cases need classifier/review handling.

Guardrail placement

Chose both pre-processing and post-processing checks over only checking input or only checking output.

Rationale: Checking both sides catches non-compliant user requests early and catches generated output before users see it.

Gotchas (what bit them)

Prompt-only compliance over-blocked legitimate accessibility queries.

Root cause: The prompt treated any semantic connection to protected classes as suspicious, producing high recall but poor precision.

Resolution: Use prompt guidance only as one layer and add contextual detection through a classifier.

Stop lists misclassify context-dependent protected-class terms.

Root cause: Strict lexical matching cannot tell whether a term such as disabilities or Indian is discriminatory, accessible-housing related, a place name, or a point of interest.

Resolution: Use stop lists sparingly for unambiguous offensive phrases and leave nuanced cases to machine learning plus review.

Full-response labels can hide the actual violating sentence.

Root cause: Long model responses may contain only one non-compliant sentence, so response-level labels make it hard for the classifier to associate the label with the offending text.

Resolution: Label generated responses at sentence level using legal-expert guidelines.

High recall can damage the user experience through false positives.

Root cause: Fair housing compliance prioritizes catching violations, but low precision makes the assistant refuse legitimate queries.

Resolution: Tune the classifier's precision/recall tradeoff and use feedback/review to improve boundary cases.

Naturally collected real-estate query data underrepresents violations.

Root cause: Most collected data was compliant, creating class imbalance for non-compliant examples.

Resolution: Sample protected attributes and discriminatory phrases, modify compliant examples, add legal/domain expert examples, generate responses, and augment the dataset.

Evaluation

Zillow evaluated the three strategies against fair-housing sample queries, analyzed precision/recall for non-compliant classification, trained a BERT sequence classifier on labeled domain examples, compared models with and without sentence-level response data, and used feedback and human review to identify false positives and false negatives.

Metrics: Precision and recall for the non-compliant label. · High recall for catching potential violations. · False positive rate as product-experience risk. · Precision lift from including sentence-level response data. · Classifier inference speed as an application-flow requirement. · Decision-boundary examples from human review.

Outcomes

• Zillow designed and deployed the combined Fair Housing Guardrails system internally for testing.
• The classifier method produced the expected results on the three sample test cases discussed in the article.
• The comprehensive system validates both user input and system output and replaces flagged content with a predefined message.
• Sentence-level response data improved classifier precision, with precision lift maximized around 0.6 recall.
• The standalone service can also be used for non-LLM NLP applications such as call transcript analytics.

Zillow frames the guardrails as necessary for web and app experiences used by hundreds of millions of online housing searchers. The article reports internal testing and system design rather than public production impact metrics.