Text-to-Floor Plan Generation Using LLMs with Prompt Engineering and Fine-Tuning

zuru · Tech · 2025 · source Agentic: no · promotional bias: med · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Architecture design automation and floor-plan generation

ZURU Tech needed Dreamcatcher users to describe buildings in natural language and receive accurate 2D floor plans, while preserving instruction adherence and mathematical correctness for dimensions, positions, and room orientation.

Constraints:

Architecture

ZURU and AWS evaluated a text-to-floor-plan system using Claude 3.5 Sonnet prompt engineering through Amazon Bedrock and Llama 3.1 8B fine-tuning through SageMaker. Supporting architecture included Bedrock Knowledge Bases, OpenSearch Serverless retrieval, Mistral 7B for feature understanding, and an evaluation framework for instruction adherence and mathematical correctness.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  natural_language_design_request["Natural-language design request"] --> feature_understanding_model["Feature-understanding model"]
  feature_understanding_model --> bedrock_knowledge_base["Bedrock Knowledge Base"]
  bedrock_knowledge_base --> floor_plan_generation_model["Floor-plan generation model"]
  floor_plan_generation_model --> evaluation_framework["Evaluation framework"]
Drag to pan · scroll or use +/− to zoom · reconstructed from the source's described component topology

Components

Component Role Interacts with
Natural-language design request User description of desired building layout. Feature-understanding model
Feature-understanding model Mistral 7B step that extracts unique house features. Natural-language design request, Bedrock Knowledge Base
Bedrock Knowledge Base Retrieval layer backed by OpenSearch Serverless. Feature-understanding model, Floor-plan generation model
Floor-plan generation model Claude 3.5 Sonnet prompt-engineered path or fine-tuned Llama 3.1 8B path. Bedrock Knowledge Base, Evaluation framework
Evaluation framework Scores instruction adherence and mathematical correctness across test cases. Floor-plan generation model

Data flow

A natural-language building description is decomposed into house features, enriched with retrieved examples or metadata, sent through a prompt-engineered or fine-tuned model, and evaluated for instruction adherence and mathematical correctness against a deduplicated test set.

4 supporting quotes
“The challenge was enabling non-technical users to describe buildings in natural language and receive accurate 2D floor plans.”summary, stated
“The solution leveraged Amazon Bedrock for prompt engineering workflows and Amazon SageMaker for fine-tuning experiments”summary, stated
“For the first step (understanding unique house features), Amazon Bedrock provides a serverless API-driven endpoint for inference using Mistral 7B”summary, stated
“The second step uses Amazon Bedrock Knowledge Bases backed by Amazon OpenSearch Serverless as a vector database.”summary, stated

Why this architecture

The system separates feature understanding, retrieval, generation, and evaluation because floor plans require both interpreting user intent and satisfying geometric constraints.

“This framework measured model outputs based on two key metrics: instruction adherence (how well the generated floor plan matches the user's requested features) and mathematical correctness”summary, stated
“Having this evaluation framework in place was essential for enabling fast R&D iteration cycles and making data-driven decisions”summary, stated

Models used: Claude 3.5 Sonnet · Llama 3.1 8B · Mistral 7B

Scale

Alternatives considered (roads not taken)

GPT-2 baseline

Rejected because: It provided initial promise but was outperformed by Claude prompt engineering and Llama fine-tuning.

“experimentation with a GPT-2 LLM showed promising results. This established their baseline for comparison”summary, stated

Prompt engineering only

Rejected because: It matched instruction adherence gains, but fine-tuning additionally improved mathematical correctness.

“Both approaches achieved a 109% improvement in instruction adherence over the baseline, while full fine-tuning additionally improved mathematical correctness by 54%.”summary, stated

Patterns

fine-tuning-adaptation (model adaptation)

ZURU fine-tuned Llama 3.1 8B for floor-plan generation and improved mathematical correctness.

“fine-tuning Llama 3.1 8B models.”summary, stated

rag-retrieval (retrieval)

Bedrock Knowledge Bases and OpenSearch Serverless retrieved supporting context for the generation pipeline.

“The second step uses Amazon Bedrock Knowledge Bases backed by Amazon OpenSearch Serverless as a vector database. This enables metadata filtering and hybrid search to retrieve the most relevant record identifiers.”summary, stated

eval-driven-development (quality)

Instruction adherence and mathematical correctness metrics guided model and prompt choices.

“This framework measured model outputs based on two key metrics: instruction adherence (how well the generated floor plan matches the user's requested features) and mathematical correctness (proper dimensions, positioning, and orientation of rooms).”summary, stated

Tradeoffs

Use Bedrock prompt engineering and SageMaker fine-tuning experiments.

Chose Compare prompt-engineered Claude and fine-tuned Llama over Single-model path.

Rationale: The comparison showed both adherence gains while distinguishing fine-tuning's mathematical correctness benefit.

“Both approaches achieved a 109% improvement in instruction adherence over the baseline, while full fine-tuning additionally improved mathematical correctness by 54%.”summary, stated

Gotchas (what bit them)

Instruction adherence and geometry are separate quality axes.

Root cause: A model can follow requested features while still producing mathematically incorrect dimensions or positions.

Resolution: Track both instruction adherence and mathematical correctness.

“mathematical correctness (proper dimensions, positioning, and orientation of rooms).”summary, stated

Duplicate prompts waste evaluation compute.

Root cause: Test datasets can contain redundant instructions.

Resolution: Deduplicate prompts before running comparative evaluation.

“A prompt deduplication system identified and consolidated duplicate instructions in the test dataset, reducing computational overhead.”summary, stated

Evaluation

ZURU evaluated generated floor plans using instruction adherence and mathematical correctness, with prompt deduplication and distribution-based sampling.

Metrics: Instruction adherence. · Mathematical correctness. · 109% instruction-adherence improvement. · 54% mathematical-correctness improvement from fine-tuning.

“A critical element of this LLMOps implementation was the creation of a novel evaluation framework.”summary, stated
“A distribution-based performance assessment filtered unique test cases and promoted representative sampling through statistical analysis.”summary, stated

Outcomes

The case reports percentage improvements but not production usage volume.

“Both approaches achieved a 109% improvement in instruction adherence over the baseline, while full fine-tuning additionally improved mathematical correctness by 54%.”summary, stated