Large-Scale OCR Processing of Academic Papers Using AI Coding Agents and Serverless GPU Infrastructure

huggingface · Tech · 2026 · source Agentic: yes · promotional bias: low · source-backed · extracted with prompt 0.5.6 (schema v2)

Generated from extraction.json — do not edit by hand. Every claim cites verbatim source evidence; stated means the quote directly asserts the claim, implied means it supports it indirectly.

Problem

Domain: Large-scale document OCR for paper-chat RAG

Hugging Face needed Markdown versions for thousands of arXiv papers so Paper Pages could support HuggingChat's chat-with-paper feature even when arXiv HTML was unavailable. The team had to choose an open OCR model, run it economically across tens of thousands of papers, avoid proprietary API costs, write orchestration code for serverless GPU jobs, store mutable daily paper outputs efficiently, and integrate the result back into Paper Pages.

Constraints:

• about 27,000 indexed papers lacked corresponding arXiv HTML pages
• the chat feature requires paper content converted to Markdown and fed to the LLM as context
• OCR output needed Markdown interleaved with HTML for images and tables
• the model had to be open and commercially usable
• single-GPU processing would take multiple weeks
• new papers are added every day, making git-versioned datasets a poor storage target
• the team needed lower cost than Chandra's proprietary API while avoiding persistent GPU infrastructure

Architecture

Hugging Face selected Chandra-OCR 2 using the OlmOCRBench leaderboard, then used OpenAI Codex Desktop as a coding agent to implement and operate the pipeline. Codex was pointed at the model card, Hugging Face CLI skill, Jobs documentation, and the list of missing arXiv IDs. It wrote scripts to run Chandra-OCR-2 with vLLM on Hugging Face Jobs serverless GPUs, ran small-scale GPU comparisons over 120 papers, selected L40S over A10G based on throughput and total cost, launched 16 parallel L40S jobs, monitored progress, switched output from a dataset to mounted Hugging Face Buckets via hf-mount, and merged 16 buckets into a single bucket. Paper Pages then used the OCR Markdown for HuggingChat context.

Diagram confidence: reconstructed (reconstructed from prose).

flowchart LR
  pages["Paper Pages"] --> chat["HuggingChat paper chat"]
  missing["Missing arXiv ID list"] --> codex["Codex coding agent"]
  leaderboard["OlmOCRBench leaderboard"] --> chandra["Chandra-OCR 2"]
  codex --> chandra
  codex --> scripts["OCR job scripts"]
  scripts --> jobs["Hugging Face Jobs"]
  jobs --> vllm["vLLM runtime"]
  vllm --> chandra
  jobs --> l40s["L40S parallel jobs"]
  l40s --> buckets["Hugging Face Buckets"]
  scripts --> mount["hf-mount"]
  mount --> buckets
  buckets --> markdown["OCR Markdown bucket"]
  markdown --> pages
  markdown --> chat

Components

Data flow

Paper Pages identify arXiv papers without HTML versions. Codex-generated scripts feed the missing arXiv IDs to Chandra-OCR-2 running with vLLM on Hugging Face Jobs. Sixteen parallel L40S jobs process assigned paper subsets and write Markdown output through hf-mount to Hugging Face Buckets. The 16 buckets are merged into a consolidated bucket, then integrated into Paper Pages. HuggingChat uses the OCR-derived Markdown as paper context for chat-with-paper when the arXiv HTML-to-Markdown path is unavailable.

Why this architecture

The team used OCR rather than arXiv HTML because the affected papers lacked HTML pages. They chose Chandra-OCR 2 because it led the relevant open OCR leaderboard and had an OpenRAIL license. They used Hugging Face Jobs because OCR was a finite batch workload suitable for serverless pay-per-second GPUs. They selected L40S by total job cost rather than hourly price, because the cheaper A10G processed fewer papers per hour. Buckets plus hf-mount replaced git-versioned datasets because daily updates would create too many commits and mounted storage removed custom upload/download code.

Models used: OpenAI Codex · Chandra-OCR 2 · vLLM · HuggingChat · OlmOCRBench · Nvidia L40S · Nvidia A10G-large

Scale

• The final article title says the team OCR'd 30,000 papers.
  “How we OCR'ed 30,000 papers using Codex, open OCR models and Jobs”article, stated
• About 27,000 indexed papers lacked arXiv HTML pages.
  “about 27,000 papers indexed on Hugging Face do not have a corresponding HTML web page on arXiv”article, stated
• The GPU comparison used a 120-paper sample.
  “some comparisons on a small scale (120 papers) to see which GPUs to use”article, stated
• L40S processed about 60 papers per hour with at most 30 pages per paper.
  “about 60/hour when parsing at most 30 pages for each paper compared to 32/hour on the A10G”article, stated
• The team ran 16 parallel jobs.
  “it recommended to run 16 jobs in parallel”article, stated
• The full run was estimated at about 29-30 hours.
  “Running 16 parallel jobs would take about 29-30 hours.”article, stated
• The self-managed Jobs run was estimated at about $850.
  “It estimated the cost to be about $850.”article, stated
• The Chandra API comparison was $1,841.07 for fast/balanced and $2,761.60 for high-accuracy.
  “$1,841.07 for "fast/balanced" mode and $2,761.60 for "high-accuracy" mode.”article, stated
• All 16 jobs finished after about a day.
  “After about a day, all 16 jobs were finished.”article, stated

Alternatives considered (roads not taken)

Use only arXiv HTML-to-Markdown conversion

Rejected because: About 27,000 Hugging Face indexed papers had no corresponding arXiv HTML page.

Use Chandra's proprietary API

Rejected because: Self-managed Hugging Face Jobs with L40S GPUs were much cheaper in the reported estimate.

Run all papers on a single L40S GPU

Rejected because: A single L40S would take multiple weeks, so Codex recommended 16 parallel jobs.

Choose the cheaper-per-hour A10G-large instances

Rejected because: A10G was slower enough that total cost would be higher than L40S.

Write OCR results to a Hugging Face dataset

Rejected because: Daily paper additions would create too many git commits, so Buckets were better for mutable storage.

Patterns

tool-use-orchestration (agentic-infrastructure)

Codex orchestrated model-card reading, CLI documentation, job-script generation, GPU experiments, job launch, monitoring, storage changes, and bucket merging.

eval-driven-development (model-selection)

The OCR model was selected from OlmOCRBench evaluation results rather than ad-hoc model choice.

cost-aware-routing (infrastructure)

Codex compared GPU choices and API pricing, then chose 16 L40S jobs because total runtime and cost beat A10G and proprietary API options.

batch-offline-processing (document-processing)

The missing papers were processed as a parallel serverless batch workload over 16 GPU jobs and merged afterward.

rag-retrieval (application)

The OCR output became Markdown context for HuggingChat's chat-with-paper feature.

Tradeoffs

OCR model selection

Chose Chandra-OCR 2 from OlmOCRBench over Manual OCR model trial-and-error.

Rationale: OlmOCRBench provided a relevant benchmark and Chandra-OCR 2 was the top model at the time with a commercial-use license.

Execution platform

Chose Hugging Face Jobs serverless GPUs over Persistent self-managed GPU cluster.

Rationale: The workload was a finite batch job, and Jobs charged by seconds used across CPU/GPU options.

GPU choice

Chose 16 L40S parallel jobs over A10G-large jobs or a single L40S.

Rationale: L40S was faster than A10G and 16-way parallelism reduced a multi-week single-GPU run to about 29-30 hours.

Storage target

Chose Mounted Buckets over Git-versioned Hugging Face dataset outputs.

Rationale: Buckets were more suitable for fast, cheap, mutable daily paper outputs, and hf-mount removed custom upload/download code.

Gotchas (what bit them)

A cheaper GPU per hour can cost more for a finite batch.

Root cause: A10G-large was cheaper hourly but processed fewer papers per hour than L40S.

Resolution: Run a 120-paper experiment and optimize for total workload cost, not hourly rate.

Git-versioned datasets are a poor fit for frequently updated OCR outputs.

Root cause: New papers are added every day, which would create a huge number of commits.

Resolution: Use Xet-backed Hugging Face Buckets for fast, cheap, mutable storage.

Custom upload/download code adds complexity for agent-generated scripts.

Root cause: Remote storage synchronization would otherwise need explicit download/upload implementation.

Resolution: Mount Hugging Face Buckets with hf-mount so scripts write as if to local files.

Benchmark-leading OCR does not automatically guarantee downstream chat quality.

Root cause: The source selects by leaderboard but does not report post-OCR user-facing chat quality checks.

Resolution: Use the leaderboard as a selection gate, but treat downstream chat quality as a separate validation need.

Parallel batches can still have uneven completion times.

Root cause: Some assigned papers had more pages to parse than others.

Resolution: Monitor the 16 jobs and wait for all to complete before merging buckets.

Evaluation

The workflow used leaderboard-based OCR model selection, small-scale GPU throughput/cost experiments on 120 papers, job-completion monitoring through Codex, and economic comparison against proprietary API modes. The source does not describe a post-run OCR quality audit beyond the benchmark-driven model choice.

Metrics: OlmOCRBench model rank · 120-paper GPU comparison sample · papers per hour by GPU · parallel job count · estimated runtime · estimated infrastructure cost · API cost comparison · job restart count · completed job count

Outcomes

• The team OCR'd the missing paper corpus into Markdown using Chandra-OCR 2 on Hugging Face Jobs.
• Sixteen L40S jobs completed successfully after about a day, with no restarts reported.
• The self-managed run was estimated around $850, lower than the quoted proprietary API alternatives.
• The 16 output buckets were merged into a single bucket and integrated into Paper Pages.
• Hugging Face users can now chat with papers on the hub even when arXiv HTML is unavailable.
• The source reports the code and resulting bucket as public resources.

The source reports 30,000 OCR'd papers in the title, about 27,000 missing HTML papers, a 120-paper GPU test, L40S throughput around 60 papers/hour versus 32/hour on A10G for up to 30 pages per paper, 16 parallel jobs, about 29-30 hours estimated runtime, about $850 estimated cost, and API comparison costs of $1,841.07 and $2,761.60.