Six OCR engines on a laptop RTX 4070: what I measured and what throughput hides
I measured six open OCR models on vLLM, on a laptop RTX 4070, to pick an OCR engine that runs on Linux. With one page at a time and the GPU idle, the time came out about three times longer. The pick was LightOnOCR-2-1B.
I needed an OCR engine that would run on Linux. On the Mac I had been using Vision, Apple’s native OCR.
The first survey of options had no measurements of my own: the performance numbers came from the vendors or from a test run on the Mac on 26/09. Before deciding, I wanted to look into options beyond the initial suggestions.
How I set up the test
The machine was a laptop with an RTX 4070 Laptop GPU (8 GB, about 95 W under load, 2.34 GHz, no throttling) and an i9-13900HX, in a WSL distro dedicated to the test, with 8 cores and 12 GB. I used vLLM, except for Unlimited-OCR-2, which ran on transformers, with one model at a time, because two don’t fit on the GPU.
The 126 pages came from a real case, in two samples of 63: a general one and another with only hard-to-read pages. Before the test, the images were downscaled to 1540 pixels on the longer side.
The reference was a transcription made by a vision model. It lets me compare how much each engine agrees with the reference, but it doesn’t tell the absolute accuracy of each one.
A few precautions in the method make the numbers comparable across engines:
- Client running on the laptop, with local traffic to vLLM. Measuring from the Mac, throughput would have measured the Wi-Fi: the network between the Mac and the laptop reached about 5 s per page just for upload.
- Cleanup before the metrics: a script strips HTML tags, <|…|> tokens and grounding boxes, so that HTML or Markdown output doesn’t pollute the Jaccard. On plain text, it changes nothing.
- Problem signals: the ratio between the letters in the output and those in the reference, on pages with at least 100 reference letters, flags excess when it goes above 2x and shortfall when it falls below 0.3x. I also count repetitions the reference doesn’t have and responses cut off at the 4096-token cap.
- Metrics: Jaccard, numbers and order (mean and median), pages with no useful text, and time per page.
- Warm-up: one page discarded per engine before measuring starts.
The one-page-at-a-time run
To keep things simple, some runs used concurrency 1 (conc 1): each request to vLLM carried one page, and only one request was in flight at a time. The test seemed to take all day, even with models like LightOnOCR-2, with 1 billion parameters. Unlimited-OCR took 18 minutes on the general sample and 26 on the hard one with conc 1, versus 7 minutes on the general sample with conc 8, that is, up to 8 pages at the same time. The vLLM log showed KV cache usage below 29%: the GPU had capacity to spare, and conc 1 was just serializing the work.
I recorded that throughput is measured with parallelism (8 pages in the 4070 test), because in the intended use the engine will receive several pages at the same time. Anyone who needs conc 1 to evaluate quality should run only a subsample and state in the report that that time is not the throughput with pages in parallel. And before concluding that an engine is slow, it’s worth checking the KV cache in the vLLM log. Below roughly 30% with conc 1, there is free space in the KV cache; that alone doesn’t indicate the GPU is idle, and the model isn’t necessarily heavy.
Here throughput with pages in parallel weighs as much as quality, because the question was whether renting a GPU is worth it for the throughput. On RunPod, an RTX 4090 starts at US$0.34 per hour on Community Cloud and US$0.89 on Secure Cloud, as of November 2026.
What the numbers showed
The table below shows, for each engine, the throughput with 8 pages in parallel, the Jaccard and the pages with no useful text on the hard sample, and the time for a single isolated page. PaddleOCR-VL led on throughput, at 1.02 pages/s.
| Arm | Throughput with 8 in parallel (pages/s) | Hard: Jaccard | Hard: pages without useful text (of 63) | Single-page time |
|---|---|---|---|---|
| PaddleOCR-VL-1.6, só o VLM | 1.02 | 0.798 | 24 | 2.8 |
| Surya 2.5 | 0.84 | 0.800 | 19 | 4.9 |
| LightOnOCR-2-1B | 0.48 | 0.811 | 10 | 4.7 |
| Chandra OCR 2, AWQ 4 bits da comunidade | 0.26 | 0.800 | 8 | 14.8 |
| Unlimited-OCR, FP8 | 0.14 | 0.036 | 22 | 5.3 |
Source: Pesquisa de OCR da versão web do Fuad Next, ensaio na RTX 4070 de notebook (03/11/2026)commit 72203f191d5dMethodologyDownload JSONDownload CSV
On agreement with the reference, Surya, LightOnOCR and Chandra came out ahead. LightOnOCR-2-1B had a median Jaccard of 0.909 on regular pages and 0.811 on hard ones. Unlimited-OCR-2 on transformers, at about 0.01 pages/s, is not viable in practice.
The chart below plots throughput against the Jaccard on the hard sample. Unlimited-OCR (FP8) was left out of it: with a Jaccard of 0.036 on the hard sample, it would flatten the scale for the other four, which landed between 0.798 and 0.811.
Throughput with 8 in parallel (pages/s)
Off the chart: Unlimited-OCR, FP8 (Hard: Jaccard 0.036, Throughput with 8 in parallel (pages/s) 0.14)
See the numbers in a table
| Arm | Hard: Jaccard | Throughput with 8 in parallel (pages/s) | Frontier |
|---|---|---|---|
| LightOnOCR-2-1B | 0.811 | 0.48 | on the frontier |
| Surya 2.5 | 0.800 | 0.84 | on the frontier |
| PaddleOCR-VL-1.6, só o VLM | 0.798 | 1.02 | on the frontier |
| Chandra OCR 2, AWQ 4 bits da comunidade | 0.800 | 0.26 | off the frontier |
| Unlimited-OCR, FP8 | 0.036 | 0.14 | off the chart |
Source: Pesquisa de OCR da versão web do Fuad Next, ensaio na RTX 4070 de notebook (03/11/2026)commit 72203f191d5dMethodologyDownload JSONDownload CSV
LightOnOCR-2-1B has 2 GB of weights. On vLLM 0.33 (torch 2.14, CUDA 13), with 8 pages in parallel, temperature 0 and a 4096-token cap, it did 0.48 pages per second and 4.7 s per isolated page, with the GPU between 97% and 99% and the KV cache at up to 29%.
I also measured Surya’s latency on 10 pages from the general sample: 5.89 s per page on average, with a median of 4.94 s.
The CPU track was dropped the same day, because the test is about GPU. It never got to measure throughput, and Docling didn’t run on the samples. The CPU numbers (Tesseract, RapidOCR and LiteParse) stand as a reference for agreement, not for throughput.
| Arm | Hard: Jaccard | General: Jaccard | Hard: pages without useful text (of 63) | Server time per page |
|---|---|---|---|---|
| RapidOCR PP-OCRv6 small | 0.800 | 0.875 | 14 | 3.80 |
| RapidOCR PP-OCRv5 latino | 0.781 | 0.875 | 15 | 2.10 |
| Tesseract best | 0.660 | 0.860 | 24 | 1.20 |
| Tesseract fast | 0.633 | 0.854 | 24 | 1.00 |
| LiteParse (Tesseract fast) | 0.613 | 0.876 | 28 | 0.73 |
Source: Pesquisa de OCR da versão web do Fuad Next, ensaio na RTX 4070 de notebook (03/11/2026)commit 72203f191d5dMethodologyDownload JSONDownload CSV
What was decided
I chose LightOnOCR-2-1B. PaddleOCR-VL-1.6 stayed as the alternative.
The test has a limit: throughput on a server GPU was not measured. The laptop 4070 is good for comparing quality, not for sizing production.
I measured the six models and closed out the GPU track report.
In an end-to-end load test done afterwards, the service with the chosen engine sustained 0.848 pages/s over 768 pages, with no failures at all.
Agent draft from the listed sources; reviewed and approved by Lucas Fuad