The engines

Six backends behind one interface. Every one exposes ocr_image(path) -> str, so they are interchangeable; resolve one by name through the registry rather than importing it directly.

See choosing a tool for which to use when, with the measurements behind it.

Tesseract, and auto-configuration

tesseract runs the engine with fixed settings. tesseract-auto calls analyse_image() first, which inspects the image and picks a contrast factor and page-segmentation mode per file.

That distinction matters more than it sounds. On the Carthay Circle postcard front, fixed settings score 0.00/0.00 and auto-configuration scores 0.49/0.86 — the same engine, on the same image.

EasyOCR and PaddleOCR

Both are deep-learning detector/recogniser pipelines, and neither reads PDFs — convert pages to images first.

On this corpus EasyOCR is middling (0.35/0.60) and PaddleOCR is weak overall (0.28/0.63) while winning two fixtures outright. That is a statement about this material rather than about PaddleOCR generally.

Both download model weights on first run, and hold them in memory afterwards, so the first call is slow and later ones are not.

Both join the auto-local pool for images, with EasyOCR ranked ahead of PaddleOCR on the scores above. EasyOCR was absent from that pool until recently: the original documentation described auto-local as using “paddle (or easyocr) for images”, but the code only ever offered PaddleOCR, so installing the EasyOCR extra bought nothing.

Adding it changed the measured outcome by nothing at all. EasyOCR wins the Carthay Circle front outright at 0.87, and still loses the selection there to a Tesseract transcript scoring 0.49, because the winner is ranked on quality weighted by word count. See results.

Marker

Layout-aware document conversion, and the only local backend that models reading order — which is why it wins the multi-column newsprint cover.

It is uneven: it also collapses on Grauman’s Chinese Theatre (0.09/0.00), where its layout model reads the whole postcard as a figure and returns nearly nothing. And it is slow: about 18 minutes of a 70-minute benchmark run on its own, and roughly 60 of those 70 once the two auto-local modes are counted, since both route to it.

Apple Vision

macOS only. pip install 'tetrak-ocr[vision]' — the extra pulls a PyObjC wrapper and nothing else, because Vision ships with the operating system. No weights, no download on first run, no network. Apple states that all of Vision’s processing happens on the device, so it sits inside the local-first guarantee exactly as Tesseract does.

On this corpus it is the strongest local engine measured, and the fastest worth using:

Eight images, the PDF excluded because Vision cannot read one. Sub-second to four seconds per image, against Marker’s minutes.

The word recall is the striking number: 0.78 beats auto-local’s 0.71, which runs several engines and picks the best of them. Vision finds more of the text than anything else local.

Where it fails is reading order. On the Inside Facts newsprint it recovers 0.88–0.90 of the words and scores 0.08–0.12 on character similarity — the widest gap between the two metrics anywhere in this benchmark, and the same multi-column failure described in the research, in a more extreme form than Tesseract manages.

Two limits worth knowing:

• Languages are queried at runtime, because Apple publishes no list. This machine reports 30 — Latin, Cyrillic, CJK, Thai, Vietnamese and Arabic. There is no Armenian, Greek, Hebrew, Georgian or Indic support.
• Handwriting is undocumented. Apple has never claimed it, and every published example is printed matter. A good score on a typed page says nothing about a manuscript one.

No preprocessing is applied: Vision’s neural detector handles rotation and perspective itself, so the contrast and page-segmentation work behind tesseract-auto has no equivalent here.

Claude

The Anthropic vision API. Reads material no local tool can — it recovers 0.97 word recall on the vaudeville poster, where the best local backend manages 0.39.

Three caveats. It needs ANTHROPIC_API_KEY and it sends your images to an API, which rules it out for confidential material. Its benchmark scores are measured against ground truth it generated itself, so they are a ceiling reference rather than a measurement — see method. And unlike every local backend, it does not reproduce between runs: on the poster its character similarity has scored 0.73, 0.24 and 0.26 across three runs while word recall held at 0.96–0.98. It recovers the same words and orders them differently, which is what an order-sensitive metric does to a document with no single reading order.

Multi-page TIFF

Every backend accepts .tif/.tiff, and always did. What none of them handled was the multi-page case, which archival TIFF often is — a scanned pamphlet or register arrives as one file with a frame per leaf.

Pillow opens such a file at frame 0 and reports nothing about the rest, so “TIFF support” meant transcribing the cover and discarding the body. A partial transcript is the worst failure available to this pipeline: it passes quality scoring, enters the archive, and looks complete to everyone who reads it afterwards. The Claude backend already guards against the same shape of bug with its max_tokens check.

So a multi-frame image is now either read in full or refused by name:

auto-local needed no special handling — it already treats a backend that raises as one that did not compete, so the complete transcript wins by default.

Not every extra frame is a page. TIFF’s NewSubfileType tag marks reduced-resolution renditions — pyramid levels and embedded thumbnails, which scanning pipelines emit routinely — and those are copies of a leaf already present, not further leaves. They are skipped, so a page plus its thumbnail counts as one page and is not refused by the frame-0 backends. Reading them would transcribe the same leaf twice: measured on the Carthay Circle postcard, appending a half-size level took the transcript from 32 words to 52.

Encoding costs more accuracy than paging does. CCITT G4 bilevel is the archival norm for text documents — tiny, and lossless for pure black-on-white. It is also where accuracy goes: the Hollywood playbill reads 297 words from its RGB original and 163 from a bilevel copy, a 45% loss, because thresholding to one bit destroys the antialiasing Tesseract uses to resolve small type. Converting back to RGB recovers nothing; the information is gone at source. If you control the scanning, keep greyscale.

Page settings are not adapted per frame. That is the conservative choice rather than a measured one: per-page auto-configuration lost badly on the PDF fixture, and there is no multi-page TIFF in the corpus to fit anything better against.

auto-local

Not an engine — a strategy. It runs every viable local backend (Marker where a GPU makes it practical, then EasyOCR, PaddleOCR and auto-tuned Tesseract for images; Marker plus Tesseract for PDFs), scores each transcript with a reference-free quality score, and keeps the best. Best local result on this corpus at 0.53/0.74, and the slowest, since it runs several engines per file.

It needs the qa extra: scoring uses a spell checker and GPT-2 perplexity. See auto-local routing.

The quality gate is separate from the strategy

If the winning transcript still scores below the threshold, auto-local raises and the batch pipeline diverts that file to triage/ with a manifest.

The same gate is available to any single backend:

tetrak-ocr batch --backend tesseract-auto --quality-gate

It is opt-in rather than always on, because turning it on changes which files reach processed/ and nobody running --backend tesseract today should suddenly find output diverted. It needs the qa extra for the same reason auto-local does — the flag checks that up front and refuses, rather than failing partway through a batch.

auto-local’s own manifest is richer: it already scored every engine it ran, so it can show the whole table. A single backend produces one transcript and one number.

The retired fast mode

There was a second strategy, auto-local-fast, which ran only the top-ranked eligible backend. The benchmark retired it. In every installed configuration it was at best equal to naming a cheap backend directly, and on a GPU machine the ranking put Marker first — so “fast” mode resolved to the most expensive engine available and cost twenty-six times tesseract-auto’s runtime for less accuracy.

The one thing it uniquely offered was a single cheap engine with the quality gate. That is what --quality-gate is for. See results for the measurement that decided it.