Armenian OCR: the engines, the numbers, and our first training run

Ten human-proofread encyclopedia pages, ten OCR configurations measured against them, and the first honest numbers from a recogniser we are training ourselves — with the code, data and weights all published in the open

30 August 2026 · 13 min read · Stephen Masters

Tetrak’s central idea is that you should not have to pick an OCR engine. The pipeline fans a document out across every local engine eligible to read it, scores the transcripts against each other without a reference, and keeps the best one. On our English corpus of Los Angeles stage and picture-palace ephemera that works: different engines win on different documents, and routing beats any single choice. The research is the measured version of that argument.

Fan-out only works if there is something to fan out to and for Armenian there is very little to choose from. Of the six local engines Tetrak can drive, only two read the script at all, and one of those needs a model file that does not ship with it. So before we could claim anything about Armenian, we had to find out what the field actually contains and measure it.

We did that yesterday. This is what we found, including the parts that do not flatter us.

What we measured, and how

Ten pages of the Armenian Soviet Encyclopedia, volume 1 (1974), from Armenian Wikisource — dense two-column 1970s letterpress, Eastern Armenian, reformed orthography.¹ Every page carries a transcript that has passed two human reviews on Wikisource, which makes this the first corpus in the project with genuine human ground truth rather than ground truth generated by a model.

The volume scan was rendered per page to JPEG at 2048px. Scoring used the same two metrics as the main benchmark: character similarity, which is order-sensitive, and word recall, which is not. The gap between the two is the interesting part, and we come back to it.

Who is building Armenian OCR

Very few people, and the picture changed substantially two months ago.

Calfa (Paris; research lead Chahan Vidal-Gorène, École Nationale des Chartes) is the serious open contributor. They released a Tesseract model for Armenian in April 2025 — hye-calfa-n, trained on twentieth-century Armenian print with a stated focus on damaged documents and old fonts, covering Classical, Western and Eastern Armenian. In July 2026 they published two more things: hye-paddle, a PP-OCRv6-tiny recognition model in PaddleOCR format, which they describe as more accurate but slower than the Tesseract one; and hye-open-ocr, a whole printed-Armenian pipeline with layout detection, reading order, ALTO XML and searchable PDF output. All three are CC BY-NC 4.0.

Portmind published armenian-ocr, a CRAFT-plus-recogniser pipeline whose modified version is used by the National Library of Armenia. We could not measure it: the README points at a single S3 archive, and the link has been reported dead in the project’s own issue tracker since January 2025. Our own attempt to fetch it failed too. Without the weights there is nothing to run.

Stock Tesseract ships hye.traineddata, trained on synthetic rendered text in modern fonts. Marker, through Surya, lists Armenian in its language table. Claude reads the script. Beyond that: EasyOCR does not list Armenian, PaddleOCR’s 106-language multilingual release does not include it, and Apple Vision on this machine reports thirty languages, none of them Armenian.

The numbers

Averages over the ten pages. Seconds are the total for all ten, on an Apple-silicon laptop.

Everything else clusters between 0.025 and 0.048 on character similarity — stock Tesseract without a language flag, Apple Vision, stock EasyOCR and stock PaddleOCR. That is noise. It is what “no Armenian support” looks like when you measure it rather than assume it, and it is the gap in Tetrak’s ability to fan-out for Armenian script.

Four things stand out.

Calfa’s Tesseract model is the local state of the art, by a wide margin. It is 0.16 ahead of stock hye on character similarity and ahead on all ten pages, reaching 0.97 on the cleanest one. Their claim about training on twentieth-century print is visible in the results.

A small purpose-trained local model currently edges the frontier cloud model. Calfa’s Tesseract model beats Claude on character similarity, on seven of the ten pages, in fifty-five seconds against Claude’s thirty-four minutes and at no marginal cost. This also calibrates our own English figures honestly: in the main Tetrak benchmark Claude is the reference that everything else is scored against, so its column reads around 0.90 by construction. Measured against human truth on a script it knows less well, it reads 0.77.

tessdata_best buys nothing over tessdata_fast for Armenian. 0.629 against 0.644 — fractionally worse, within noise. For some languages the variant matters. Here, anyone benchmarking “Tesseract on Armenian” is measuring effectively the same model either way.

Tetrak’s own auto-configuration actively harms these pages. 0.259 against 0.644 for plain tesseract -l hye, with word recall flat at 0.713. That flat recall is the diagnosis: the engine is reading the same words and assembling them in the wrong order. Our auto-configuration bands were fitted to Los Angeles theatrical ephemera, and dense two-column encyclopedia text sits in exactly the region they mis-map. We knew the bands might not generalise. Now we know they do not, on a second corpus, and we have the measurements.

When one model beats another

The averages hide the most useful finding, which is that the two metrics disagree in a specific and informative way.

Marker/Surya has the best word recall of anything measured — 0.862 — and a mediocre 0.592 on character similarity. On page 108 that split is 0.24 against 0.88. A model that misread letters would lose on both. Losing only on the order-sensitive metric means Surya is recognising the words correctly and Marker is assembling them in an order the reference does not follow. So: for building a search index over an Armenian collection, Marker is already useful. For producing a reading edition, it is not, yet.

Calfa’s Tesseract model is the balanced choice, strongest on character similarity and close to the top on recall, which is what you want when the output is going to be read rather than only searched.

Claude leads narrowly on word recall and shows the same signature as Marker on its weaker pages — high recall with depressed similarity — so its difficulty on dense two-column text is layout, not letters. It is also the option that sends your collection off the machine, which for a lot of archives settles the question before accuracy does.

This is exactly the shape of result Tetrak’s fan-out exists to exploit. Nobody should have to know any of the above. The pipeline should run the eligible engines and keep the best transcript per page, and once the router is language-aware, “the eligible engines” for Armenian will include Calfa’s models alongside anything else worth adding.

Our own model, and where it actually stands

EasyOCR is the gap we decided to close. It is Apache 2.0, Tetrak already drives it, it is CRAFT detection feeding a compact CTC recogniser — the same architecture family that works for Armenian elsewhere — and it has a documented mechanism for loading a custom recognition network. Detection needs no training, because CRAFT is script-agnostic. All the Armenian-specific work concentrates in one small trainable model.

So we are training one. tetrak-easyocr-armenian is the library: EasyOCR’s own generation2 architecture (VGG, BiLSTM, CTC head), the full Armenian alphabet, the և ligature, Armenian punctuation, digits and basic Latin for mixed pages. tetrak-hy-trainer is the pipeline that produces the weights. Both are Apache 2.0.

It is very early days. We have done one training run. Here is what it did.

The v0 model trained overnight — 150,000 iterations, 8.7 hours on an M-series Mac — on synthetic Armenian word crops rendered in Noto Armenian faces. On its own held-out synthetic validation set it scores 99.7%.

That number is worthless, and it is worth explaining why we are printing it anyway. A validation set drawn from the same crisp synthetic renderer as the training set measures whether the model learned the renderer. It says almost nothing about scans. The honest number is the one from running the packaged model through stock EasyOCR against the real encyclopedia pages:

Read that plainly. We are at 0.079 where Calfa are at 0.799. On this material their model is an order of magnitude better than ours, and any framing that dressed 0.079 up as progress against them would be dishonest.

What the run does show is that the thing is alive. Word recall of 0.237 is ten times stock EasyOCR’s 0.023, against a baseline that is pure noise. The model is finding Armenian text on a page and reading a quarter of the words back correctly — from a standing start, where the engine previously had no concept of the script at all. That is the difference between a model that has learned something and a model that has not.

The gap between 99.7% synthetic and 0.079 real is a domain gap, and a fairly diagnosable one. v0 trained on crisp single-word crops at generous size. Real pages are degraded, line-shaped, and much smaller in x-height. The v1 recipe addresses all three: line samples of one to four consecutive tokens drawn from roughly 1,190 harvested proofread pages across five encyclopedia volumes (published, along with the crops, as a Hugging Face dataset — more on that below), crop sizes extending down to the 18px that real page renders actually produce, and a degradation pipeline — downscale cycle, blur, tone, rotation, JPEG round-trip — applied to the validation set as well as the training set. That last one is the lesson from v0: a validation set that never sees a degradation is not measuring anything you care about.

There is one thing our model already has that Calfa’s does not, and it is not accuracy. Calfa’s releases are CC BY-NC 4.0. Ours is Apache 2.0, which means it can be bundled, embedded and used commercially without a licence conversation. If Calfa’s models turn out to do the job better on every register we care about, then this project’s justification shrinks to “a permissively licensed alternative”. That is a real reason to build something.

Built in the open

Everything this article rests on is public, and that is a decision, not an accident.

The code is on GitHub: tetrak-easyocr-armenian, the library that loads the model, and tetrak-hy-trainer, the pipeline that trains it, and we are publishing both to PyPI as ordinary installable packages. The training data is on Hugging Face: tetrak/armenian-ocr-crops holds the 1,190 proofread encyclopedia pages with their Wikisource provenance — down to the exact revision of each page — and all 181,800 synthetic crops from the v0 run. So are the weights: tetrak/easyocr-armenian is the v0 model measured above, published with a provenance record naming the dataset revision that trained it, low score and all. The code and the weights are Apache 2.0 — use them, bundle them, sell things built on them, no licence conversation needed. The dataset is CC BY-SA, the same terms as the encyclopedia text inside it.

One consequence surprised us. The Hub has Armenian datasets for speech, paraphrase and sentiment, and Calfa publish admirable open ground truth for Arabic manuscripts and historical Chinese — but as far as we can find, nobody had published a dataset for training Armenian OCR there before this one. We would genuinely prefer that not to be a distinction. It should not be possible to be first at this in 2026.

Which is the point of doing it this way. Armenian OCR is not a market to be cornered; it is a small field where every group that trains a model has so far had to start by building its own data. If our pages, crops, recipes and honest benchmark numbers save the next group that start, and their work improves on ours, the tools the archives end up with get better for everyone — and Tetrak, which fans out across every engine it can drive, benefits from every one of those improvements by construction. Openness here is not a virtue we are performing; it is the mechanism by which the quality of Armenian OCR as a public good actually rises.

On publishing numbers we do not like

A benchmark you only publish when you win is marketing. We would rather run this the other way round.

The commitment is straightforward. We publish the whole table, including the rows where our own components lose — such as the auto-configuration result above, which says a feature we built makes a specific class of document worse. We publish our model’s real score against a competitor that beats it by a factor of ten. When something we have unmeasured is likely to beat our best number, we say so rather than leaving it out. If and when our model does turn out to be better at some class of document, we will report that with the same evidence and the same caveats.

This is not only a principle, it is how the product works. Tetrak fans out and keeps the best result. A better model from anyone — Calfa, Portmind, a research group in Yerevan — makes the pipeline better, so there is no version of this where we benefit from talking down someone else’s engine. The useful question is never “whose model is best” but “which model is best for this page”, and the only way to answer that is to measure everything honestly and let the router decide.

Next

The archival-register fixtures are being gathered now, and the same table run against those is the one that will actually decide anything — clean encyclopedia print is not what the shelves hold. hye-paddle and hye-open-ocr need measuring. The v1 training run is queued. And Tetrak itself needs a --language option before any of this reaches the pipeline properly: today the quality scorer is English-only, which means auto-local would rank Armenian transcripts on noise, and the triage gate would hold back every file.

We will report all of it here, including whatever the numbers turn out to be.

• armenian
• ocr
• benchmarks
• project-notes
• open-source