Show HN: HieraticBench – Can AI read ancient Egyptian handwriting?
I'm obsessed with Egyptology. During covid, I wanted to get a hieratic tattoo. For those who don't know, hieratic is the cursive form of hieroglyphs that was used in day-to-day life in ancient Egypt. It's as old as hieroglyphs.<p>I then went on a mission to have a sentence translated into hieratic. Luckily, in 2022, I was able to commission an Oxford Egyptologist to write it for me. After that, I took it to a hieratic specialist I found on Instagram to redraw it with "biblical" accuracy.<p>When generative AI came out, I realized that models very rarely identify hieratic and almost never translate it correctly. Every single time a new model came out, I ended up testing my sentence and hieratic texts in general, and I realized this could actually become a benchmark. Since so many benchmarks are becoming saturated, we need to find weird ways to test new models.<p>That's why I decided to build HieraticBench.<p>The first version of the benchmark has 268 items, including real documents, signs, and other Egyptian scripts, which I used as controls, as well as my unpublished sentence in two different renditions. The current state-of-the-art foundation models are hilariously bad at identifying my sentence, especially the one the Egyptologist wrote with her pen. Some of them said Tibetan, Urdu, Korean, and, funnily enough, "Reformed Egyptian," which is from the Book of Mormon.<p>On real documents, the best model in the benchmark identified hieratic correctly 95% of the time, and the best score for reading individual signs was around 13%. That's when we tell the model specifically that it's hieratic. None of them can really translate a sealed sentence that's never been published and has no answer key.<p>I haven't run every model on every task just to keep costs down. My next steps were to run Astra and Gemini 3.1 Pro on real documents. Currently, only the Claude models have done the sign readings.<p>And just FYI, I have no experience whatsoever running benchmarks or evaluations, so not 100% sure I scored things right.<p>Things we need: - Running the models I haven't covered - More sealed sentences and labeled signs<p>If you know hieratic or have an Egyptologist friend, reach out at [email protected].<p>Source code: <a href="https://github.com/alymoursy/hieraticbench" rel="nofollow">https://github.com/alymoursy/hieraticbench</a><p>I'll be around to answer questions.
Wettbewerber & Alternativen
Show HN: PaperMono, e-ink fridge magnet shopping list with mobile web page
Fridge magnet shopping list on an M5Stack PaperMono (ESP32-S3, e-ink touchscreen), synced with a phone web app over Wi-…
Web Dashboard Decky
All your data, charts, websites in one dashboard. Refreshed automatically. - Decky brings a new level of efficiency and…
Anyvids: All-in-one AI video and image creation platform
Anyvids brings together top AI models and a complete creative toolkit in one place, so content creators, designers, and…
Diginyze - AI-Powered eCommerce Platform: AI-powered eCommerce Platform for Enterprises
Diginyze is an all-inclusive eCommerce suite that provides online businesses with a comprehensive range of tools and se…
HttpStatus.com: API testing and developer tools platform with 700
HttpStatus is an API testing, monitoring, and developer tools platform built for modern development workflows. With ove…