Overview of our approach to train a cuneiform sign detector to localize and classify cuneiform signs in tablet images. The sign annotations necessary for training the sign detector are automatically generated by localizing signs of existing transliterations in their tablet images. This alignment turns weak supervision of the transliteration into full supervision in terms of bounding boxes.
Detecting cuneiform signs requires discrimination of fine-grained details. Here the cuneiform signs are shown in their idealized form as characters of a Unicode font.
Detection results for a specific class of cuneiform sign known as "LA".
A web application for cuneiform sign detection has been developed as part of this project.
Cuneiform Sign Detection
Cuneiform clay tablets are the oldest written records of humankind providing invaluable insights into our ancient history. Our group has developed a cuneiform sign detector that can localize and classify cuneiform signs in images of clay tablets by training a deep learning-based model without requiring manual sign annotations.
Our approach leverages existing transliterations (sign-by-sign representation of tablet content in Latin script) in order to supervise the sign detector training. Since a transliteration does not provide the absolute position of cuneiform signs in a tablet image, our method learns how to align hundreds of tablet images and corresponding transliterations in parallel in order to generate localized annotations for detector training.
This is significant because the proposed method shows how to perform weakly supervised learning in a difficult fine-grained detection setting. To the best of our knowledge, this is the first work to demonstrate strong sign detection for over a hundred sign code classes evaluated on a large and diverse collection of tablet images. Since our method is applicable to cuneiform script of different periods, it enables large-scale digital analysis of cuneiform script in the field of Assyriology, thus helping to more efficiently open up the treasures of human civilization. To directly support Assyriologists in their analysis of cuneiform texts, we also present a web application of the cuneiform sign detector, thus making it easily applicable.
Resources
Demo System - please send an email to this address to request login data.
Materials and Methods
Full details about the method, its implementation, and a detailed performance analysis can be found on the website of our article at PLOS ONE.
We provide the dataset and the implementation of our method in two public repositories. The cuneiform sign detection dataset is available in this data repo and the code and pre-trained models of the cuneiform sign detector are available in this code repo. In a third repository, we also provide the code for the web application of the cuneiform sign detector which is available in this web app repo.
For demonstration purposes, we also host an instance of the web application as a demo system. If you would like to test the web application, please send an email to this address to request login data.
Publication
Dencker, Tobias, Klinkisch, Pablo, Maul, Stefan M., and Ommer, Björn (2020): Deep Learning of Cuneiform Sign Detection with Weak Supervision using Transliteration Alignment, PLOS ONE, 15:12, , pp. 1–21 URL: https://doi.org/10.1371/journal.pone.0243039