Texts in CroALa sometimes come from OCR (e. g. done by Google Books). Such texts may include unwanted homographs, or homoglyphs — Cyrillic a instead of Latin a, with further problems in searching, parsing, lemmatizing etc. To clean this up, an XSL stylesheet testing for Unicode blocks comes handy. See the Github Gist of the …
Continue reading “Testing for Unicode entities from certain blocks”