Language complexity in human and machine translation: a preliminary study

Abstract

Systematic comparison between machine translation (MT) and human translation (HT) is mostly limited to the evaluation of MT output with HT as reference, as opposed to a more general study of the properties of MT and HT output texts. We present preliminary experiments investigating di erences between MT and HT in terms of readability and language complexity. We perform both quantitative and qualitative comparison of the outputs of machine and human translation, using samples of English text across multiple domains and genres and their Hungarian translations created by humans and by the state-of-the-art machine translation system deepl. Our results show that machine translation produces somewhat simpler text than human translation on 3 out of 4 samples, and on 2 samples this effect is caused primarily by human translators using a higher number of complex words. We release all software used in our experiments to facilitate further studies on larger samples, additional languages and domains, and using alternative MT systems.