[PDF][PDF] Billions of parallel words for free: Building and using the eu bookshop corpus

R Skadiņš, J Tiedemann, R Rozis… - Proceedings of …, 2014 - researchgate.net
Proceedings of LREC, 2014researchgate.net
Abstract The European Union is a great source of high quality documents with translations
into several languages. Parallel corpora from its publications are frequently used in various
tasks, machine translation in particular. A source that has not systematically been explored
yet is the EU Bookshop–an online service and archive of publications from various
European institutions. The service contains a large body of publications in the 24 official of
the EU. This paper describes our efforts in collecting those publications and converting them …
Abstract
The European Union is a great source of high quality documents with translations into several languages. Parallel corpora from its publications are frequently used in various tasks, machine translation in particular. A source that has not systematically been explored yet is the EU Bookshop–an online service and archive of publications from various European institutions. The service contains a large body of publications in the 24 official of the EU. This paper describes our efforts in collecting those publications and converting them to a format that is useful for natural language processing in particular statistical machine translation. We report our procedure of crawling the website and various pre-processing steps that were necessary to clean up the data after the conversion from the original PDF files. Furthermore, we demonstrate the use of this dataset in training SMT models for English, French, German, Spanish, and Latvian.
researchgate.net
Showing the best result for this search. See all results