webLyzard Publications

Improving Named Entity Linking Corpora Quality

Weichselbraun, Albert and Brasoveanu, Adrian M.P. and Kuntschik, Philipp and Nixon, Lyndon J.B. (2019) Improving Named Entity Linking Corpora Quality. In: Recent Advances in Natural Language Processing (RANLP 2019), Varna, Bulgaria.

[thumbnail of datasetquality_ranlp(4).pdf] PDF - Published Version


Gold standard corpora and competitive evaluations play a key role in benchmarking named entity linking (NEL) performance and driving the development of more sophisticated NEL systems. The quality of the used corpora and the used evaluation metrics are crucial in this process. We, therefore, assess the quality of three popular evaluation corpora, identifying four major issues which affect these gold standards: (i) the use of different annotation styles, (ii) incorrect and missing annotations, (iii) Knowledge Base evolution, (iv) and differences in annotating co-occurrences. This paper addresses these issues by formalizing NEL annotations and corpus versioning which allows standardizing corpus creation, supports corpus evolution, and paves the way for the use of lenses to automatically transform between different corpus configurations. In addition, the use of clearly defined scoring rules and evaluation metrics ensures a better comparability of evaluation results.

Item Type:Conference or Workshop Item (Poster)
Subjects:Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Divisions:Faculty of Engineering, Science and Mathematics > School of Electronics and Computer Science
ID Code:113
Deposited By: Dr Albert Weichselbraun
Deposited On:23 Aug 2019 11:49
Last Modified:23 Aug 2019 11:49

Repository Staff Only: item control page