<div class="csl-bib-body">
<div class="csl-entry">Tsaneva, S., & Sabou, R. M. (2022, June 14). <i>Hybrid Human-Machine Evaluation of Knowledge Graphs</i> [Conference Presentation]. First international workshop on Human-Centered Design of Symbiotic Hybrid Intelligence, Amsterdam, Netherlands (the). http://hdl.handle.net/20.500.12708/176961</div>
</div>
-
dc.identifier.uri
http://hdl.handle.net/20.500.12708/176961
-
dc.description.abstract
Abstract. Knowledge Graphs (KGs) enable real-world knowledge to be structured in a machine-readable format and act as a skeleton to many advanced intelligent applications (e.g. conversational agents). The correctness and quality of KGs are of high importance, as incorrectly represented information or controversial concepts modeled from a single viewpoint can lead to invalid application outputs and biased systems. Several KG quality issues can be detected automatically, such as syntax
errors or hierarchy cycles, however, others require human involvement, e.g., identifying incorrectly modeled statements, or discovering concepts not compliant with how humans think. Human Computation and Crowdsourcing (HC&C) techniques have been used as a promising approach to outsource human-centric tasks to human contributors at a lower cost. Nevertheless, the scalable evaluation of large KGs
relying on HC&C techniques alone remains a challenge and the need for a hybrid human-machine approach arises. In this paper, a hybrid intelligence system for the use case of Knowledge Verification is proposed, which utilizes techniques from the Semantic Web, HC&C and Multi-Agent Systems communities for ensuring a transparent, well-coordinated and auditable KG evaluation process.
en
dc.language.iso
en
-
dc.subject
Human-in-the-loop
en
dc.subject
Knowledge Graph Evaluation
en
dc.subject
Multi-Agent System
en
dc.subject
Hybrid Intelligence
-
dc.title
Hybrid Human-Machine Evaluation of Knowledge Graphs
