Connecting Data, People and Ideas since 2016.
27 May 2021

CoDEx: A Comprehensive Knowledge Graph Completion Benchmark

A talk by Tara Safavi

Researcher, University of Michigan



Slides available here


The field of automatic knowledge graph completion, which is motivated by the fact that knowledge graphs are usually incomplete, is an active research direction spanning several subfields of artificial intelligence


CoDEx is a set of knowledge graph COmpletion Datasets EXtracted from Wikidata and Wikipedia that improve upon existing knowledge graph completion benchmarks in scope and level of difficulty.


In terms of scope, CoDEx comprises three knowledge graphs varying in size and structure, multilingual descriptions of entities and relations, and tens of thousands of hard negative triples that are plausible but verified to be false.


CoDEx is characterized by thorough empirical analyses and benchmarking experiments.


  • First, each CoDEx dataset is analyzed in terms of logical relation patterns.
  • Next, baseline link prediction and triple classification results are reported on CoDEx for five extensively tuned embedding models.
  • Finally, CoDEx differentiates from the popular FB15K-237 knowledge graph completion dataset by showing that CoDEx covers more diverse and interpretable content, and is a more difficult link prediction benchmark.


Data, code, and pretrained models are available at

Connected Data World 2021  All Rights Reserved.

Connected Data is a trading name of Neural Alpha LTD.

Edinburgh House - 170 Kennington Lane
Lambeth, London - SE11 5DP