Format of Input Data

DGL-KE toolkits provide commands for training, evaluation and inference. Different commands require different kinds of input data, including:

  • Knowledge Graph The knowledge graph used in train, evaluation and inference.
  • Trained Embeddings The embedding generated by dglke_train or dglke_dist_train.
  • Other Data Extra input data that used by inference tools.

A knowledge graph is usually stored in the form of triplets (head, relation, tail). Heads and tails are entities in the knowledge graph. All of them can be identified with unique IDs. In general, there exists two types of IDs for entities and relations in DGL-KE:

  • Raw ID The entities and relations can be identified by names, usually in the format of strings.
  • KGE ID They are used during knowledge graph training, evaluation and inference. Both entities and relations are identified with integers and should start from 0 and be contiguous.

If the input file of a knowledge graph uses Raw IDs for entities and relations and does not provide a mapping between Raw IDs and KGE Ids, DGL-KE will generate an ID mapping automatically.

The following table gives the overview of the input data for different toolkits. (Y for necessary and N for no-usage)

DGL-KE Toolkit Knowledge Graph Trained Embeddings Other Data
  Triplets ID Mapping Embeddings  
dglke_train Y Y N N
dglke_eval Y N Y N
dglke_partition Y Y N N
dglke_dist_train Use data generated by dglke_partition
dglke_predict N Y Y Y
dglke_emb_sim N Y Y Y

Format of Knowledge Graph Used by DGL-KE

DGL-KE support three kinds of Knowledge Graph Input:

  • Built-in Knowledge Graph Built-in knowledge graphs are preprocessed datasets provided by DGL-KE package. There are five built-in datasets: FB15k, FB15k-237, wn18, wn18rr, Freebase.
  • Raw User Defined Knowledge Graph Raw user defined knowledge graph dataset uses the Raw IDs. Necessary ID convertion is needed before training a KGE model on the dataset. dglke_train, dglke_eval and dglke_partition provides the basic ability to do the ID convertion automatically.
  • KGE User Defined Knowledge Graph KGE user defined knowledge graph dataset already uses KGE IDs. The entities and relations in triplets are integers.

Format of Built-in Knowledge Graph

DGL-KE provides five built-in knowledge graphs:

Dataset #nodes #edges #relations
FB15k 14,951 592,213 1,345
FB15k-237 14,541 310,116 237
wn18 40,943 151,442 18
wn18rr 40,943 93,003 11
Freebase 86,054,151 338,586,276 14,824

Each of these built-in datasets contains five files:

  • train.txt: training set, each line contains a triplet [h, r, t]
  • valid.txt: validation set, each line contains a triplet [h, r, t]
  • test.txt: test set, each line contains a triplet [h, r, t]
  • entities.dict: ID mapping of entities
  • relations.dict: ID mapping of relations

Format of Raw User Defined Knowledge Graph

Raw user defined knowledge graph dataset uses the Raw IDs. The knowledge graph can be stored in a single file (only providing the trainset) or in three files (trainset, validset and testset). Each file stores the triplets of the knowledge graph. The order of head, relation and tail can be arbitry, e.g. [h, r, t]. A delimiter should be used to seperate them. The recommended delimiter includes \t, |, , and ;. The ID mapping is automatically generated by DGL-KE toolkits for raw user defined knowledge graphs.

Following gives an example of Raw User Defined Knowledge Graph files:

train.txt:

"Beijing","is_capital_of","China"
"Pairs","is_capital_of","France"
"London","is_capital_of","UK"
"UK","located_at","Europe"
"China","located_at","Asia"
"Tokyo","is_capital_of","Japan"

valid.txt:

"France","located_at","Europe"

test.txt:

"Japan","located_at","Asia"

Format of User Defined Knowledge Graph

User Defined Knowledge Graph uses the KGE IDs, which means both the entities and relations have already been remapped. The entity IDs and relation IDs are both start from 0 and be contiguous. The knowledge graph can be stored in a single file (only providing the trainset) or in three files (trainset, validset and testset) along with two ID mapping files (one for entity ID mapping and another for relation ID mapping). The knowledge graph is stored as triplets in files. The order of head, relation and tail can be arbitry, e.g. [h, r, t]. A delimiter should be used to seperate them. The recommended delimiter includes \t, |, , and ;. The ID mapping information is stored as pairs in mapping files with pair[0] as the integer ID and pair[1] as the original raw ID. The dglke_train and dglke_dist_train will do some integrity check of the IDs according to the mapping files.

Following gives an example of User Defined Knowledge Graph files:

train.txt:

0,0,1
2,0,3
4,0,5
5,1,6
1,1,7
8,0,9

valid.txt:

3,1,6

test.txt:

9,1,7

Following gives an example of entity ID mapping file:

entities.dict:

0,"Beijing"
1,"China"
2,"Pairs"
3,"France"
4,"London"
5,"UK"
6,"Europe"
7,"Asia"
8,"Tokyo"
9,"Japan"

Following gives an example of relation ID mapping file:

relations.dict:

0,"is_capital_of"
1,"located_at"

Format of Trained Embeddings

The trained embeddings are generated by dglke_train or dglke_dist_train CMD. The trained embeddings are stored in npy format. Usually there are two files:

  • Entity embeddings Entity embeddings are stored in a file named in format of dataset_name>_<model>_entity.npy and can be loaded through numpy.load().
  • Relation embeddings Relation embeddings are stored in a file named in format of dataset_name>_<model>_relation.npy and can be loaded through numpy.load()

Format of Input Data Used by DGL-KE Inference Tools

Both dglke_predict and dglke_emb_sim require user provied list of inferencing object.

Format of Raw Input Data

Raw Input Data uses the Raw IDs. Thus the input file contains objects in raw IDs and necessary ID mapping file(s) are required. Each line of the input file contains only one object and it can contains multiple lines. The ID mapping file store mapping information in pairs with pair[0] as the integer ID and pair[1] as the original raw ID.

Following gives an example of raw input files for dglke_predict:

head.list:

"Beijing"
"London"

rel.list:

"is_capital_of"

tail.list:

"China"
"France"
"UK"

entities.dict:

0,"Beijing"
1,"China"
2,"Pairs"
3,"France"
4,"London"
5,"UK"
6,"Europe"

relations.dict:

0,"is_capital_of"
1,"located_at"

Format of KGE Input Data

KGE Input Data uses the KGE IDs. Thus the input file contains objects in KGE IDs, i.e., intergers. Each line of the input file contains only one object and it can contains multiple lines.

Following gives an example of raw input files for dglke_predict:

head.list:

0
4

rel.list:

0

tail.list:

1
3
5