HGB

KO | EN

gitlite — search

#hacktoberfest #ai #golang #go #android #rust #docker #macos #java #awesome #kubernetes #awesome-list #mcp #openai

(★ 344)

Revisiting, benchmarking, and refining Heterogeneous Graph Neural Networks.

File Explorer

Download Latest Version (.zip)

- - .gitignore
  - README.md
  - test.py
- - LICENSE
  - README.md
- - README.md
- - GNN.py
  - LP_MAGNN_training history.txt
  - README.md
  - run_LastFM.py
  - run_LastFM_GNN.py
- - GNN.py
  - gnn_link_predict.py
  - HomGNN.py
  - link_predict.py
  - model.py
  - README.md
  - utils.py
- - README.md
- - .gitignore
  - README.md
  - test.py
- - .gitignore
  - Data_Preprocessing.ipynb
  - gcn.py
  - get_acm_data.py
  - GNN.py
  - GTN.png
  - inits.py
  - main.py
  - main_gnn.py
  - main_sparse.py
  - messagepassing.py
  - model.py
  - model_sparse.py
  - README.md
  - utils.py
- - GNN.py
  - main.py
  - model.py
  - model_hetero.py
  - README.md
  - utils.py
- - README.md
- - .gitignore
  - get_lastfm_data.py
  - get_lastfm_data_same.py
  - GNN.py
  - preprocess_DBLP.ipynb
  - preprocess_IMDB.ipynb
  - preprocess_LastFM.ipynb
  - README.md
  - run_DBLP.py
  - run_DBLP_gnn.py
  - run_IMDB.py
  - run_LastFM.py
- - entity_classify.py
  - model.py
  - README.md
- - README.md
- - LICENSE
  - README.md
- - LICENSE
  - README.md
- - .gitignore
  - LICENSE
  - README.md
- - README.md
- README.md
- - build_data.py
  - build_features.py
  - build_network.py
  - README.md
  - tagMe.py
  - utils.py
.gitignore
README.md

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Heterogeneous Graph Benchmark

Revisiting, benchmarking, and refining Heterogeneous Graph Neural Networks.

**2023.3.2 update**: We make benchmark data including test set pulic. You can download data as follows:

* Node Classification: https://drive.google.com/drive/folders/10-pf2ADCjq_kpJKFHHLHxr_czNNCJ3aX?usp=sharing
* Link Prediction: https://drive.google.com/drive/folders/1RNOPAQ_jrHlNWMhfNT4wqclnW_O1PqYL?usp=sharing

Therefore, you can get your metric scores locally. Actually, when you run Simple-HGN in [NC-benchmark](NC/benchmark/methods/baseline/) and [LP-benchmark](LP/benchmark/methods/baseline/), data will be downloaded automatically.

***Warning***: As we have opened test data, you should try not to overfit or leak data during training. For example, the order of test data is not random permuted. If you use BatchNorm, you will get a biased norm value.

## Roadmap

We organize our repo by task, and one sub-folder per task. Currently, we have four tasks, i.e., node classification (NC), link prediction (LP), knowledge-aware recommendation (Recom) and text classification (TC).

### Revisiting

This part refers to Section 3 and Table 1 in our paper.

* [HAN](./NC/HAN)
* [GTN](./NC/GTN)
* [RSHN](./NC/RSHN)
* [HetGNN](./NC/HetGNN)
* [MAGNN-nc](./NC/MAGNN) and [MAGNN-lp](./LP/benchmark/methods/MAGNN_ini)

### Benchmarking and Refining

This part refers to Section 4,5,6 in our paper.

* [Node classification benchmark](./NC/benchmark), [NC-baseline](./NC/benchmark/methods/baseline)
* [Link prediction benchmark](./LP/benchmark), [LP-baseline](./LP/benchmark/methods/baseline)
* [Knowledge-aware recommendation benchmark](./Recom), [Recom-baseline](./Recom/baseline)

**We only make half of test labels public to prevent data leakage issues.** The public half data is to help you debug locally. If you want to obtain test scores, you need to **rename all the xxx_full to xxx in data folder** and submit your prediction to our [website](https://www.biendata.xyz/hgb/).

For node classification and link prediction tasks, you can submit online. But for recommendation task, since the prediction files are too large to submit, you have to test offline by yourself.

**If you want to show your method on our official leaderboard on HGB website, you can make an issue on this repo. Once your code or paper is verified, your method will be displayed on the official leaderboard.**

### FAQ

1. **Where is the code for all HGNNs in paper for HGB?**

Follow this roadmap in this repo:

```
NC or LP
|--benchmark
    |--methods
        |--HGNNs
```

2. **How to take part in HGB?**

See [Get Started](https://www.biendata.xyz/hgb/#/about).

3. **Why can't I obtain test score locally?**

To prevent data leakage issues, we randomly replace the test set labels. Therefore, you can only get test scores after you submit your prediction to biendata competitions.

4. **What is the format of training data and submission files?**

You can read the instructions in biendata competitions in "Data" and "Evaluation" panel. What should be noticed is that, your prediction files should be on the top level of the zipped file.

For example, you should submit a zip like this:

```
submit.zip
|--ACM_1.txt
|--ACM_2.txt
|--...
```

Instead of

```
submit.zip
|--submit/
    |--ACM_1.txt
    |--ACM_2.txt
    |--...
```

It is recommended that zip a file using ```zip``` command rather than right click. Because subfolder may be automatically built for some operating systems when using right click.

### More

**This repo is actively under development.** Therefore, there are some extra experiments in this repo beyond our paper, such as graph-based text classification. For more information, see our [website](https://www.biendata.xyz/hgb/). Welcome contribute new tasks, datasets, methods to HGB!

Moreover, we also have an implementation of Simple-HGN in [cogdl](https://github.com/THUDM/cogdl/tree/master/examples/simple_hgn).

## Citation

* **Title:** Are we really making much progress? Revisiting, benchmarking and refining the Heterogeneous Graph Neural Networks.
* **Authors:** Qingsong Lv\*, Ming Ding\*, Qiang Liu, Yuxiang Chen, Wenzheng Feng, Siming He, Chang Zhou, Jianguo Jiang, Yuxiao Dong, Jie Tang.
* **In proceedings:** KDD 2021.

HGB

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

HGB

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

Download README