easygraph.datasets.hypergraph.hypergraph_dataset_base module

class easygraph.datasets.hypergraph.hypergraph_dataset_base.BaseData(name: str, data_root=None)

Bases: object

The Base Class of all datasets.

self._content = {
    'item': {
        'upon': [
            {'filename': 'part1.pkl', 'md5': 'xxxxx',},
            {'filename': 'part2.pkl', 'md5': 'xxxxx',},
        ],
        'loader': loader_function,
        'preprocess': [datapipe1, datapipe2],
    },
    ...
}

Attributes:

content

Return the content of the dataset.

Methods

fetch_files(files)	Download and check the files if they are not exist.
needs_to_load(item_name)	Return whether the item_name of the dataset needs to be loaded.
raw(key)	Return the key of the dataset with un-preprocessed format.

property content

Return the content of the dataset.

fetch_files(files: List[Dict[str, str]])

Download and check the files if they are not exist.

Parameters:

files (List[Dict[str, str]]) – The files to download, each element in the list is a dict with at lease two keys: filename and md5. If extra key bk_url is provided, it will be used to download the file from the backup url.

needs_to_load(item_name: str) → bool

Return whether the item_name of the dataset needs to be loaded.

Parameters:

item_name (str) – The name of the item in the dataset.

raw(key: str) → Any

Return the key of the dataset with un-preprocessed format.