Graphlet AI Knowledge Graph Factory
Project description
Graphlet AI Knowledge Graph Factory
This is the PyPi module for the Graphlet AI Knowledge Graph Factory. Our mission is to create a Spark-based wizard for building large knowledge graphs that makes them easier to build for fewer dolalrs and with less risk.
Core Features
This project is new, some features we are building are:
-
Create a Pydantic/PySpark base class for transforming multiple entities into a uniform ontology
-
Create a generic, configurable system for entity resolution of heterogeneous networks
-
Create an efficient pipeline for computing network motifs and aggregating higher order networks
-
Implement efficient motif searching via neural subgraph matching
System Architecture
License
This project is created and published under the Apache License, version 2.0.
Conventions
This project uses pre-commit hooks to enforce its conventions: git will reject commits that don't comply with our various flake8 plugins.
We use numpy docstring format on all Python classes and functions, which is enforced by pydocstring and flake8-docstrings.
We run black
, flake8
, isort
and mypy
in .pre-commit-config.yaml. All of these are configured in pyproject.toml except for flake8 which uses .flake8
.
Flake8 uses the following plugins. We will consider adding any exceptions to the flake config that are warranted, but please document them in your pull requests.
flake8-docstrings = "^1.6.0"
pydocstyle = "^6.1.1"
flake8-simplify = "^0.19.2"
flake8-unused-arguments = "^0.0.10"
flake8-class-attributes-order = "^0.1.3"
flake8-comprehensions = "^3.10.0"
flake8-return = "^1.1.3"
flake8-use-fstring = "^1.3"
flake8-builtins = "^1.5.3"
flake8-functions-names = "^0.3.0"
flake8-comments = "^0.1.2"
Developer Setup
This project is in a state of development, things are still forming and changing. If you are here, it must be to contribute :)
Dependencies
We manage dependencies with poetry which are managed (along with most settings) in pyproject.toml.
To build the project, run:
poetry install
To add a PyPi package, run:
poetry add <package>
To add a development package, run:
poetry add --dev <package>
If you do edit pyproject.toml you must update to regenerate poetry.lock:
poetry update
Pre-Commit Hooks
We use pre-commit to run black, flake8, isort and mypy. This is configured in .pre-commit-config.yaml.
VSCode Settings
The following VSCode settings are defined for the project in .vscode/settings.json to ensure code is formatted consistent with our pre-commit hooks:
{
"editor.rulers": [90, 120],
"[python]": {
"editor.defaultFormatter": "ms-python.python",
"editor.formatOnSave": true,
"editor.codeActionsOnSave": {"source.organizeImports": true},
},
"python.jediEnabled": false,
"python.languageServer": "Pylance",
"python.linting.enabled": true,
"python.formatting.provider": "black",
"python.sortImports.args": ["--profile", "black"],
"python.linting.pylintEnabled": false,
"python.linting.flake8Enabled": true,
"autoDocstring.docstringFormat": "numpy",
}
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.