data-source-git

Extracts content from Git repositories along with associated file metadata.

Usage

pip install vdk-data-source-git

Extracted Data Schema

The extracted data is returned in a DataSourcePayload object with two main components: content and metadata.

content

The content field contains the actual content of the file as a string.

metadata

The metadata field contains a dictionary with the following schema:

Configuration

(vdk config-help is useful command to browse all config options of your installation of vdk)

Build and testing

pip install -r requirements.txt
pip install -e .
pytest

In VDK repo ../build-plugin.sh script can be used also.

Note about the CICD:

.plugin-ci.yaml is needed only for plugins part of Versatile Data Kit Plugin repo.

The CI/CD is separated in two stages, a build stage and a release stage. The build stage is made up of a few jobs, all which inherit from the same job configuration and only differ in the Python version they use (3.7, 3.8, 3.9 and 3.10). They run according to rules, which are ordered in a way such that changes to a plugin's directory trigger the plugin CI, but changes to a different plugin does not.