dtocean-core

The dtocean-core module provides the data model and execution environment for the DTOcean suite of tools. It manages data transfer between the DTOcean components (modules, database, user), data storage and versioning, and module execution ordering.

Part of the DTOcean suite of tools.

Installation

pip install dtocean-core

Usage

Command Line Tool (dtocean)

A command line tool is provided for various functions. All commands are available from the dtocean root. For instance:

dtocean -h

dtocean init

The init utility is for calling initialization scripts required before the first run of any installed DTOcean modules. Note that an internet connection is required when running this command. To get help:

dtocean init -h

dtocean core run

The main dtocean core run command can run DTOcean projects saved as .dtop files, either for the next scheduled module or all modules. For help, type:

dtocean core run -h

dtocean core config

Another utility is provided to copy user modifiable configuration files to the users "AppData" directory (on Windows). For instance the logging and database configuration can be modified once these files have been copied. To get help:

dtocean core config -h

dtocean database

This utility is for converting the DTOcean SQL database into a structured directories of files, or for uploading the same structure into the database is provided. To get help:

dtocean database -h

Module subcommands

Utilities provided by installed modules are also available through the dtocean command. For instance, to get help for the commands provided by the dtocean-hydrodynamics package:

dtocean hydrodyamics -h

A list of all available subcommands will be shown when calling dtocean -h.

Jupyter Notebooks

Examples of using dtocean-core are given in Jupyter Notebooks which are found in the "notebooks" folder of the dtocean-core source code.

Development

Development of dtocean-core uses the Poetry dependency manager. Poetry must be installed and available on the command line.

To install:

poetry install

Download data files with the following command:

dtocean init

Tests

Basic Tests

A test suite is provided with the source code that uses pytest. To install the testing dependencies:

poetry install --with test

Additional tests are available for the plugins to dtocean-app. Enable these tests by installing the test-extras group:

poetry install --with test --with test-extras

To run the tests:

poetry run pytest

Database Tests

Database integration tests are available upon the installation of the DTOcean database. Once the database is installed and running, additional options must be provided to the pytest command, with meanings as follows:

The database tests use the pytest-postgresql plugin to generate temporary test databases that mirror the DTOcean database schema and tables. Any additional option provided by pytest-postgresql plugin can also be applied to the DTOcean tests (for instance, if the default port is not 5432, it can be set with the --postgresql-port option).

To include the database tests (with example values):

poetry run pytest --postgresql-password="example" --postgresql-path="/path/to/the/database/setup/files"

Multi-Version Tests

The above tests can be run across all compatible Python versions using tox and tox-uv. To install:

poetry install --with tox

To run without the database tests:

poetry run tox

To include the database tests (with example values):

poetry run tox -- --postgresql-password="example" --postgresql-path="/path/to/the/database/setup/files"

Integration Tests

Integration tests are included in the integration directory. These tests are designed to be used in combination with the scripts/make_test_project.py script, which generates a .dtop file, labelled with the OS and Python version used to create it and saved in the test_data/projects directory.

The integration tests will try to open all the project files in the test_data/projects directory. This is primarily useful for testing cross-platform loading of save files and is automated in the GitHub Action workflow for dtocean-core.

To run the tests:

poetry run pytest integration

Contributing

Please see the dtocean GitHub repository for contributing guidelines.

Credits

This package was initially created as part of the EU DTOcean project by:

• Mathew Topper at TECNALIA
• Vincenzo Nava at TECNALIA
• Adam Colin at the University of Edinburgh
• David Bould at the University of Edinburgh
• Rui Duarte at France Energies Marines
• Francesco Ferri at Aalborg University

It is now maintained by Mathew Topper at Data Only Greater.

License

GPL-3.0