Nise

license PyPI Build Status Unittests codecov

About

A tool for generating sample cost and usage data for testing purposes.

To submit an issue please visit https://issues.redhat.com/projects/COST/.

Getting Started

This is a Python project developed using Python 3.11. Make sure you have at least this version installed.

Development

To get started developing against Nise first clone a local copy of the git repository.

git clone https://github.com/project-koku/nise

Developing inside a virtual environment is recommended. This project utilizes uv. By default, uv creates a virtual env and installs the project in editable-mode. To run nise using the virtual env, use uv:

uv run nise -h

To build the command line tool run

uv build

For generating sample data for developing or testing Koku, please refer to Ingesting Nise data with Koku.

Updating packages

All packages (including dev) can be updated with:

uv sync --upgrade --all-groups

or

make upgrade-requirements

Testing

Nise uses tox to standardize the environment used when running tests. Install tox-uv ([docs]https://github.com/tox-dev/tox-uv):

uv tool install tox --with tox-uv # use uv to install

tox manages its own virtual environment and a copy of required dependencies to run tests. To ensure a clean tox environment run

tox -r

This will rebuild the tox virtual env and then run all tests.

The test env does not need to be rebuilt every time unit tests are run. To run all unit tests without rebuild:

tox

To run unit tests from a single file, e.g.:

tox -- tests.test_aws_generator

Linting

This repository uses pre-commit to check and enforce code style. It uses Ruff to format and lint the Python code. Other formats and text files are linted as well.

To run pre-commit checks:

pre-commit run --all-files

or:

make lint

Publishing

Publishing is achieved by merging a new version into main. Increment the version in [nise/init.py]. This can be done via hatch. For example, to bump the version to 4.9.9, one would do:

$ uv run hatch version 4.9.9
Old: 4.7.0
New: 4.9.9

Add the version change in nise/__init__.py to your PR and ask for a review. As soon as your PR is merged to main, a new koku-nise package will built, tagged, and deployed to PyPI.

Finer Publishing Details

All of the deployment is driven entirely by a Github Action workflow, so if issues ever crop up, start in publish-to-pypi.yml. When a branch is merged into main, the Action will kick off. There are three things that must happen before a deployment is successful, a successful artifact build, and the tag must not yet exist in git. After the artifact is published, it'll be available at PyPI.

Usage

nise is a command line tool.

Usage:
    nise ( report | yaml )
    nise report ( aws | azure | gcp | ocp ) [options]
    nise yaml ( aws | azure | ocp | ocp-on-cloud ) [options]

Report Options:
    -s, --start-date YYYY-MM-DD             required if not using --static-report-file FILE_NAME
                                            (static file dates overwrite this start date)
    -e, --end-date YYYY-MM-DD               optional, defaults:
                                                AWS/GCP/OCP: today at 23:59
                                                Azure: now() + 24 hours
    -w, --write-monthly                     optional, keep the generated report files in the local dir.
    --file-row-limit ROW_LIMIT              optional, default is 100,000. AWS and OCP only. Multiple reports
                                            will be generated with line counts not exceeding the ROW_LIMIT.
    --static-report-file YAML_NAME          optional, static report generation based on specified yaml file.
                                            See example_[provider]_static_data.yml for examples.
    -c --currency CURRENCY_CODE             optional, default is USD.

AWS Report Options:
    --aws-s3-bucket-name BUCKET_NAME        optional, must include --aws-s3-report-name.
                                            Use local directory path to populate a "local S3 bucket".
    --aws-s3-report-name REPORT_NAME        optional, must include --aws-s3-bucket-name.
    --aws-s3-report-prefix PREFIX_NAME      optional
    --aws-finalize ( copy | overwrite )     optional, finalize choice

Azure Report Options:
    --azure-container-name
    --azure-report-name
    --azure-report-prefix

GCP Report Options:
    --gcp-report-prefix PREFIX_NAME
    --gcp-bucket-name BUCKET_NAME

OCP Report Options:
    --ocp-cluster-id CLUSTER_ID             REQUIRED
    --insights-upload UPLOAD_URL            optional, Use local directory path to populate a
                                            "local upload directory".
    --ros-ocp-info                          Optional, Generate ROS for Openshift data.
    --constant-values-ros-ocp               Optional, Generate constant values for ROS for OpenShift data only
                                            when used with the ros-ocp-info parameter.

Common YAML Options:
    -o, --output YAML_NAME                  REQUIRED, Output file path (i.e "large.yml").
    -c, --config ( CONFIG | default )       optional, Config file path. If "default" is provided,
                                            use internal config file
    -s, --start-date YYYY-MM-DD             optional, must include -e, --end-date
                                                Start date (default is first day of last month)
    -e, --end-date YYYY-MM-DD               optional, must include -s, --start-date
                                                End date (default is last day of current month)
    -r, --random                            optional, default=False
                                                Randomize the number of
                                                    AWS: data generators
                                                    Azure: data generators
                                                    OCP: nodes, namespaces, pods, volumes, volume-claims
    -t, --template template                 optional, Template file path.

OCP Yaml Options:
    -n, --num-nodes INT                     optional, Number of nodes to generate (used with OCP
                                            only; default is 1)

OCP-on-Cloud Options:
    -c, --config ( CONFIG | default )       REQUIRED, Config file path. If "default" is provided,
                                            use internal config file
    -n, --num-nodes INT                     optional, Number of nodes to generate (default is 1)

Notes

  1. If --aws-s3-report-name or --aws-s3-report-prefix are specified they should match what is configured in the AWS cost usage report settings.
  2. For --aws-finalize:
    • copy will create a local copy of the data with a -finalized suffix and invoice id populated.
    • overwrite will generate a regular report with the invoice id populated.
  3. If --insights-upload is specified and pointing to a URL endpoint, you must have HCC_SERVICE_ACCOUNT_ID and HCC_SERVICE_ACCOUNT_SECRET set in your environment. Payloads for insights uploads will be split on a per-file basis.
  4. If --static-report-file is used start_date will default to first day of current month. start_date: last_month will be first day of previous month. start_date: today will start at the first hour of current day. end_date can support relative days from the start_date. i.e end_date: 2 is two days after start date.
  5. --static-report-file usage dates has a special full_period key value which will specify a usage for the entire start_date - end_date range.
  6. --ros-ocp-info when we generate ros data along with this parameter then we will be getting ros-ocp metrix too.

Examples

Example cost and usage report generation.

Example YAML generation.

Contributing

Please refer to Contributing.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

3 maintainers

PyPI data
Data sourced directly from PyPI's database.
Avatar for infinitewarp from gravatar.com infinitewarp Avatar for project-koku from gravatar.com project-koku Avatar for samdoran from gravatar.com samdoran

Credits

Maintainer: Project Koku

License

GNU Affero General Public License v3

Requires

Python >=3.11

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

koku_nise-5.4.2.tar.gz (97.8 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

koku_nise-5.4.2-py3-none-any.whl (149.8 kB view details)

Uploaded Python 3

File details

Details for the file koku_nise-5.4.2.tar.gz.

File metadata

  • Download URL: koku_nise-5.4.2.tar.gz
  • Upload date:
  • Size: 97.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.11.30 {"installer":{"name":"uv","version":"0.11.30","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for koku_nise-5.4.2.tar.gz
Algorithm Hash digest
SHA256 595f11e17915a907095c7308ee878eef152372d4b8e1aca957da40ca69ff2aae
MD5 574cb0647a941ecec75628c46f9e5e7c
BLAKE2b-256 96350945ca8c233a6f1afb6c786f38af68e72784ab8ba74f8929d469ddafe4fb

See more details on using hashes here.

File details

Details for the file koku_nise-5.4.2-py3-none-any.whl.

File metadata

  • Download URL: koku_nise-5.4.2-py3-none-any.whl
  • Upload date:
  • Size: 149.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.11.30 {"installer":{"name":"uv","version":"0.11.30","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for koku_nise-5.4.2-py3-none-any.whl
Algorithm Hash digest
SHA256 137a68ac001e0b04954291da9e4ed14da492b63eca92e5c54093ef83436ea36a
MD5 2c0902057cd70dd9e1affc7f4f2b797a
BLAKE2b-256 3470b8e036948d896273db12773527a0ce0b4040155f8aa85dcd3c7e1d9d515e

See more details on using hashes here.

Release history Release notifications | RSS feed

This release

5.4.2

5.4.1

5.4.0

5.3.9

5.3.8

5.3.7

5.3.6

5.3.5

5.3.4

5.3.3

5.3.2

5.3.1

5.3.0

5.2.2

5.2.1

5.2.0

5.1.15

5.1.14

5.1.11

5.1.10

5.1.9

5.1.8

5.1.7

5.1.6

5.1.5

5.1.4

5.1.3

5.1.2

5.1.1

5.1.0

5.0.3

5.0.1

5.0.0

4.7.5

4.7.4

4.7.3

4.7.2

4.7.1

4.7.0

4.6.9

4.6.8

4.6.7

4.6.6

4.6.5

4.6.4

4.6.3

4.6.2

4.6.1

4.6.0

4.5.7

4.5.6

4.5.5

4.5.4

4.5.3

4.5.2

4.5.1

4.5.0

4.4.18

4.4.17

4.4.16

4.4.15

4.4.13

4.4.12

4.4.11

4.4.10

4.4.9

4.4.8

4.4.7

4.4.6

4.4.5

4.4.4

4.4.3

4.4.2

4.4.1

4.4.0

4.3.3

4.3.2

4.3.1

4.3.0

4.2.9

4.2.8

4.2.7

4.2.6

4.2.5

4.2.4

4.2.3

4.2.2

4.2.1

4.2.0

4.1.0

4.0.4

4.0.3

4.0.2

4.0.1

4.0.0

3.1.0

3.0.9

3.0.8

3.0.7

3.0.6

3.0.5

3.0.4

3.0.3

3.0.2

3.0.1

3.0.0

2.9.9

2.9.8

2.9.7

2.9.6

2.9.5

2.9.4

2.9.2

2.9.1

2.9.0

2.8.1

2.8.0

2.7.2

2.7.1

2.7.0

2.6.1

2.6.0

2.5.6

2.5.5

2.5.4

2.5.3

2.5.2

2.5.1

2.5.0

2.4.13

2.4.12

2.4.11

2.4.10

2.4.9

2.4.7

2.4.6

2.4.5

2.4.4

2.4.3

2.4.2

2.4.1

2.4.0

2.3.9

2.3.8

2.3.5

2.3.4

2.3.3

2.3.2

2.3.1

2.3

2.2.3

2.2.2

2.2.1

2.2.0

2.1.5

2.1.4

2.1.3

2.1.2

2.1.1

2.1.0

2.0.18

2.0.17

2.0.16

2.0.15

2.0.14

2.0.13

2.0.12

2.0.11

2.0.10

2.0.9

2.0.8

2.0.7

2.0.6

2.0.5

2.0.4

2.0.3

2.0.2

2.0.1

2.0.0

1.0.16

1.0.15

1.0.14

1.0.13

1.0.12

1.0.11

1.0.10

1.0.9

1.0.8

1.0.7

1.0.6

1.0.5

1.0.4

1.0.3

1.0.2

1.0.1

1.0.0

0.0.1

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page