Skip to main content

"A Python library for the Demisto SDK"

Project description

Demisto SDK

PyPI version CircleCI Coverage Status Ruff Black

The Demisto SDK is a Python library designed to aid the development process, both to validate entities being developed and to assist in the interaction between your development setup and Cortex XSOAR or Cortex XSIAM.

Requirements:

  • Python 3.9, 3.10 or 3.11.
  • git installed.
  • A linux, mac or WSL2 machine.

Windows machines are not supported - use WSL2 or run in a container instead.

Usage

Installation

  • Install - pip3 install demisto-sdk
  • Upgrade - pip3 install --upgrade demisto-sdk

Environment Variable Setup

Connect demisto-sdk with Cortex XSOAR server - Some SDK commands require you to have an interaction with the Cortex XSOAR or Cortex XSIAM server. Examples of such interactions include uploading and downloading entities to or from XSOAR or XSIAM and running commands in the CLI.

To use these functions, Set the following environment variables, or place an .env file at the root of the content pack:

For Cortex XSOAR 6.x Only

  1. Get your API key by going to the Cortex XSOAR server -> Settings -> Integrations -> API Keys -> Get Your Key -> Give your key a name and press Generate Key.

  2. Copy the given key.

  3. Add the following parameters to ~/.zshrc and ~/.bash_profile:

    export DEMISTO_BASE_URL=<http or https>://<demisto-server url or ip>:<port>
    export DEMISTO_API_KEY=<API key>
    

For Cortex XSIAM or Cortex XSOAR 8.x and above

  1. The base URL should be retrieved from the XSIAM instance by navigating to Settings -> Configurations -> API Keys -> clicking on the Copy URL button located at the top right corner of the page, and not the browser URL.

  2. The API key should be set to a Standard security level and must have the Instance Administrator role.

  3. The XSIAM_AUTH_ID environment variable needs to be set. Get it from the ID column of the created API Key.

  4. Add the following parameters to ~/.zshrc and ~/.bash_profile:

    export DEMISTO_BASE_URL=<BASE_URL>
    export DEMISTO_API_KEY=<API_KEY>
    export XSIAM_AUTH_ID=<THE_XSIAM_AUTH_ID>
    
  • Please note that once the XSIAM_AUTH_ID environment variable is set, the SDK commands will be configured to work with a Cortex XSIAM / XSOAR 8.x instance. In order to set Demisto SDK to work with a Cortex XSOAR 6.x instance, you need to delete the XSIAM_AUTH_ID parameter from your environment. To do this, please run the following command:

    unset XSIAM_AUTH_ID
    
  • In case the primary git branch is not master, or the upstream is not named origin, set them with environment variables:

    export DEMISTO_DEFAULT_BRANCH = <branch name here>
    export DEMISTO_DEFAULT_REMOTE = <upstream name here>
    

    For more configurations, check the demisto-py repo (the SDK uses demisto-py to communicate with Cortex XSOAR).

  • For the Validate and Format commands to work properly: - Install node.js, and make sure @mdx-js/mdx, fs-extra and commander are installed in node-modules folder (npm install ...). - Set the DEMISTO_README_VALIDATION environment variable to True.

     MDX is used to validate markdown files, and make sure they render properly on XSOAR and [xsoar.pan.dev](https://xsoar.pan.dev).
    
  • Reload your terminal.


CLI usage

You can use the SDK in the CLI as follows:

demisto-sdk <command> <args>

For more information, run demisto-sdk -h.

For more information on a specific command execute demisto-sdk <command> -h.

Content path

The demisto-sdk is made to work with Cortex content, structured similar to the official Cortex content repo.

Demisto-SDK commands work best when called from the content directory or any of its subfolders. To run Demisto-SDK commands from other folders, you may set the DEMISTO_SDK_CONTENT_PATH environment variable.

We recommend running all demisto-SDK commands from a folder with a git repo, or any of its subfolders. To suppress warnings about running commands outside a content repo folder, set the DEMISTO_SDK_IGNORE_CONTENT_WARNING environment variable.

Version Check

demisto-sdk will check against the GitHub repository releases for a new version every time it runs and will issue a warning if you are not using the latest and greatest. If you wish to skip this check you can set the environment variable: DEMISTO_SDK_SKIP_VERSION_CHECK. For example:

export DEMISTO_SDK_SKIP_VERSION_CHECK=yes

Run using Docker image

You can run the Demisto-SDK using a docker image. For more details go to Demisto-SDK Docker.


Commands

Supported commands:

  1. init
  2. Validate
  3. Pre-Commit
  4. Secrets
  5. Prepare-Content
  6. Split
  7. Format
  8. Run
  9. Run-playbook
  10. Upload
  11. Download
  12. Generate-docs
  13. Generate-test-playbook
  14. Generate-outputs
  15. Update-release-notes
  16. Zip-packs
  17. openapi-codegen
  18. postman-codegen
  19. generate-integration
  20. generate-yml-from-python
  21. generate-unit-tests
  22. setup-env

Logs

Log files are generated and stored automatically by default in the user's home directory:
Linux / macOS: $HOME/.demisto-sdk/logs
Windows: %USERPROFILE%\.demisto-sdk\logs

The default directory can be overriden using the --log-file-path flag, or the DEMISTO_SDK_LOG_FILE_PATH environment variable.


Customizable command configuration

You can create your own configuration for the demisto-sdk commands by creating a file named .demisto-sdk-conf within the directory from which you run the commands. This file will enable you to set a default value to the existing command flags that will take effect whenever the command is run. This can be done by entering the following structure into the file:

[command_name]
flag_name=flag_default_value

Note: Make sure to use the flag's full name and input _ instead of a - if it exists in the flag name (e.g. instead of no-docker-checks use no_docker_checks).

Here are a few examples:

  • As a user, I would like to include untracked git files in my validation when running the validate command. In the .demisto-sdk-conf file I'll enter:

    [validate]
    include_untracked=true
    
  • As a user, I would like to automatically use minor version changes when running the update-release-notes command. In the .demisto-sdk-conf file I'll enter:

    [update-release-notes]
    update_type=minor
    

License

MIT - See LICENSE for more information.


How to setup a development environment?

Follow the guide found here to setup your demisto-sdk dev environment. The development environment is connected to the branch you are currently using in the SDK repository.


Contributions

Contributions are welcome and appreciated. For information regarding contributing, press here.


Internet Connection

An internet connection is required for the following commands to work properly:

  1. Format
  2. Validate
  3. Update-release-notes
  4. Pre-commit

Note that the Download command will work only partially without an internet connection, it will fail when using the '-fmt, --run-format' argument.

  • When working offline (or in an airgapped environment), set the DEMISTO_SDK_OFFLINE_ENV environment variable to true:

    export DEMISTO_SDK_OFFLINE_ENV=TRUE
    

    When set, Demisto-SDK features requiring an internet connection will not be run, often saving some time and avoiding errors.


Docker Usage

Docker is required to run certain commands.

The following command requires Docker:

  1. setup-env

However, some commands can be executed partially without Docker:

  1. Format - To run without Docker, use the --no-graph flag.
  2. Generate-docs - To run without Docker, use the --no-graph flag.
  3. Validate - To skip Docker validation, use the --no-docker-checks flag.
  4. pre-commit - To run without Docker hooks, use the --no-docker flag.

XSOAR CI/CD

For information regarding XSOAR CI/CD, please see this article

Custom Container Registry

By default, the demisto-sdk will use dockerhub as the container registry to pull the integrations and scripts docker image. In order configure a custom container registry, the following environment variables must be set:

  • DEMISTO_SDK_CONTAINER_REGISTRY: the URL of the container registry.
  • DEMISTO_SDK_CR_USER: the username to use in the container registry.
  • DEMISTO_SDK_CR_PASSWORD: the password to use in the container registry.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

demisto_sdk-1.32.5.tar.gz (1.8 MB view details)

Uploaded Source

Built Distribution

demisto_sdk-1.32.5-py3-none-any.whl (2.3 MB view details)

Uploaded Python 3

File details

Details for the file demisto_sdk-1.32.5.tar.gz.

File metadata

  • Download URL: demisto_sdk-1.32.5.tar.gz
  • Upload date:
  • Size: 1.8 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.8.4 CPython/3.10.12 Linux/6.5.0-1025-azure

File hashes

Hashes for demisto_sdk-1.32.5.tar.gz
Algorithm Hash digest
SHA256 ce2ec1d8f331a576a38a4addb6a574c1c89a0bd1f9c1e0c6ce142329c38b5fc3
MD5 721ed109f6e69f32b310346192d45647
BLAKE2b-256 0d9fc06aa23549b4d8bbe73d26f4346f7abd292aaceef1aadd629c2624cdb4b8

See more details on using hashes here.

File details

Details for the file demisto_sdk-1.32.5-py3-none-any.whl.

File metadata

  • Download URL: demisto_sdk-1.32.5-py3-none-any.whl
  • Upload date:
  • Size: 2.3 MB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.8.4 CPython/3.10.12 Linux/6.5.0-1025-azure

File hashes

Hashes for demisto_sdk-1.32.5-py3-none-any.whl
Algorithm Hash digest
SHA256 2b829b1640457cd046626eb9d71ce4453887cc5e9a1c64eb63e0c06e39a28a89
MD5 0a86e7547719b33be4aab7ba89000b96
BLAKE2b-256 c09dbba070c04a2b19a35073ecc214eed1e5e5cf675dffac049f6da0009a5955

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page