Skip to main content

Test IT TMS adapter for Pytest

Test IT

Release Status Downloads GitHub contributors

Getting Started

Installation

pip install testit-adapter-pytest

Usage

Configuration

Description File property Environment variable CLI argument
Location of the TMS instance url TMS_URL tmsUrl
API secret key How to getting API secret key? privateToken TMS_PRIVATE_TOKEN tmsPrivateToken
ID of project in TMS instance How to getting project ID? projectId TMS_PROJECT_ID tmsProjectId
ID of configuration in TMS instance How to getting configuration ID? configurationId TMS_CONFIGURATION_ID tmsConfigurationId
ID of the created test run in TMS instance.
It's necessary for adapterMode 0 or 1
testRunId TMS_TEST_RUN_ID tmsTestRunId
Parameter for specifying the name of test run in TMS instance (It's optional). If it is not provided, it is created automatically testRunName TMS_TEST_RUN_NAME tmsTestRunName
Test run tags (It's optional). Comma-separated list or JSON array, e.g. smoke,nightly or ["smoke","nightly"] testRunTags TMS_TEST_RUN_TAGS tmsTestRunTags
Test run links (It's optional). JSON array of objects with required url and optional title, description, type. Applied at create/start (not only at the end). Link types: Related, BlockedBy, Defect, Issue, Requirement, Repository testRunLinks TMS_TEST_RUN_LINKS tmsTestRunLinks
Adapter mode. Default value - 0. The adapter supports following modes:
0 - in this mode, the adapter filters tests by test run ID and configuration ID, and sends the results to the test run
1 - in this mode, the adapter sends all results to the test run without filtering or with filtering CLI
2 - in this mode, the adapter creates a new test run and sends results to the new test run
adapterMode TMS_ADAPTER_MODE tmsAdapterMode
It enables/disables certificate validation (It's optional). Default value - true certValidation TMS_CERT_VALIDATION tmsCertValidation
Mode of automatic creation test cases (It's optional). Default value - false. The adapter supports following modes:
true - in this mode, the adapter will create a test case linked to the created autotest (not to the updated autotest)
false - in this mode, the adapter will not create a test case
automaticCreationTestCases TMS_AUTOMATIC_CREATION_TEST_CASES tmsAutomaticCreationTestCases
Mode of automatic updation links to test cases (It's optional). Default value - false. The adapter supports following modes:
true - in this mode, the adapter will update links to test cases
false - in this mode, the adapter will not update link to test cases
automaticUpdationLinksToTestCases TMS_AUTOMATIC_UPDATION_LINKS_TO_TEST_CASES tmsAutomaticUpdationLinksToTestCases
Mode of import type selection when launching autotests (It's optional). Default value - true. The adapter supports following modes:
true - in this mode, the adapter will create/update each autotest in real time
false - in this mode, the adapter will create/update multiple autotests
importRealtime TMS_IMPORT_REALTIME tmsImportRealtime
Url of proxy server (It's optional) tmsProxy TMS_PROXY tmsProxy
Name (including extension) of the configuration file If it is not provided, it is used default file name (It's optional) - TMS_CONFIG_FILE tmsConfigFile
Sync storage port (It's optional, 49152 by default) syncStoragePort TMS_SYNC_STORAGE_PORT syncStoragePort

File

Add [testit] block to your pyproject.toml or create connection_config.ini file in the root directory of the project:

[testit]
URL = URL
privateToken = USER_PRIVATE_TOKEN
projectId = PROJECT_ID
configurationId = CONFIGURATION_ID
testRunId = TEST_RUN_ID
testRunName = TEST_RUN_NAME
testRunTags = smoke,nightly
testRunLinks = [{"url":"https://gitlab.example.com/group/project/-/jobs/12345","title":"CI Job","type":"Related"}]
adapterMode = ADAPTER_MODE
certValidation = CERT_VALIDATION
automaticCreationTestCases = AUTOMATIC_CREATION_TEST_CASES
automaticUpdationLinksToTestCases = AUTOMATIC_UPDATION_LINKS_TO_TEST_CASES
importRealtime = IMPORT_REALTIME

# This section are optional. It enables debug mode.
[debug]
tmsProxy = TMS_PROXY

Examples

Launch with a pyproject.toml or connection_config.ini file in the root directory of the project:

$ pytest --testit

Enable debug logs:

$ pytest --testit --log-cli-level=DEBUG

Launch with command-line parameters:

$ pytest --testit --tmsUrl=URL --tmsPrivateToken=USER_PRIVATE_TOKEN --tmsProjectId=PROJECT_ID --tmsConfigurationId=CONFIGURATION_ID --tmsTestRunId=TEST_RUN_ID --tmsTestRunName=TEST_RUN_NAME --tmsTestRunTags=smoke,nightly --tmsTestRunLinks='[{"url":"https://ci.example/jobs/1","title":"CI Job","type":"Related"}]' --tmsAdapterMode=ADAPTER_MODE --tmsProxy='{"http":"http://localhost:8888","https":"http://localhost:8888"}' --tmsCertValidation=CERT_VALIDATION --tmsAutomaticCreationTestCases=AUTOMATIC_CREATION_TEST_CASES --tmsAutomaticUpdationLinksToTestCases=AUTOMATIC_UPDATION_LINKS_TO_TEST_CASES --tmsImportRealtime=IMPORT_REALTIME

If you want to enable debug mode then see How to enable debug logging?

Run with filter

To create filter by autotests you can use the Test IT CLI (use adapterMode 1 for run with filter):

$ export TMS_TOKEN=<YOUR_TOKEN>
$ testit autotests_filter 
  --url https://tms.testit.software \
  --configuration-id 5236eb3f-7c05-46f9-a609-dc0278896464 \
  --testrun-id 6d4ac4b7-dd67-4805-b879-18da0b89d4a8 \
  --framework pytest \
  --output tmp/filter.txt
  
$ pytest "$(cat tmp/filter.txt)" --tmsTestRunId=6d4ac4b7-dd67-4805-b879-18da0b89d4a8 --tmsAdapterMode=1 --testit

Decorators

Decorators can be used to specify information about autotest.

Description of decorators:

  • testit.workItemIds - a method that links autotests with manual tests. Receives the array of manual tests' IDs
  • testit.displayName - internal autotest name (used in Test IT)
  • testit.externalId - unique internal autotest ID (used in Test IT)
  • testit.title - autotest name specified in the autotest card. If not specified, the name from the displayName method is used
  • testit.description - autotest description specified in the autotest card
  • testit.labels - labels listed in the autotest card
  • testit.tags - tags listed in the autotest card
  • testit.links - links listed in the autotest card
  • testit.step - the designation of the step called in the body of the test or other step
  • testit.nameSpace - directory in the TMS system (default - file's name of test)
  • testit.className - subdirectory in the TMS system (default - class's name of test)

All decorators support the use of parameterization attributes

Description of methods:

  • testit.addWorkItemIds - a dynamic method that links autotests with manual tests. Receives the array of manual tests' IDs
  • testit.addDisplayName - a dynamic method for adding internal autotest name (used in Test IT)
  • testit.addExternalId - a dynamic method for adding unique internal autotest ID (used in Test IT)
  • testit.addTitle - a dynamic method for adding autotest name specified in the autotest card. If not specified, the name from the displayName method is used
  • testit.addDescription - a dynamic method for adding autotest description specified in the autotest card
  • testit.addLabels - a dynamic method for adding labels listed in the autotest card
  • testit.addTags - a dynamic method for adding tags listed in the autotest card
  • testit.addLinks - links in the autotest result
  • testit.addAttachments - uploading files in the autotest result
  • testit.addMessage - information about autotest in the autotest result
  • testit.addNameSpace - a dynamic method for adding directory in the TMS system (default - file's name of test)
  • testit.addClassName - a dynamic method for adding subdirectory in the TMS system (default - class's name of test)
  • testit.addParameter - a dynamic method for adding parameter in the autotest result
  • testit.step - usage in the "with" construct to designation a step in the body of the test

Examples

Simple test

import pytest
import testit


# Test with a minimal set of decorators
@testit.externalId('Simple_autotest2')
def test_2():
    """Simple autotest 2"""
    assert oneStep()
    assert twoStep()


@testit.step
def oneStep():
    assert oneOneStep()
    assert oneTwoStep()
    return True


@testit.step
def twoStep():
    return True


@testit.step('step 1.1', 'description')
def oneOneStep():
    return True


@testit.step('step 2')
def oneTwoStep():
    return True


@testit.externalId('Simple_test_skip')
@testit.displayName('Simple test skip')
@testit.links(url='https://dumps.example.com/module/JCP-777')
@testit.links(url='https://dumps.example.com/module/JCP-777',
              title='JCP-777',
              type=testit.LinkType.RELATED,
              description='Description of JCP-777')
@pytest.mark.skipif(True, reason='Because i can')
def test_skip():
    assert True

Parameterized test

[!WARNING] When linking a parameterized autotest to a parameterized test case, please consider the problematic points:

  • In TMS test cases have a table with parameters, but autotests do not. They are not equal entities, so there may be incompatibility in terms of parameters
  • Running a parameterized test case, TMS expects the results of all related autotests with all the parameters specified in the test case table
  • In TMS, the parameters are limited to the string type, so the adapter transmits absolutely all the autotest parameters as a string. This implies the following problematic point for the test case table
  • TMS expects a complete textual match of the name and value of the parameters of the test case table with the autotest parameters
# Parameterized test with a full set of decorators
from os.path import join, dirname

import pytest
import testit


@testit.workItemIds(627)
@testit.displayName('Simple autotest 1 - {name}')
@testit.externalId('Simple_autotest1_{name}')
@testit.title('Authorization')
@testit.description('E2E_autotest')
@testit.tags('{tags}')
@testit.links(links=[
    {'url': '{url}', 'type': '{link_type}', 'title': '{link_title}', 'description': '{link_desc}'},
    {'url': '{url}', 'type': '{link_type}', 'title': '{link_title}', 'description': '{link_desc}'}
])
@pytest.mark.parametrize('name, tags, url, link_type, link_title, link_desc', [
    ('param 1', ['E2E', 'test'], 'https://dumps.example.com/module/JCP-777', testit.LinkType.DEFECT, 'JCP-777',
     'Desc of JCP-777'),
    ('param 2', (), 'https://dumps.example.com/module/docs', testit.LinkType.RELATED, 'Documentation',
     'Desc of JCP-777'),
    ('param 3', ('E2E', 'test'), 'https://dumps.example.com/module/projects', testit.LinkType.REQUIREMENT, 'Projects',
     'Desc of Projects'),
    ('param 4', {'E2E', 'test'}, 'https://dumps.example.com/module/', testit.LinkType.BLOCKED_BY, '', ''),
    ('param 5', 'test', 'https://dumps.example.com/module/repository', testit.LinkType.REPOSITORY, 'Repository',
     'Desc of Repository')
])
def test_1(name, tags, url, link_type, link_title, link_desc):
    testit.addLinks(url='https://dumps.example.com/module/some_module_dump', title='component_dump.dmp',
                    type=testit.LinkType.RELATED, description='Description')
    testit.addLinks(url='https://dumps.example.com/module/some_module_dump')
    testit.addLinks(links=[
        {'url': 'https://dumps.example.com/module/some_module_dump', 'type': testit.LinkType.BLOCKED_BY,
         'title': 'component_dump.dmp', 'description': 'Description'},
        {'url': 'https://dumps.example.com/module/some_module_dump', 'type': testit.LinkType.DEFECT},
        {'url': 'https://dumps.example.com/module/some_module_dump', 'type': testit.LinkType.RELATED,
         'title': 'component_dump.dmp'},
        {'url': 'https://dumps.example.com/module/some_module_dump', 'type': testit.LinkType.REQUIREMENT,
         'title': 'component_dump.dmp', 'description': 'Description'},
        {'url': 'https://dumps.example.com/module/some_module_dump', 'type': testit.LinkType.REPOSITORY,
         'description': 'Description'},
        {'url': 'https://dumps.example.com/module/some_module_dump'}
    ])
    with testit.step('Log in the system', 'system authentication'):
        with testit.step('Enter the login', 'login was entered'):
            with testit.step('Enter the password', 'password was entered'):
                assert True
        with testit.step('Create a project', 'the project was created'):
            with testit.step('Enter the project', 'the contents of the project are displayed'):
                assert True
            with testit.step('Create a test case', 'test case was created'):
                assert True
    with testit.step('Attachments'):
        testit.addAttachments(
            join(dirname(__file__), 'docs/text_file.txt'),
            join(dirname(__file__), 'pictures/picture.jpg'),
            join(dirname(__file__), 'docs/document.docx')
        )
        testit.addAttachments(
            join(dirname(__file__), 'docs/document.doc'),
            join(dirname(__file__), 'docs/logs.log')
        )
        assert True

Contributing

You can help to develop the project. Any contributions are greatly appreciated.

  • If you have suggestions for adding or removing projects, feel free to open an issue to discuss it, or directly create a pull request after you edit the README.md file with necessary changes.
  • Please make sure you check your spelling and grammar.
  • Create individual PR for each suggestion.
  • Please also read through the Code Of Conduct before posting your first idea as well.

License

Distributed under the Apache-2.0 License. See LICENSE for more information.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

testit_adapter_pytest-5.1.1.post580.tar.gz (22.1 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

testit_adapter_pytest-5.1.1.post580-py3-none-any.whl (16.4 kB view details)

Uploaded Python 3

File details

Details for the file testit_adapter_pytest-5.1.1.post580.tar.gz.

File metadata

File hashes

Hashes for testit_adapter_pytest-5.1.1.post580.tar.gz
Algorithm Hash digest
SHA256 e1f420a17afc47f3cdd5fb67b689e269a556e694d33900171163d61627b5b2b4
MD5 313916f1f7b901f51afa720bedc72c5c
BLAKE2b-256 e487ce2c4ebcc697190f9a80a0629352e9f60519eed29cea1fcd37d4d266ef0f

See more details on using hashes here.

File details

Details for the file testit_adapter_pytest-5.1.1.post580-py3-none-any.whl.

File metadata

File hashes

Hashes for testit_adapter_pytest-5.1.1.post580-py3-none-any.whl
Algorithm Hash digest
SHA256 0e6a604752feaab5d67cc1bd1f29a923d1ee861e916d0058f169c3434a798466
MD5 9d80dfe53bfae0be5927a29a9ce2a468
BLAKE2b-256 656e233f8316d4d5029392c0c07f56efae7d44beec4aa8c53926abee3f5f59f8

See more details on using hashes here.

Release history Release notifications | RSS feed

5.2.0

2 files

5.1.6

2 files

5.1.5.post580

2 files

5.1.5

2 files

5.1.4.post580

2 files

5.1.4

2 files

5.1.3.post580

2 files

5.1.3

2 files

5.1.2.post580

2 files

5.1.2

2 files

This release

5.1.1.post580 This release

2 files

5.1.1

2 files

5.1.0.post580

2 files

5.1.0

2 files

5.0.0.post580

2 files

5.0.0

2 files

4.3.2.post580

2 files

4.3.1.post580

2 files

4.3.0.post580

2 files

4.2.12

2 files

4.2.11

2 files

4.2.10

2 files

4.2.9

2 files

4.2.8.post570

2 files

4.2.8

2 files

4.2.7.post570

1 file

4.2.6.post570

1 file

4.2.6

1 file

4.2.5.post570

1 file

4.2.5

1 file

4.2.4.post570

1 file

4.2.4

1 file

4.2.3.post570

1 file

4.2.3

1 file

4.2.2.post570

1 file

4.2.2

1 file

4.2.1.post570

1 file

4.2.0.post570

1 file

4.2.0

1 file

4.1.14

1 file

4.1.13

1 file

4.1.12.post570

1 file

4.1.12

1 file

4.1.11

1 file

4.1.10.post570

1 file

4.1.10

1 file

4.1.9.post570

1 file

4.1.9

1 file

4.1.8.post570

1 file

4.1.8

1 file

4.1.7.post570

1 file

4.1.7

1 file

4.1.6.post570

1 file

4.1.6

1 file

4.1.5.post570

1 file

4.1.4.post570

1 file

4.1.4

1 file

4.1.3.post570

1 file

4.1.3

1 file

4.1.2.post570

1 file

4.1.2

1 file

4.1.1.post570

1 file

4.1.1

1 file

4.1.0.post570

1 file

4.1.0

1 file

4.0.0

1 file

3.12.4

1 file

3.12.3

1 file

3.12.2

1 file

3.12.1

1 file

3.12.0

1 file

3.11.6.post560

1 file

3.11.5.post560

1 file

3.11.4.post560

1 file

3.11.3.post560

1 file

3.11.2.post560

1 file

3.11.2

1 file

3.11.1.post560

1 file

3.11.1

1 file

3.11.0.post560

1 file

3.10.9.post550

1 file

3.10.9

1 file

3.10.8.post550

1 file

3.10.8

1 file

3.10.7.post550

1 file

3.10.7

1 file

3.10.6

1 file

3.10.5.post550

1 file

3.10.5

1 file

3.10.4.post550

1 file

3.10.4

1 file

3.10.2

1 file

3.10.1.post550

1 file

3.10.1

1 file

3.10.0

1 file

3.9.2.post540

1 file

3.9.1.post540

1 file

3.9.1

1 file

3.9.0

1 file

3.8.0

1 file

3.7.2.post540

1 file

3.7.2.post530

1 file

3.7.2

1 file

3.7.1.post540

1 file

3.7.1.post530

1 file

3.7.1

1 file

3.7.0.post540

1 file

3.7.0.post530

1 file

3.7.0

1 file

3.6.7.post540

1 file

3.6.7.post530

1 file

3.6.7

1 file

3.6.6.post540

1 file

3.6.6.post530

1 file

3.6.6

1 file

3.6.5.post540

1 file

3.6.5.post530

1 file

3.6.5

1 file

3.6.4.post530

1 file

3.6.4

1 file

3.6.3

1 file

3.6.2

1 file

3.6.1.post530

1 file

3.6.1

1 file

3.6.0.post530

1 file

3.6.0

1 file

3.5.5.post530

1 file

3.5.4

1 file

3.5.3

1 file

3.5.2

1 file

3.5.1

1 file

3.5.0

1 file

3.4.1

1 file

3.4.0

1 file

3.3.2

1 file

3.3.1

1 file

3.3.0

1 file

3.2.6

1 file

3.2.5

1 file

3.2.4

1 file

3.2.3

1 file

3.2.2

1 file

3.2.1

1 file

3.2.0

1 file

3.1.1

1 file

3.1.0

1 file

3.0.2

1 file

3.0.1

1 file

3.0.0

1 file

2.8.3

1 file

2.8.2

1 file

2.8.1

1 file

2.8.0

1 file

2.7.1

1 file

2.7.0

1 file

2.6.0

1 file

2.5.3

1 file

2.5.2

1 file

2.5.1

1 file

2.5.0

1 file

2.4.3

1 file

2.4.2

1 file

2.4.1

1 file

2.4.0

1 file

2.3.1

1 file

2.3.0

1 file

2.2.4

1 file

2.2.3

1 file

2.2.2

1 file

2.2.1

1 file

2.2.0

1 file

2.1.11

1 file

2.1.10

1 file

2.1.9

1 file

2.1.8

1 file

2.1.7

1 file

2.1.6

1 file

2.1.5

1 file

2.1.4

1 file

2.1.3

1 file

2.1.2

1 file

2.1.1

1 file

2.1.0

1 file

2.0.10

1 file

2.0.9

1 file

2.0.8

1 file

2.0.7

1 file

2.0.6

1 file

2.0.4

1 file

2.0.3

1 file

2.0.2

1 file

2.0.1

1 file

2.0.0

1 file

1.1.2

1 file

1.1.1

1 file

1.1.0

1 file

1.0.1

1 file

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page