Test Engines for the Asset Administration Shell

The Asset Administration Shell (AAS) is a standard for Digital Twins. More information can be found here.

The tools in this repository offer measures to validate compliance of AAS implementations against the AAS standard.

Installation

You can install the AAS Test Engines via pip:

python -m pip install --upgrade aas_test_engines

If you want to try the latest development version, you might want to install from git directly:

python -m pip install --upgrade pip install git+https://github.com/admin-shell-io/aas-test-engines.git

Command Line Interface

You may want to invoke the test tools using the command line interface:

# Check file
aas_test_engines check_file test.aasx
aas_test_engines check_file test.json --format json
aas_test_engines check_file submodel.aasx --model_type Submodel

# Check server
aas_test_engines check_server https://localhost https://admin-shell.io/aas/API/3/0/AssetAdministrationShellRepositoryServiceSpecification/SSP-002

# Generate test data
aas_test_engines generate_files output_dir

# Alternative output formats (work for all commands)
aas_test_engines check_file test.aasx --output html > output.html
aas_test_engines check_file test.aasx --output json > output.json

Note that the Test Engines return zero in case of compliance and non-zero otherwise so that you can integrate them into ci.

For more detailed instructions on how to test your AAS Software, see Test Setups. If you want to include the Test Engines into your software, see Python Module Interface.

Supported Versions, Suites and Templates

By default, the Test Engines test against the latest version 3.0, precisely: metamodel => 3.0.1 and API => 3.0.3. In case of Files, the IDTA specifies submodel templates. For a full list, please visit AAS Submodel Templates. The following templates are supported:

Name	Semantic ID	Support in test-engine
Contact Information	https://admin-shell.io/zvei/nameplate/1/0/ContactInformations	✅
Digital Nameplate for Industrial Equipment	https://admin-shell.io/zvei/nameplate/2/0/Nameplate	✅

For a detailed list of what is checked (and what is not), see here.

In case of API, the IDTA specifications define service specifications and profiles. Below tables describes the supported API profiles by the current test-engine. For more information about these profiles, please visit IDTA Specifications for API.

Name	Profile Identifier	Description	Support in test-engine
AAS Repository Read Profile	https://admin-shell.io/aas/API/3/0/AssetAdministrationShellRepositoryServiceSpecification/SSP-002	Only read operations for the AAS Repository Service	✅
AAS Read Profile	https://admin-shell.io/aas/API/3/0/AssetAdministrationShellServiceSpecification/SSP-002	Only read operations for the AAS Service	✅
Submodel Repository Read Profile	https://admin-shell.io/aas/API/3/0/SubmodelRepositoryServiceSpecification/SSP-002	Only read operations for the Submodel Repository Service	✅
Submodel Read Profile	https://admin-shell.io/aas/API/3/0/SubmodelServiceSpecification/SSP-002	Only read operations for the Submodel Repository Service	✅
AAS Registry Read Profile	https://admin-shell.io/aas/API/3/0/AssetAdministrationShellRegistryServiceSpecification/SSP-002	Only read operations for AAS Registry Service	✅*
Submodel Registry Read Profile	https://admin-shell.io/aas/API/3/0/SubmodelRegistryServiceSpecification/SSP-002	Only read operations for Submodel Registry Service	✅*
Discovery Service Full Profile	https://admin-shell.io/aas/API/3/0/DiscoveryServiceSpecification/SSP-001	All operations for Discovery Service	✅*
Concept Description Repository Profile	https://admin-shell.io/aas/API/3/0/ConceptDescriptionRepositoryServiceSpecification/SSP-001	Only Read operations for the Concept Description Service	✅*

*IDTA members only

For a detailed list of what is checked (and what is not), see here.

Test Setups

Check File for Compliance

Given a file in either AASX package format or XML/JSON "raw" format, you can use the Test Engines to check compliance of these files. Assuming your file is named my_aas.aasx, you can invoke the Test Engines using the command line:

aas_test_engines check_file my_aas.aasx

This will first check, if your aasx package is correct (in terms of relationships etc.). Then it checks if all contained AAS are compliant to the meta-model and all constraints hold as defined by Part 1 and Part 3a of the specification. Finally, the Test Engines will iterate over all submodels in your AAS and check if these are compliant to the corresponding submodel templates.

In case you want to check other formats, use the --format parameter:

aas_test_engines check_file my_aas.json --format json
aas_test_engines check_file my_aas.xml --format xml

Check Server for compliance

To test compliance of an AAS server to the HTTP/REST API, the Test Engines send a series of requests. Your server should then answer according to the behavior as defined by Part 2 of the specification. The Test Engines check the conformance of the responses. For each operation (aka endpoint) we execute a set of negative tests. These set parameters to invalid values like negative integers for the limit parameter. Afterwards we execute a set of positive tests which set all parameters to valid values and check the response accordingly.

Before starting the actual testing procedure, you need to populate some test data at your server which is stored at bin/check_servers/test_data. Then you start the testing by running passing the url of your server and a profile name:

aas_test_engines check_server \
    http://my-server.com/api/v3.0 \
    https://admin-shell.io/aas/API/3/0/AssetAdministrationShellRepositoryServiceSpecification/SSP-002

This starts the testing procedure which may take some seconds. You may prefer the HTML output for better readability by running:

aas_test_engines check_server \
    http://my-server.com/api/v3.0 \
    https://admin-shell.io/aas/API/3/0/AssetAdministrationShellRepositoryServiceSpecification/SSP-002 \
    --output html > result.html

Running a Subset of the Tests

By default, all tests of the selected suite/profile are executed. If you want to limit the test cases to a subset, you can pass --filter, so that only operations whose name matches the filter are tested:

aas_test_engines check_server ... --filter GetAll*

A filter consists of a list of glob patterns separated by :. Every operation whose name matches this glob pattern is executed. Optionally you can provide negative patterns which start after ~. Every operation whose name matches a negative glob pattern is omitted from test execution.

Examples:

• GetAll*: Tests all operations starting with GetAll
• GetAll*~GetAllShells: Tests all operations starting with GetAll except for GetAllShells
• *~Post*:Delete*: Test all operations except for the ones starting with Post or Delete

Handling Authentication

In case your server applies some authentication mechanism for security, you need to pass credentials to the Test Engines. You can use the --header option to do so by providing credentials within header fields:

aas_test_engines check_server ... --header 'Authorization: Basic YWxhZGRpbjpvcGVuc2VzYW1l'

If you need a more sophisticated authentication mechanism, you should use the Python module interface and provide your own aas_test_engines.http.HttpClient class.

Python Module Interface

Check AAS Type 1 (File)

Check AASX:

from aas_test_engines import file

with open('aas.aasx', 'rb') as f:
    result = file.check_aasx_file(f)
# result.ok() == True

result.dump()
# try result.to_html() to get an interactive representation

Check JSON:

from aas_test_engines import file

# Check file
with open('aas.json') as f:
    result = file.check_json_file(f)
# result.ok() == True

# Or check data directly
aas = {
    'assetAdministrationShells': [],
    'submodels': [],
    'conceptDescriptions': []
}
result = file.check_json_data(aas)
# result.ok() == True

result.dump()

Check XML:

from aas_test_engines import file
from xml.etree import ElementTree

# Check file
with open('aas.xml') as f:
    result = file.check_xml_file(f)
# result.ok() == True

# Or check data directly
data = ElementTree.fromstring(
    '<environment xmlns="https://admin-shell.io/aas/3/0" />')
result = file.check_xml_data(data)
# result.ok() == True

result.dump()

Checking older versions:

from aas_test_engines import file

print(file.supported_versions())
print(file.latest_version())
with open('aas.aasx', 'rb') as f:
    result = file.check_aasx_file(f, version="3.0")
# result.ok() == True

result.dump()

Check AAS Type 2 (HTTP API)

Check a running server instance:

from aas_test_engines import api, config, http

client = http.HttpClient(
    host="http://localhost",
)
conf = config.CheckApiConfig(
    suite="https://admin-shell.io/aas/API/3/0/AssetAdministrationShellRepositoryServiceSpecification/SSP-002",
)

result, mat = api.execute_tests(client, conf)
result.dump()

To check older versions pass version to the ApiConfig:

from aas_test_engines import api, config
print(api.supported_versions())
print(api.latest_version())

conf = config.CheckApiConfig(
    suite="https://admin-shell.io/aas/API/3/0/AssetAdministrationShellRepositoryServiceSpecification/SSP-002",
    version='3.0'
)

Generating test data for software testing

If you develop an AAS application like an AAS editor you may want to use test data to verify correctness of your application. The test engines allow to generate a set of AAS files which are compliant with the standard and you can therefore use to assess your application as follows:

from aas_test_engines import file

for is_valid, sample in file.generate():
    print(sample) # or whatever you want to do with it