Skip to main content

Toolkit to perform known-answer testing in Docker-isolated environments

Project description

KArTFire

kartfire is the "Known-Answer-Testing Framework". It is intended to run a number of student's solutions inside a well-defined, network-isolated runtime environment defined by a Docker container. The solutions can be programming-language agnostic (i.e., compiled and interpreted languages are supported, depending on the target container) and it is intended to deal with a wide array of faulty solutions:

  • Solutions that do not produce any output or that produce unparsable output
  • Solutions that terminate with error codes
  • Solutions that consume unlimited memory
  • Solutions that do not terminate at all

Boundary conditions

Any testcase is defined by a JSON file that has the following form:

{
	"action": "xyz",
	...
}

I.e., it needs to always be a dictionary that at least has an "action" key in the top-level. The answer can be any valid JSON object. For example, these would be valid testcases with their appropriate solutions:

{
	"action": "add",
	"a": 4,
	"b": 5
}

Expected answer:

{
	"sum": 9
}

Execution

TODO

License

GNU-GPL 3.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

kartfire-0.0.6.tar.gz (35.2 kB view details)

Uploaded Source

File details

Details for the file kartfire-0.0.6.tar.gz.

File metadata

  • Download URL: kartfire-0.0.6.tar.gz
  • Upload date:
  • Size: 35.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.0 CPython/3.12.3

File hashes

Hashes for kartfire-0.0.6.tar.gz
Algorithm Hash digest
SHA256 f74fd2a527848e7e6f1128ea47731f2b6699f1d6f6f170513770a05ac846b903
MD5 3f6532a0d0d3d2d9e474430f125b2096
BLAKE2b-256 5777ee5fb3b17e71e451d7153f0abb10f98838e7ac8387afe8e70f6f34bf9f65

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page