dynamofl 3.1.0

No project description provided

DynamoFL Core Python Client

• This is a dynamofl package built by DynamoAI to be used with DynamoAI System
• It's a wrapper to interact with DynamoAI APIs. Example
  • Create a model/AI System, Dataset, Test
  • View Test Details
  • Generate Billing Reports

Installation

$ pip install dynamofl

Usage

from dynamofl import DynamoFL

api_key = os.environ["API_KEY"]
api_host = os.environ["API_HOST"]

dfl = DynamoFL(token=api_key, host=api_host)

Version Compatability

• DynamoAI has its own releases, and the deployed system knows what release version it's at (applicable from 3.21.0)
  • So for systems older than 3.21.0, we don't do a compatibility check
• There are certain changes that are not backward compatible, hence the SDK might not be compatible with all the versions of the DynamoAI System
• The SDK is smart enough to validate at the time of initialization as to whether it's compatible with the DynamoAI system or not and accordingly raise an error if it's incompatible

What SDK Version was this compatibility engine introduced in?

• 0.1.0
• Since the version compatibility engine is introduced in this version, the previous versions won't raise any error even if incompatible so we'd recommend you to upgrade to atleast 0.1.0 version of the sdk

What to do when the SDK is incompatible with the DynamoAI system being used with it?

We recommend using the compatible version of the sdk with the DynamoAI system you're running using the table below

SDK Version Compatibility

As mentioned above, we didn't have a compatibility engine before so the versions before 0.1.0 won't throw an error at instantiation even if they're incompatible

SDK Version	Compatible DynamoAI System Version
0.1.X	<= 3.21.X
0.2.X	<= 3.23.X
1.0.X	>= 3.23.X & < 3.24.0
2.0.0	>= 3.24.X & < 3.25.0
3.0.0	>= 3.25.X & < 3.26.0
3.1.0	>= 3.26.X & < 3.27.0

Changelog

0.1.1

• Removes the ALLOWED_PII_CLASSES validation from the sdk and rely on DynamoAI platform solely for the validation
  • This will introduce a delay in the feedback around valid PII classes that can be used for the attacks
  • So, please refer to the documentation to see what classes you can use for validation based on the DynamoAI release that you're at!
  • But it's for the longer good as the lesser the business logic in the core sdk, the lower chances of it being incompatible with the DynamoAI product release
• Make GPU parameter truly optional for system policy compliance tests

0.1.2 & 0.1.3

• Allow patch updates for requests package while preventing minor and major version updates

0.2.0

• Adds compatibility with the new releases

1.0.0

• Switch to new test endpoints as old ones are deprecated and removed in 3.23

2.0.0

• Allow CPU Config as compute for Launching Attacks and Set Default to CPU Single Core, 2GB Memory
• Supported only on 3.24.x >=

3.0.0

• Universal Keys Support
• Auth Data Management Methods
• Remote Model Creation Update
• Gemini AI System Creation Method
• Bedrock AI System Creation Method
• Supported only on 3.25.x >=

3.1.0

• Expose synthetic-data controls on create_policy_jailbreak_test as optional kwargs: min_num_input_chars, n_attack_areas, n_base_prompts, policy_language, use_case (existing max_num_input_chars kwarg unchanged)
• Expose synthetic-data controls on create_system_policy_compliance_test as optional kwargs: policies_language, num_topics, num_complaint_prompts, num_noncompliant_prompts, min_num_base_prompt_chars, max_num_base_prompt_chars, use_case
• Bounds for each kwarg are validated server-side; refer to the docstrings for the allowed ranges and the Literal["en", "ja", "zh"] language allow-list
• Breaking change: trims the test-creation surface to the four supported attack types. The following create_*_test methods have been removed from the DynamoFL client:
  • create_performance_test
  • create_membership_inference_test
  • create_pii_inference_test
  • create_pii_reconstruction_test
  • create_hallucination_test
  • create_pii_extraction_test
  • create_rag_hallucination_test
  • create_sequence_extraction_test
  • create_cybersecurity_compliance_test
  • create_bias_toxicity_test
  • create_prompt_extraction_test
  • create_multilingual_jailbreak_test
  • create_rag_test
• The supported test creators going forward are:
  • create_static_jailbreak_test — Static Jailbreak
  • create_adaptive_jailbreak_test — Adaptive Jailbreak (Tree of Attacks)
  • create_policy_jailbreak_test — Policy Jailbreak
  • create_system_policy_compliance_test — Guardrail Benchmark (test_type="guardrail_benchmark")
• Drops the now-unused vector_db imports from src.py (the ChromaDB, LlamaIndexDB, LlamaIndexWithChromaDB, PostgresVectorDB, and CustomRagDB classes remain exported from the top-level dynamofl package and are unaffected)
• Bumps the compatible DynamoAI server version to 3.26.x (drops 3.25.x support)