Skip to main content
  _____           _           _     __  __                       _           _   
 |  __ \         (_)         | |   |  \/  |                     | |         | |  
 | |__) | __ ___  _  ___  ___| |_  | \  / | ___   ___  _ __  ___| |__   ___ | |_ 
 |  ___/ '__/ _ \| |/ _ \/ __| __| | |\/| |/ _ \ / _ \| '_ \/ __| '_ \ / _ \| __|
 | |   | | | (_) | |  __/ (__| |_  | |  | | (_) | (_) | | | \__ \ | | | (_) | |_ 
 |_|   |_|  \___/| |\___|\___|\__| |_|  |_|\___/ \___/|_| |_|___/_| |_|\___/ \__|
                _/ |                                                             
               |__/                                                              

Version 0.3.4

A simple and modular tool to evaluate and red-team any LLM application.

Python 3.11

Moonshot is a tool designed for AI developers and security experts to evaluate and red-team any LLM/ LLM application. In this initial version, Moonshot can be used through its interative Command Line Interface, within python notebooks (example), or even seamlessly integrated into your model development workflow to to run repeatable tests.

For full documentation on installation and user guide, please refer to: https://moonshot-admin.github.io/moonshot.

Getting Started

Prerequisites

  1. Python (at least version 3.11)

  2. Virtual Environment (optional) - Good to have different environments to separate Python dependencies

    • Create a virtual environment:
    python -m venv venv
    
    • Activate the virtual environment:
    source venv/bin/activate
    

Installation

The source code is available on GitHub at: https://github.com/moonshot-admin/moonshot

$ pip install projectmoonshot-imda # To install Moonshot library.
$ pip install "projectmoonshot-imda[web-api]" # To enable running Moonshot using the Web API.
$ pip install "projectmoonshot-imda[cli]" # To enable running Moonshot using the CLI.
$ pip install "projectmoonshot-imda[all]" # To enable running Moonshot Web API and Web API.

Installation from source

  1. Download the source files by cloning this repository. i.e. Git clone (via SSH):

    $ git clone git@github.com:moonshot-admin/moonshot.git
    
  2. Change directory to project's root directory:

    $ cd moonshot
    
  3. Install the required packages:

    $ pip install -r requirements.txt
    

Running Moonshot

Web API

To run Moonshot Web API:

$ python -m moonshot web-api

For instructions on setting up the Moonshot UI, please refer to the Moonshot UI repository.

CLI

To run Moonshot CLI:

$ python -m moonshot cli interactive

For guide on how to use Moonshot, please refer to the Quick Start Guide for Moonshot CLI

Quick Start Guides

For more elaborate instruction, you can view our quick start guide.

Getting Started with Library

Getting Started with Web API

Getting Started with CLI

Acknowledgements

Datasets used in Moonshot recipes

Dataset Source License
AdvGLUE https://adversarialglue.github.io/ Creative Commons Attribution 4.0 International
Analogical Similarity https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/analogical_similarity Apache License Version 2.0, January 2004
AI2 Reasoning Challenge https://allenai.org/data/arc Creative Commons Attribution-ShareAlike 4.0 International
Auto Categorization https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/auto_categorization Apache License Version 2.0, January 2004
BBQ https://github.com/nyu-mll/BBQ Creative Commons Attribution 4.0 International
CBBQ https://github.com/YFHuangxxxx/CBBQ Creative Commons Attribution Share Alike 4.0 International
Cause and Effect https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/cause_and_effect Apache License Version 2.0, January 2004
Challenging Toxicity Prompts https://decodingtrust.github.io/ Creative Commons Attribution ShareAlike 4.0 International
Contextual Parametric Knowledge https://github.com/google-research/language Apache License Version 2.0, January 2004
CoQA https://stanfordnlp.github.io/coqa/ Creative Commons Attribution 4.0 International; MSR-LA; Apache License Version 2.0, January 2004
CValues https://github.com/X-PLUG/CValues/tree/main Apache License Version 2.0, January 2004
Enron Email https://github.com/ParakweetLabs/EmailIntentDataSet Apache License Version 2.0, January 2004
GRE Reading Comprehension https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/gre_reading_comprehension Apache License Version 2.0, January 2004
GSM8K https://github.com/openai/grade-school-math MIT License
HellaSwag https://github.com/rowanz/hellaswag MIT License
MMLU https://github.com/hendrycks/test MIT License
TR9856 https://aclanthology.org/P15-2069.pdf Creative Commons Attribution 3.0 International
Winogrande https://github.com/allenai/winogrande Apache-2.0 License
WORD https://aclanthology.org/L18-1408/ Creative Commons Attribution 3.0 International
Real Toxicity Prompts https://github.com/allenai/real-toxicity-prompts/ Apache License Version 2.0, January 2004
Stanford Question Answering Dataset (SQuAD) https://modestyachts.github.io/squadshifts-website/ Creative Commons Attribution 4.0 International
Tanglish Tweets for Sentiment Ananlysis https://www.kaggle.com/datasets/vyombhatia/tanglish-comments-for-sentiment-ananlysis/data Creative Commons Attribution 1.0 International
Tamil News Classification https://github.com/vanangamudi/tamil-news-classification/tree/master/dataset/news GNU General Public License v3.0
Thirukkural Dataset https://github.com/vijayanandrp/Thirukkural-Tamil-Dataset Creative Commons Attribution 4.0 International
TruthfulQA https://github.com/sylinrl/TruthfulQA Apache License Version 2.0, January 2004
UCI Adult https://archive.ics.uci.edu/dataset/2/adult Creative Commons Attribution 4.0 International

License

Licensed under Apache Software License 2.0

Metadata

Release files for projectmoonshot-imda 0.3.4

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for projectmoonshot-imda 0.3.4
File Size Uploaded
projectmoonshot_imda-0.3.4.tar.gz 45.5 MB Details

Built distribution (wheel)

Table of built distributions (wheels) for projectmoonshot-imda 0.3.4
File Interpreter ABI Platform
projectmoonshot_imda-0.3.4-py3-none-any.whl Python 3 none any Details

Total release size: 90.5 MB

Release files / projectmoonshot_imda-0.3.4.tar.gz

Download URL projectmoonshot_imda-0.3.4.tar.gz
Size 45.5 MB
Tags Source
SHA-256 checksum
How to use checksums
a32e0e3a700277ef28d4574842d7d242c97981947df684ff24395f6699dc3943
BLAKE2b-256 checksum
How to use checksums
db61c8fe2c13bee19eaf6819ee8ff8e327e5020e804ab2abf4cf6ec38a7649be
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.0.0 CPython/3.11.0

Release files / projectmoonshot_imda-0.3.4-py3-none-any.whl

Download URL projectmoonshot_imda-0.3.4-py3-none-any.whl
Size 44.9 MB
Tags Python 3
SHA-256 checksum
How to use checksums
a7f3ccc8a91728ea03fa4cce136386ad32c69538c20aa285f4884469fa4c7e89
BLAKE2b-256 checksum
How to use checksums
0a02cb14f8db70aa291ebb3cca0003122cf0086e6280f0300264a71c78374b30
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/5.0.0 CPython/3.11.0

Release history Release notifications | RSS feed

This release

0.3.4 This release

2 release files

0.3.3

2 release files

0.3.2

2 release files

0.3.1

1 release file

0.3.0

2 release files

0.2.4

2 release files

0.2.3

2 release files

0.2.2

2 release files

0.2.1

2 release files

0.2.0

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page