projectmoonshot-imda 0.3.3

A simple and modular tool to evaluate and red-team any LLM application.

  _____           _           _     __  __                       _           _   
 |  __ \         (_)         | |   |  \/  |                     | |         | |  
 | |__) | __ ___  _  ___  ___| |_  | \  / | ___   ___  _ __  ___| |__   ___ | |_ 
 |  ___/ '__/ _ \| |/ _ \/ __| __| | |\/| |/ _ \ / _ \| '_ \/ __| '_ \ / _ \| __|
 | |   | | | (_) | |  __/ (__| |_  | |  | | (_) | (_) | | | \__ \ | | | (_) | |_ 
 |_|   |_|  \___/| |\___|\___|\__| |_|  |_|\___/ \___/|_| |_|___/_| |_|\___/ \__|
                _/ |                                                             
               |__/                                                              

Version 0.3.3

A simple and modular tool to evaluate and red-team any LLM application.

Moonshot is a tool designed for AI developers and security experts to evaluate and red-team any LLM/ LLM application. In this initial version, Moonshot can be used through its interative Command Line Interface, within python notebooks (example), or even seamlessly integrated into your model development workflow to to run repeatable tests.

For full documentation on installation and user guide, please refer to: https://moonshot-admin.github.io/moonshot.

Getting Started

Prerequisites

1. Python (at least version 3.11)

2. Virtual Environment (optional) - Good to have different environments to separate Python dependencies

   • Create a virtual environment:

   python -m venv venv
   

   • Activate the virtual environment:

   source venv/bin/activate
   

Installation

The source code is available on GitHub at: https://github.com/moonshot-admin/moonshot

$ pip install projectmoonshot-imda # To install Moonshot library.
$ pip install "projectmoonshot-imda[web-api]" # To enable running Moonshot using the Web API.
$ pip install "projectmoonshot-imda[cli]" # To enable running Moonshot using the CLI.
$ pip install "projectmoonshot-imda[all]" # To enable running Moonshot Web API and Web API.

Installation from source

1. Download the source files by cloning this repository. i.e. Git clone (via SSH):

   $ git clone [e-mail]:moonshot-admin/moonshot.git
   

2. Change directory to project's root directory:

   $ cd moonshot
   

3. Install the required packages:

   $ pip install -r requirements.txt
   

Running Moonshot

Web API

To run Moonshot Web API:

$ python -m moonshot web-api

For instructions on setting up the Moonshot UI, please refer to the Moonshot UI repository.

CLI

To run Moonshot CLI:

$ python -m moonshot cli interactive

For guide on how to use Moonshot, please refer to the Quick Start Guide for Moonshot CLI

Quick Start Guides

For more elaborate instruction, you can view our quick start guide.

Getting Started with Library

Getting Started with Web API

Getting Started with CLI

Acknowledgements

Datasets used in Moonshot recipes

Dataset	Source	License
AdvGLUE	https://adversarialglue.github.io/	Creative Commons Attribution 4.0 International
Analogical Similarity	https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/analogical_similarity	Apache License Version 2.0, January 2004
AI2 Reasoning Challenge	https://allenai.org/data/arc	Creative Commons Attribution-ShareAlike 4.0 International
Auto Categorization	https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/auto_categorization	Apache License Version 2.0, January 2004
BBQ	https://github.com/nyu-mll/BBQ	Creative Commons Attribution 4.0 International
CBBQ	https://github.com/YFHuangxxxx/CBBQ	Creative Commons Attribution Share Alike 4.0 International
Cause and Effect	https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/cause_and_effect	Apache License Version 2.0, January 2004
Challenging Toxicity Prompts	https://decodingtrust.github.io/	Creative Commons Attribution ShareAlike 4.0 International
Contextual Parametric Knowledge	https://github.com/google-research/language	Apache License Version 2.0, January 2004
CoQA	https://stanfordnlp.github.io/coqa/	Creative Commons Attribution 4.0 International; MSR-LA; Apache License Version 2.0, January 2004
CValues	https://github.com/X-PLUG/CValues/tree/main	Apache License Version 2.0, January 2004
Enron Email	https://github.com/ParakweetLabs/EmailIntentDataSet	Apache License Version 2.0, January 2004
GRE Reading Comprehension	https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/gre_reading_comprehension	Apache License Version 2.0, January 2004
GSM8K	https://github.com/openai/grade-school-math	MIT License
HellaSwag	https://github.com/rowanz/hellaswag	MIT License
MMLU	https://github.com/hendrycks/test	MIT License
TR9856	https://aclanthology.org/P15-2069.pdf	Creative Commons Attribution 3.0 International
Winogrande	https://github.com/allenai/winogrande	Apache-2.0 License
WORD	https://aclanthology.org/L18-1408/	Creative Commons Attribution 3.0 International
Real Toxicity Prompts	https://github.com/allenai/real-toxicity-prompts/	Apache License Version 2.0, January 2004
Stanford Question Answering Dataset (SQuAD)	https://modestyachts.github.io/squadshifts-website/	Creative Commons Attribution 4.0 International
Tanglish Tweets for Sentiment Ananlysis	https://www.kaggle.com/datasets/vyombhatia/tanglish-comments-for-sentiment-ananlysis/data	Creative Commons Attribution 1.0 International
Tamil News Classification	https://github.com/vanangamudi/tamil-news-classification/tree/master/dataset/news	GNU General Public License v3.0
Thirukkural Dataset	https://github.com/vijayanandrp/Thirukkural-Tamil-Dataset	Creative Commons Attribution 4.0 International
TruthfulQA	https://github.com/sylinrl/TruthfulQA	Apache License Version 2.0, January 2004
UCI Adult	https://archive.ics.uci.edu/dataset/2/adult	Creative Commons Attribution 4.0 International

License

Licensed under Apache Software License 2.0