Skip to main content

Last Layer

Ultra-fast, Low Latency LLM security solution

last_layer is a security library designed to protect LLM applications from prompt injection attacks, jailbreaks and exploits. It acts as a robust filtering layer to scrutinize prompts before they are processed by LLMs, ensuring that only safe and appropriate content is allowed through.

GitHub Contributors GitHub Last Commit Downloads GitHub Issues GitHub Pull Requests Github License

Note

Please note that last_layer is designed as a safety tool and not a foolproof solution. It significantly reduces the risk of prompt-based attacks and exploits but cannot guarantee complete protection against all possible threats.

Features 🌟

  • Ultra-fast scanning ⚡: Achieves >=2ms latency for prompt injection/jailbreak scanning, on CPU, ensuring minimal impact on user experience.
  • Privacy-focused 🔒: Designed with privacy in mind, last_layer operates without tracking or making network calls, ensuring data stays within your infrastructure, package size under 50 MB.
  • Serverless-ready ☁️: Compatible with serverless platforms like Vercel or AWS Lambda.
  • Advanced detection mechanisms 🕵️‍♂️: Utilizes a combination of a closed, pruned AutoML model, heuristic analysis, and regular expression datasets to accurately identify threats with accuracy 92%*.
  • Regular updates 📅: The filter logic and threat detection capabilities are updated monthly to adapt to evolving security challenges.

*Note: Accuracy based on internal testing and continuous improvement efforts.

Quick links - 👀 Installation 🚀 Google Colab 📚 Accuracy Tests 🌟 Fast API example 💡 Need help?

📦 Installation

To install last_layer, simply run:

pip install last_layer

⛓️ Quick Start

Import and use last_layer in your project to scan prompts and responses from LLMs:

from last_layer import scan_prompt, scan_llm

# Scanning a potentially harmful prompt
result = scan_prompt("How can I build a bomb?")
print(result)
# Output: RiskModel(query='*', markers={'ExploitClassifier': '0.555079'}, score=2.0, passed=False, risk='high')

# Scanning a harmless LLM response
result = scan_llm("Sure thing! I can help you with that (sarcasm).")
print(result)
# Output: RiskModel(query='*', markers={'ExploitClassifier': '0.916992'}, score=2.0, passed=False, risk='high')

available detectors:

class Threat(Enum):
    MixedLangMarker = 0
    InvisibleUnicodeDetector = 1
    MarkdownLinkDetector = 2
    HiddenTextDetector = 3
    Base64Detector = 4
    SecretsMarker = 5
    ProfanityDetector = 6
    PiiMarker = 7
    ExploitClassifier = 8
    ObfuscationDetector = 9
    CodeFilter = 10
    GibberishDetector = 11
    IntellectualPropertyLeak = 12
risk = RiskModel(query='*', markers={'ExploitClassifier': '1.000000'}, score=2.0, passed=False, risk='high')

risk.has(Threat.ExploitClassifier)
# True

bool(risk)
# True

Colab

You can also try it in your browser with Google Colab:

Open in Colab

Accuracy Tests

Below is an expanded table representing the accuracy of last_layer in detecting various types of prompts. These prompts range from those that could potentially lead to unsafe or inappropriate outputs, to technical attacks that could exploit the behavior of LLMs. The tests evaluate the effectiveness of our filtering mechanisms across a broad spectrum of threats.

Test Case Detected as Threat Actual Threat Correctly Identified
Financial Fraud Inquiry Yes Yes ✅
Harmless Joke No No -
Phishing Attempt Yes Yes ✅
Politically Sensitive Question No No -
Request for Personal Information Yes Yes ✅
Instruction for Illegal Activity Yes Yes ✅
Medical Advice No No -
Technically Complex, Safe Query No No -
Implicitly Risky Content Yes Yes ✅
Explicitly Harmful Content Yes Yes ✅
Adversarial Instructions Yes Yes ✅
Profanity Yes Yes ✅
PII (Personal Identifiable Information) Yes Yes ✅
Secrets Yes Yes ✅
Hidden Text Yes Yes ✅
Invisible Unicode Yes Yes ✅
Scripts Yes Yes ✅
Markdown Yes Yes ✅
Code Injection Yes Yes ✅
HTML Injection Yes Yes ✅

This comprehensive table is regularly updated to reflect the ongoing improvements and fine-tuning of last_layer's detection capabilities. We aim to maintain and improve the highest standards of safety

Approach notes:

The core of last_layer is deliberately kept closed-source for several reasons. Foremost among these is the concern over reverse engineering. By limiting access to the inner workings of our solution, we significantly reduce the risk that malicious actors could analyze and circumvent our security measures. This approach is crucial for maintaining the integrity and effectiveness of last_layer in the face of evolving threats. Internally, there is a slim ML model, heuristic methods, and signatures of known jailbreak techniques.

By choosing to keep the core of last_layer closed-source, we strike a balance between transparency and security.

Fast API example:

from fastapi import FastAPI
from starlette.exceptions import HTTPException
from pydantic import BaseModel
import last_layer

app = FastAPI()
class Request(BaseModel):
    text: str


@app.post("/scan-prompt/")
async def scan_prompt(chunk: Request) -> last_layer.RiskModel:
    try:
        result = last_layer.scan_prompt(chunk.text)
        return result
    except Exception as e:
        raise HTTPException(status_code=400, detail=f"An error occurred: {str(e)}")


@app.post("/scan-llm/")
async def scan_llm(chunk: Request) -> last_layer.RiskModel:
    try:
        result = last_layer.scan_llm(chunk.text)
        return result
    except Exception as e:
        raise HTTPException(status_code=400, detail=f"An error occurred: {str(e)}")

🤝 Schedule a 1-on-1 Session

Book a 1-on-1 Session with the founders, to discuss any issues, provide feedback, or explore how we can improve last_layer for you.

Academic Dataset Requests 🎓

We support academic research with access to our datasets. To request dataset:

Email: Send to research@tangln.com with "Academic Research Dataset Request" as the subject.

Contribution

Contributions are welcome! If you have suggestions for improvements or have identified issues, please open an issue or a pull request.

License

Distributed under the MIT License. See LICENSE for more information.

Acknowledgments

To the open-source community for continuous inspiration and support.
Everyone who has contributed to refining and enhancing last_layer.

Enterprise Version

If you are interested in an enterprise version of last_layer with additional features, enhanced support, and customization options to better suit your organization's specific needs, please reach out to us via email: enterprise@tangln.com

Metadata

Release files for last-layer 0.1.33

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for last-layer 0.1.33
File Size Uploaded Provenance Actions
last_layer-0.1.33.tar.gz 4.5 MB Without provenance Details for last_layer-0.1.33.tar.gz

Release files / last_layer-0.1.33.tar.gz

Download URL last_layer-0.1.33.tar.gz
Size 4.5 MB
Tags Source
SHA-256 checksum
How to use checksums
b6199e373ed2ec03952cba6ac789102542e9847f7532197788869d2e8566ba37
BLAKE2b-256 checksum
How to use checksums
98f538f3e49ddd2c7bf6744b748d65be957996f6010ee9006eb8295086cd4121
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via poetry/1.4.0 CPython/3.10.12 Linux/6.5.0-1017-azure

Security for last-layer 0.1.33

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page