replit-code-exec

An interface to interact with Replit's code-exec API.

This is an interface for the simpler use cases of https://replit.com/@luisreplit/eval-python, which is a stateless API that is optimized to be called from AI Agents so that they can perform numerical reasoning through the evaluation of generated Python code. In general, this allows the execution of an untrusted snippet of Python code and returns whatever is printed to standard output / standard error as the result. The execution is done inside an ephemeral unprivileged container created on the fly running in Replit Deployments using https://github.com/omegaup/omegajail as code sandbox.

To set up your own copy of the API server, you need to follow these easy 2-3 steps (the second is optional):

1. Open the https://replit.com/@luisreplit/eval-python in your browser and Fork it to your account.
2. (Optional): if you want to change the Docker container, run evalctl image ${DOCKER_IMAGE} (e.g. evalctl image python:3 or evalctl image replco/python-kitchen-sink:latest).
   • Open .replit and change the EVAL_FILENAME, EVAL_RUN_COMMAND, EVAL_ENV to suite the new container, if needed.
3. Deploy the Repl! (just pressing Run is not enough)
   • This is only compatible with the Autoscale Deployments.
   • Make sure you set the EVAL_TOKEN_AUTH Deployments secrets when doing so for authentication.

Sample usage

Here is a small example of this library being used as a small math solver using OpenAI API:

import openai
import replit_code_exec

code_exec = replit_code_exec.build_code_exec(...)

def solve_math(prompt: str, model: str = 'gpt-3.5-turbo-0613') -> str:
    completion = openai.ChatCompletion.create(
        model=model,
        temperature=0.7,
        functions=[code_exec.openai_schema],
        function_call={"name": "code_exec"},
        messages=[
            {
                "role": "system",
                "content": ("You are an assistant that knows how to solve math " +
                            "problems by converting them into Python programs."),
            },
            {
                "role": "user",
                "content": ("Please solve the following problem by creating a Python " +
                            "program that prints the solution to standard output using " +
                            "`print()`: " + prompt),
            },
        ],
    )
    return code_exec.from_response(completion)