ghostos-moss 0.3.3

the code-driven python interface for llms, agents and project GhostOS

MOSS Protocol

The frameworks of mainstream AI Agents currently use methods represented by JSON Schema Function Call to operate the capabilities provided by the system. An increasing number of frameworks are beginning to use code generated by models to drive, with OpenInterpreter being representative.

The GhostOS project envisions that the main means of interaction between future AI Agents and external systems will be based on protocol-based interactions, which include four aspects:

• Code As Prompt: The system directly reflects code into Prompts for large models through a series of rules, allowing large models to call directly.
• Code Interpreter: The system executes code generated by large models directly in the environment to drive system behavior.
• Runtime Injection: The system injects various instances generated at runtime into the context.
• Context Manager: The system manages the storage, use, and recycling of various variables in multi-turn conversations.

This entire set of solutions is defined as the MOSS protocol in GhostOS, with the full name being Model-oriented Operating System Simulator .

MOSS

MOSS implementations ghostos_moss is meant to be a independent package.

Purpose

The design goal of MOSS is to enable human engineers to read a code context as easily as a Large language model does, with what you see is what you get. We take SpheroBoltGPT (driven by code to control the toy SpheroBolt) as an example:

from ghostos.prototypes.spherogpt.bolt import (
    RollFunc,
    Ball,
    Move,
    LedMatrix,
    Animation,
)
from ghostos_moss import Moss as Parent


class Moss(Parent):
    body: Ball
    """your sphero ball body"""

    face: LedMatrix
    """you 8*8 led matrix face"""

This piece of code defines a Python context for controlling Sphero Bolt.

Both Large language models and human engineers reading this code can see that the behavior of SpheroBolt can be driven through moss.body or moss.face. The referenced libraries such as RollFunc, Ball, and Move in the code are automatically reflected as Prompts, along with the source code, submitted to the LLM to generate control code.

This way, LLM can be requested to generate a function like:

def run(moss: Moss):
    # body spin 360 degree in 1 second.
    moss.body.new_move(True).spin(360, 1)

The MossRuntime will compile this function into the current module and then execute the run function within it.

Abstract Classes

Core interface of MOSS are:

• MossCompiler: Compile any Python module to generate a temporary module.
• MossPrompter: Reflect a Python module to generate a prompt for the Large Language Model.
• MossRuntime: Execute the code generated by the Large Language Model within the temporary compiled module, and get result.

Get MossCompiler

MossCompiler registered into IoC Container. Get instance of it by:

from ghostos.bootstrap import get_container
from ghostos_moss import MossCompiler

compiler = get_container().force_fetch(MossCompiler)

PyContext

MossCompiler use PyContext to manage a persistence context.

It can be used to store variables defined and modified at runtime; it can also manage direct modifications to Python code for the next execution.

Each MossCompiler inherits an independent IoC Container, which can be used for dependency injection registration.

from ghostos_moss import MossCompiler
from ghostos_container import Provider

compiler: MossCompiler = ...


class Foo:
    ...


f: Foo = ...

some_provider: Provider = ...

compiler.bind(Foo, f)  # 绑定到 compiler.container()
compiler.register(some_provider)  # 注册 provider 到 compiler.container()

attr_value = ...

compiler.with_locals(attr_name=attr_value)  # 在目标 python module 注入一个本地变量 attr_name 

Compile Runtime

Using MossCompiler, you can compile a temporary module based on PyContext or a Python module name.

from ghostos.bootstrap import get_container
from ghostos_moss import MossCompiler, PyContext

pycontext_instance: PyContext = ...
compiler = get_container().force_fetch(MossCompiler)

# join python context to the compiler
compiler.join_context(pycontext_instance)

runtime = compiler.compile(None)

Get Compiled Module

Get the compiled module:

from types import ModuleType
from ghostos_moss import MossRuntime

runtime: MossRuntime = ...

module: ModuleType = runtime.module()

Moss Prompter

With MossRuntime we can get a MossPrompter, useful to generate Prompt for LLM:

from ghostos_moss import MossRuntime

runtime: MossRuntime = ...

with runtime:
    prompter = runtime.prompter()

    # get the full Prompt
    prompt = prompter.dump_module_prompt()

    # prompt is composed by: 

    # 1. source code of the module
    code = prompter.get_source_code()  # 获取模块的源码

    # each prompt of the imported attrs
    for attr_name, attr_prompt in prompter.imported_attr_prompts():
        print(attr_name, attr_prompt)

    attr_prompt = prompter.dump_imported_prompt() 

Hide Code to LLM

Modules compiled by MossCompiler will provide all their source code to the Large Language Model. If you want to hide a portion of the code, you can use the # <moss-hide> marker.

# <moss-hide>
from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from ghostos_moss import MossPrompter


# The code defined here will execute normally but will not be submitted to the LLM.
# This code is typically used to define the logic within the lifecycle of MossCompiler/Runtime operations.
# Shielding these logics helps the LLM to focus more.

def __moss_module_prompt__(prompter: "MossPrompter") -> str:
    ...

# </moss-hide>

Code Reflection

We utilize reflection mechanisms to automatically generate Prompts from code information and provide them to the Large Language Model. The basic idea is similar to how programmers view reference libraries, only allowing the LLM to see the minimal amount of information it cares about, mainly the definitions of classes and functions along with key variables. Instead of directly providing all the source code to the model.

Default Reflection Pattern

MossRuntime reflects variables imported into the current Python module and generates their Prompts according to certain rules. The current rules are as follows:

• Function & Method: Only reflect the function name + doc
• Abstract class: Reflect the source code
• pydantic.BaseModel: Reflect the source code

Additionally, any class that implements ghostos_common.prompter.PromptAbleClass will use its __class_prompt__ method to generate the reflection result.

Custom Attr Prompt

If the target Python module file defines the magic method __moss_attr_prompts__, it will use the provided results to override the automatically reflected results.

def __moss_attr_prompts__() -> "AttrPrompts":
    yield "key", "prompt"

If the returned prompt is empty, then ignore it to the LLM.

Runtime Execution

Based on MossRuntime, you can execute the code generated by the Large Language Model directly within a temporarily compiled module. The benefits of doing this are:

1. The LLM does not need to import all libraries, saving the overhead of tokens.
2. Accelerate the generation speed, expecting to surpass the output of JSON schema in many cases.
3. Avoid pollution of the context module by code generated by the Large Language Model.
4. Compared to executing code in Jupyter or a sandbox, temporarily compiling a module aims to achieve a "minimum context unit."

The basic principle is to use the current module as the context to compile and execute the code generated by the Large Language Model. The internal logic is as follows:

import ghostos_moss

runtime: ghostos_moss.MossRuntime = ...
pycontext = runtime.dump_pycontext()
local_values = runtime.locals()

generated_code: str = ...

filename = pycontext.module if pycontext.module is not None else "<MOSS>"
compiled = compile(generated_code, filename=filename, mode='exec')
# 直接编译
exec(compiled, local_values)

We can request that the code generated by the Large Language Model be a main function. After MossRuntime compiles the code, we can immediately execute this function.

import ghostos_moss

runtime: ghostos_moss.MossRuntime = ...
# 包含 main 函数的代码
generated_code: str = ...

with runtime:
    result = runtime.execute(target="main", code=generated_code, local_args=["foo", "bar"])

    # 执行过程中的 std output
    std_output = runtime.dump_std_output()
    # 获取变更过的 pycontext
    pycontext = runtime.dump_pycontext()

Custom Lifecycle functions

MossRuntime, during its lifecycle, attempts to locate and execute magic methods within the compiled modules. All magic methods are defined in ghostos_moss.lifecycle. For details, please refer to the file. The main methods include:

__all__ = [
    '__moss_compile__',  # prepare moss compiler, handle dependencies register
    '__moss_compiled__',  # when moss instance is compiled
    '__moss_attr_prompts__',  # generate custom local attr prompt
    '__moss_module_prompt__',  # define module prompt
    '__moss_exec__',  # execute the generated code attach to the module
]

Moss 类

In the target module compiled by MossCompiler, you can define a class named Moss that inherits from ghostos_moss.Moss. This allows it to receive key dependency injections during its lifecycle, achieving a what-you-see-is-what-you-get (WYSIWYG) effect.

The Moss class serves two purposes:

1. Automated Dependency Injection: Abstract classes mounted on Moss will receive dependency injection from the IoC container.
2. Managing Persistent Context: Data objects on the Moss class will be automatically stored in PyContext.

The existence of this class is default; even if you do not define it, an instance named moss will be generated in the compiled temporary module. The moss instance can be passed to functions in code generated by the Large Language Model.

For example, regarding context:

from abc import ABC
from ghostos_moss import Moss as Parent


class Foo(ABC):
    ...


class Moss(Parent):
    int_val: int = 0

    foo: Foo  # the abstract class bound to Moss will automatically get injection from MossRuntime.container()

The LLM generated code are:

# 大模型生成的 main 函数
def main(moss) -> int:
    moss.int_var = 123
    return moss.int_var

Executing this function will change the value of Moss.int_val to 123 in the future.

The purpose of this is to manage the context in a WYSIWYG manner. There are several default rules:

1. Variable Storage: All variables bound to the Moss instance, including those of type pydantic.BaseModel and int | str | float | bool, will be automatically stored in PyContext.
2. Abstract Class Dependency Injection: Any class mounted on Moss will automatically attempt to inject instances using the IoC Container.
3. Lifecycle Management: If a class implements ghostos_moss.Injection, its on_injection and on_destroy methods will be automatically called when injected into the moss instance.
4. Defining a Moss class will not pollute or disrupt the original functionality of the target file.

You can also use MossRuntime to obtain all the injection results for the Moss class.

from ghostos_moss import Moss, MossRuntime

runtime: MossRuntime = ...

moss_class = runtime.moss_type()
assert issubclass(moss_class, Moss)

moss_instance = runtime.moss()
assert isinstance(moss_instance, moss_class)

injections = runtime.moss_injections()

MOSS TestSuite

All source files that can be compiled by MossCompiler are also referred to as MOSS files.

In these files, the functions, variables, and classes defined can be unit tested, but runtime dependency injection requires the construction of a test suite.

GhostOS provides a default suite called ghostos_moss.testsuite.MossTextSuite. For more details, please refer to the code.