LLMPU

Large Language Model Processing Unit

Imagine a processing unit powered by LLM and infinite registers. Each register can store a string for prompts or codes. There is no fixed prompt. Instead, the contents of the first several registers are presented to the LLM. By generating code, the processing unit will be able to read and write any register, and directly execute the content of any register as Python code. Then the processing unit can be used as a general intelligent computing engine that potentially can improve itself by rewriting some of its own prompts or codes in registers.

The following instruction set is provided to the processing unit as Python functions:

• read(r) returns content in register number r
• write(r, content) store string content in register number r
• run(r) execute the content in register number r as Python code

And their description is not hard coded, but stored in register 0, for example, as a "firmware".

Get Started

pip install llmpu

import llmpu
llmpu.init({ # default config here
    "api_key": "", # llm api key
    "VR": 16, # visibile register number
    "L": 10000, # hard character number limit for register
    "model": "gemini-flash-latest", # llm model
    "llm_config": {}, # llm config
    "EXEC": { # provided to execution environment, can be used by llm
        "read": llmpu.read, "write": llmpu.write, "run": llmpu._run
    }
})

Instruction set functions:

# use register 100 as an example
llmpu.write(100, "print('hello')") # truncate if exceed config["L"]
llmpu.read(100) # "print('hello')"
llmpu.run(100) # execute code in register 100
# llmpu.run also supports code string

Note: llmpu._run shares the caller's locals, while llmpu.run is isolated.

Control functions:

# main cycle: generate instructions
code = llmpu.cycle()
llmpu.run(code) # run the code

# core dump
llmpu.dump("dump.json") # dump state to a json file
llmpu.load("dump.json") # load state from a json file

Server and UI:

Network server uses srpc protocol. It only allows localhost traffic for security reason. Exposing the interface allows arbitrary code execution!

llmpu.serve(port=22222, browser=True)

Minimum Example

# using default config
firmware = """You are a self-improving processing unit with registers: r0-r15 visible, more available but hidden. Perform operations by generating Python code.

API:
- read(r: int) -> str: read register r.
- write(r: int, content: str): write content in r (truncated to 5000 chars).
- run(r: int): Run register r's content as Python code.
"""
llmpu.write(0, firmware)
llmpu.write(1, "load r10001 to r1")
llmpu.write(10000, "do nothing")
llmpu.write(10001, "print hello to the screen, then call r10010")
llmpu.write(10010, "print('hello again')\nwrite(1, read(10000))")

llmpu.run(llmpu.cycle()) # r10001 will be loaded to r1
llmpu.run(llmpu.cycle()) # print "hello", then call r10010, which will print "hello again" and write r10000 to r1
llmpu.run(llmpu.cycle()) # pass