femtocrux 0.6.3

Femtosense Compiler

femtocrux

Compiler API for the Femotsense Sparse Processing Unit (SPU).

This package drives compilation of machine learning models to SPU binaries, as well as simulation of those binaries on SPU hardware.

The package itself is a thin Python wrapper communicating to the actual compiler / simulator via gRPC.

Supported ML model representations:

• Femtosense Quantized IR (FQIR)

Installation

You will need to have python 3.10 and Docker installed. Install Docker by following these instructions.

femtocrux is available on PyPI via

pip install femtocrux

Basic Usage

ManagedCompilerClient

ManagedCompilerClient is the recommended way to use femtocrux within a context manager.

from femtocrux import ManagedCompilerClient, FQIRModel

# fqir_graph = ... # Assuming we have an FQIR graph using fmot
# inputs = ...     # Assuming we have numpy array inputs to the model

fqir_model = FQIRModel(fqir_graph, batch_dim=0, sequence_dim=1)
with ManagedCompilerClient() as client:
    
    # simulate execution, view power, energy, and latency metrics
    simulator = client.simulate(fqir_model)
    outputs, metrics = simulator.simulate(inputs)

    # compile the model to a bitfile
    bitstream = client.compile(fqir_model)
    with open('my_bitfile.zip', 'wb') as f: 
        f.write(bitstream)

The bitfile can be used to generate program files to run on the SPU, using femtodriverpub

CompilerClient

CompilerClient is another interface to the compiler. This will be deprecated in future releases, so we recommend using ManagedCompilerClient.

from femtocrux import CompilerClient, FQIRModel

# fqir_graph = ... # Assuming we have an FQIR graph using fmot
# inputs = ...     # Assuming we have numpy array inputs to the model

fqir_model = FQIRModel(fqir_graph, batch_dim=0, sequence_dim=1)
client = CompilerClient()
    
# simulate execution, view power, energy, and latency metrics
simulator = client.simulate(fqir_model)
outputs, metrics = simulator.simulate(inputs)

# compile the model to a bitfile
bitstream = client.compile(fqir_model)
with open('my_bitfile.zip', 'wb') as f: 
    f.write(bitstream)

# close the client (important to avoid background docker containers that continue running)
client.close()