Transformers.js.py 🤗

Use Transformers.js on Pyodide and Pyodide-based frameworks such as JupyterLite, stlite (Streamlit), Shinylive (Shiny for Python), PyScript, HoloViz Panel, and so on.

The original Transformers can't be used in a browser environment. Transformers.js is a JavaScript version of Transformers that can be installed on browsers, but we can't use it from Pyodide. This package is a thin wrapper of Transformers.js to proxy its API to Pyodide.

API

The API is more like Transformers.js than the original Transformers.

import { pipeline } from '@xenova/transformers';

// Allocate a pipeline for sentiment-analysis
let pipe = await pipeline('sentiment-analysis');

let out = await pipe('I love transformers!');
// [{'label': 'POSITIVE', 'score': 0.999817686}]

from transformers_js_py import import_transformers_js

transformers = await import_transformers_js()
pipeline = transformers.pipeline

# Allocate a pipeline for sentiment-analysis
pipe = await pipeline('sentiment-analysis')

out = await pipe('I love transformers!')
# [{'label': 'POSITIVE', 'score': 0.999817686}]

See the Transformers.js document for available features.

Special Case: as_url()

Certain methods of Transformers.js accept a URL as an input. However, when using Transformers.js.py on Pyodide, there may be instances where we want to pass a local file path from the virtual file system, rather than a URL. In such scenarios, the as_url() function can be used to convert a local file path into a URL.

# Example
from transformers_js_py import import_transformers_js, as_url

transformers = await import_transformers_js()
pipeline = transformers.pipeline
pipe = await pipeline('image-classification')

local_image_path = "/path/to/image.jpg"

input_url = as_url(local_image_path)  # Converts a local file path into a URL that can be passed to `pipe()`
result = await pipe(input_url)

Examples

JupyterLite

👉Try this code snippet on https://jupyter.org/try-jupyter/lab/index.html

%pip install transformers_js_py

from transformers_js_py import import_transformers_js

transformers = await import_transformers_js()
pipeline = transformers.pipeline

pipe = await pipeline('sentiment-analysis')

out = await pipe('I love transformers!')

print(out)

Stlite (In-browser Streamlit)

👉 Online Demo: try out this code online.

import streamlit as st

from transformers_js_py import import_transformers_js

st.title("Sentiment analysis")

text = st.text_input("Input some text", "I love transformers!")

if text:
    with st.spinner():
        transformers = await import_transformers_js()
        pipeline = transformers.pipeline
        if "pipe" not in st.session_state:
            st.session_state["pipe"] = await pipeline('sentiment-analysis')
        pipe = st.session_state["pipe"]
        out = await pipe(text)
    st.write(out)

Gradio-Lite

Save the following code as an HTML file and open it in a browser.

<!DOCTYPE html>
<html lang="en">
<head>
  <script type="module" crossorigin src="https://cdn.jsdelivr.net/npm/@gradio/lite/dist/lite.js"></script>
  <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@gradio/lite/dist/lite.css" />
  <title>Transformers.js with Gradio-lite</title>
  <style>
    html, body {
      margin: 0;
      padding: 0;
      height: 100%;
      width: 100%;
    }
    gradio-lite {
      height: 100%;
      width: 100%;
    }
  </style>
</head>
<body>
<gradio-lite>

<gradio-requirements>
transformers_js_py
</gradio-requirements>

<gradio-file name="app.py" entrypoint>
import gradio as gr
from transformers_js_py import import_transformers_js

transformers = await import_transformers_js()
pipeline = transformers.pipeline

pipe = await pipeline('sentiment-analysis')

async def process(text):
    return await pipe(text)

demo = gr.Interface(fn=process, inputs="text", outputs="json")

demo.launch()
</gradio-file>

</gradio-lite>
</body>
</html>

👉 Online demo

The following articles help learn more:

• Gradio-Lite: Serverless Gradio Running Entirely in Your Browser
• Building Serverless Machine Learning Apps with Gradio-Lite and Transformers.js

Shinylive

👉 Online demo : try out this code online.

from shiny import App, render, ui
from transformers_js_py import import_transformers_js

app_ui = ui.page_fluid(
    ui.input_text("text", "Text input", placeholder="Enter text"),
    ui.output_text_verbatim("txt"),
)


def server(input, output, session):
    @output
    @render.text
    async def txt():
        if not input.text():
            return ""

        transformers = await import_transformers_js()
        pipeline = transformers.pipeline

        pipe = await pipeline('sentiment-analysis')

        out = await pipe(input.text())

        return str(out)


app = App(app_ui, server, debug=True)

PyScript

👉Try this code snippet on https://pyscript.com/

<html>
  <head>
    <link rel="stylesheet" href="https://pyscript.net/latest/pyscript.css" />
    <script defer src="https://pyscript.net/latest/pyscript.js"></script>
  </head>
  <body>
    <input type="text" value="" id="text-input" />
    <button py-click="run()" id="run-button">Run</button>

    <py-config>
        packages = ["transformers-js-py"]
    </py-config>
    <py-script>
        import asyncio
        from transformers_js_py import import_transformers_js

        text_input = Element("text-input")

        async def main(input_data):
            transformers = await import_transformers_js()
            pipeline = transformers.pipeline
            pipe = await pipeline('sentiment-analysis')
            out = await pipe(input_data)
            print(out)

        def run():
            print("Start")
            input_data = text_input.value
            if input_data.strip() == "":
                print("No data input.")
                return

            future = asyncio.ensure_future(main(input_data))
    </py-script>
  </body>
</html>

Panel

With HoloViz Panel you develop your app on your laptop and convert it to Pyodide or PyScript by running panel convert.

Install the requirements

pip install panel transformers_js_py

Create the app.py file in your favorite editor or IDE.

import panel as pn

pn.extension(sizing_mode="stretch_width", design="material")

@pn.cache
async def _get_pipeline(model="sentiment-analysis"):
    from transformers_js_py import import_transformers_js
    transformers = await import_transformers_js()
    return await transformers.pipeline(model)


text_input = pn.widgets.TextInput(placeholder="Send a message", name="Message")
button = pn.widgets.Button(name="Send", icon="send", align="end", button_type="primary")

@pn.depends(text_input, button)
async def _response(text, event):
    if not text:
        return {}
    pipe = await _get_pipeline()
    return await pipe(text)

pn.Column(
    text_input, button, pn.pane.JSON(_response, depth=2)
).servable()

Convert the app to Pyodide. For more options like hot reload check out the Panel Convert guide.

panel convert app.py --to pyodide-worker --out pyodide --requirements transformers_js_py

Now serve the app

python -m http.server -d pyodide

Finally you can try out the app by opening localhost:8000/app.html

import panel as pn

MODEL = "sentiment-analysis"
pn.chat.ChatMessage.default_avatars["hugging face"] = "🤗"

pn.extension(design="material")

@pn.cache
async def _get_pipeline(model):
    from transformers_js_py import import_transformers_js
    transformers = await import_transformers_js()
    return await transformers.pipeline(model)

async def callback(contents: str, user: str, instance: pn.chat.ChatInterface):
    pipe = await _get_pipeline(MODEL)
    response = await pipe(contents)
    label, score = response[0]["label"], round(response[0]["score"], 2)
    return f"""I feel a {label} vibe here (score: {score})"""

welcome_message = pn.chat.ChatMessage(
    f"I'm a Hugging Face Transformers `{MODEL}` model.\n\nPlease *send a message*!",
    user="Hugging Face",
)

pn.chat.ChatInterface(
    welcome_message, placeholder_text="Loading the model ...",
    callback=callback, callback_user="Hugging Face",
).servable()

Solara on py.cafe

Py.cafe is an online Python sandbox environment that supports many web frameworks, here is an example using Solara

👉 Online demo : try out this code online on .

import solara

clicks = solara.reactive(0)
input = solara.reactive("I love transformers")

from transformers_js_py import import_transformers_js

@solara.lab.task
async def run(input):

    transformers = await import_transformers_js()
    pipeline = transformers.pipeline

    # Allocate a pipeline for sentiment-analysis
    pipe = await pipeline('sentiment-analysis')

    out = await pipe(input)
    # [{'label': 'POSITIVE', 'score': 0.999817686}]
    return out

@solara.component
def Page():
    if run.error:
        solara.Error(repr(run.exception))
    with solara.Card("Sentiment analysis"):
        solara.ProgressLinear(run.pending)
        with solara.Div():
            solara.InputText(label="Input", value=input)
            solara.Button(label=f"Analyze sentiment", on_click=lambda: run(input.value), color="primary", filled=True)
        if run.finished:
            solara.Text(repr(run.value))