iris-embedded-python-wrapper 0.5.17

Wrapper for embedded python on InterSystems IRIS

iris-embedded-python-wrapper

This is a module that wraps embedded python in the IRIS Dataplateform. It provides a simple interface to run python code in IRIS.

More details can be found in the IRIS documentation

Pre-requisites

To make use of this module, you need to have the IRIS Dataplatform installed on your machine (more details can be found here).

Then you must configure the service callin to allow the python code to be executed and set the environment variables.

Configuration of the service callin

In the Management Portal, go to System Administration > Security > Services, select %Service_CallIn, and check the Service Enabled box.

More details can be found in the IRIS documentation

Environment Variables

Set the following environment variables :

• IRISINSTALLDIR: The path to the IRIS installation directory
• LD_LIBRARY_PATH: The path to the IRIS library
• IRISUSERNAME: The username to connect to IRIS
• IRISPASSWORD: The password to connect to IRIS
• IRISNAMESPACE: The namespace to connect to IRIS

For Linux and MacOS

For Linux and MacOS, you can set the environment variables as follows:

export IRISINSTALLDIR=/opt/iris
export LD_LIBRARY_PATH=$IRISINSTALLDIR/bin:$LD_LIBRARY_PATH
# for MacOS
export DYLD_LIBRARY_PATH=$IRISINSTALLDIR/bin:$DYLD_LIBRARY_PATH
# for IRIS username
export IRISUSERNAME=SuperUser
export IRISPASSWORD=<password>
export IRISNAMESPACE=USER

For windows

For windows, you can set the environment variables as follows:

set IRISINSTALLDIR=C:\path\to\iris

For Python 3.8 and newer, the wrapper automatically registers the IRIS bin directory with os.add_dll_directory() when IRISINSTALLDIR is set. Update PATH only when using older Python versions or external tools that need IRIS DLLs:

set PATH=%IRISINSTALLDIR%\bin;%PATH%

Set the IRIS username, password, and namespace

set IRISUSERNAME=SuperUser
set IRISPASSWORD=SYS
set IRISNAMESPACE=USER

For PowerShell

For PowerShell, you can set the environment variables as follows:

$env:IRISINSTALLDIR="C:\path\to\iris"
$env:IRISUSERNAME="SuperUser"
$env:IRISPASSWORD="<password>"
$env:IRISNAMESPACE="USER"

Installation

pip install iris-embedded-python-wrapper

Running tests

Local .venv

For pure unit tests, use the project virtual environment and keep CPF merge tests on temporary files:

python3 -m venv .venv
. .venv/bin/activate
python -m pip install --upgrade pip setuptools wheel
python -m pip install -e . pytest
env -u ISC_CPF_MERGE_FILE python -m pytest tests -q

Embedded-local e2e tests from python3 also need an IRIS installation and the platform loader path configured before Python starts:

export IRISINSTALLDIR=/opt/iris
export LD_LIBRARY_PATH=$IRISINSTALLDIR/bin:$LD_LIBRARY_PATH
python -m pytest tests/iris/test_dbapi_e2e.py -q

On macOS use DYLD_LIBRARY_PATH where your shell and Python launcher allow it. On Windows, the wrapper registers the IRIS bin directory with os.add_dll_directory() when IRISINSTALLDIR is set.

Docker

Run the test suite in Docker with the vanilla official InterSystems IRIS community image:

./scripts/test-docker.sh

Pass any pytest selector or option after the script name:

./scripts/test-docker.sh tests/iris/test_dbapi_embedded.py -q

scripts/test-docker.sh starts docker-compose-test-preview.yml, waits for IRIS, unlocks the default test passwords, and then delegates pytest execution to scripts/run-pytest-in-iris.sh. The in-container runner is the single source of truth for GitHub Actions and local Docker runs.

The container test flow is source-based:

• the repository is mounted at /irisdev/app read-only
• PYTHONPATH=/irisdev/app exposes the working tree
• the test virtual environment is created under /tmp
• pytest bytecode/cache writes are disabled
• ISC_CPF_MERGE_FILE is unset before pytest so tests cannot rewrite the repo merge file

By default IRIS_E2E_MODES=embedded,remote, so remote DB-API e2e tests run and the embedded runtime plus embedded DB-API SQL are required from python3.

To test another IRIS image tag:

IRIS_IMAGE_TAG=latest-preview ./scripts/test-docker.sh

Usage

You can use this module in three ways:

1. Run python code in IRIS
2. Bind a virtual environment to embedded python in IRIS
3. Unbind a virtual environment from embedded python in IRIS

Run python code in IRIS

Now you can use the module to run python code in IRIS. Here is an example:

import iris
iris.system.Version.GetVersion()

Output:

'IRIS for UNIX (Apple Mac OS X for x86-64) 2024.3 (Build 217U) Thu Nov 14 2024 17:29:23 EST'

Unified runtime context

The wrapper now uses a unified runtime API through iris.runtime.

Embedded runtime

The wrapper can run in two embedded contexts:

• embedded-kernel: Python is launched by IRIS, for example with iris python iris or iris session iris followed by :py
• embedded-local: regular python3 loads the IRIS embedded Python libraries from an installed IRIS instance

In embedded-kernel, IRIS has already loaded the runtime. Set PYTHONPATH to the project or installed package location when you need the wrapper instead of the built-in iris module:

PYTHONPATH=/path/to/iris-embedded-python-wrapper iris python iris

For an interactive session:

PYTHONPATH=/path/to/iris-embedded-python-wrapper iris session iris
USER>:py
>>> import iris
>>> iris.runtime.get().state
'embedded-kernel'

In embedded-local, configure the IRIS install directory and loader path before starting Python, or provide the install directory at runtime with iris.connect(path=...) as described below.

Runtime model

• iris.runtime.mode: selected policy (auto, embedded, native)
• iris.runtime.state: detected runtime (embedded-kernel, embedded-local, native-remote, unavailable)
• iris.runtime.embedded_available: whether embedded backend can be used
• iris.runtime.iris: currently bound native object API handle (optional)
• iris.runtime.dbapi: optional explicitly bound DB-API connection

Runtime control API

• iris.runtime.get()
• iris.runtime.configure(mode="auto", install_dir=None, iris=None, dbapi=None, native_connection=None)
• iris.runtime.reset()

mode is optional in runtime.configure(...).

• If iris, native_connection, or dbapi is provided, runtime infers native mode.
• If no connection handle is provided, runtime stays in auto/embedded detection flow.

runtime.configure(...) also accepts an IRISConnection and auto-converts it to an IRIS handle via createIRIS(...) for iris.cls(...) routing.

Examples

Force native object API routing:

import iris

conn = iris.connect("localhost", 1972, "USER", "SuperUser", "<password>")
iris.runtime.configure(mode="native", native_connection=conn)

obj = iris.cls("Ens.StringRequest")._New()

Native routing with inferred mode and auto-conversion from IRISConnection:

import iris

conn = iris.connect("localhost", 1972, "USER", "SuperUser", "<password>")
iris.runtime.configure(native_connection=conn)

obj = iris.cls("Ens.StringRequest")._New()

Force embedded routing:

import iris

iris.runtime.configure(mode="embedded")
obj = iris.cls("Ens.StringRequest")._New()

Enable embedded routing with an explicit IRIS installation directory:

import iris

iris.connect(path="/opt/iris")
obj = iris.cls("Ens.StringRequest")._New()

This is useful when IRISINSTALLDIR is not set. On Linux and macOS, the native library path still needs to be configured before Python starts as shown in the environment setup section; path=... configures the wrapper, but it cannot change Unix dynamic loader resolution for already-started processes.

Reset to automatic detection:

import iris

iris.runtime.reset()

DB-API (iris.dbapi)

The wrapper exposes a DB-API facade at iris.dbapi.

Supported subset

• iris.dbapi.connect(...)
• Connection: cursor(), close(), commit(), rollback()
• Cursor: execute(), fetchone(), fetchmany(), fetchall(), iteration, close()
• PEP 249 metadata: apilevel, threadsafety, paramstyle
• PEP 249 exceptions: Error, InterfaceError, OperationalError, and related subclasses

Value normalization

For the embedded %SQL.Statement backend, the wrapper normalizes IRIS SQL/ObjectScript string boundary values to Python values so embedded and remote DB-API behave the same way:

• SQL NULL is returned as Python None
• SQL empty string is returned as Python ""
• Python None passed as a parameter remains SQL NULL
• Python "" passed as a parameter is written as an SQL empty string, not SQL NULL

This normalization is limited to the embedded DB-API path. Native/remote DB-API values are returned by the official driver.

For the native object proxy path (iris.cls(...) with iris.runtime configured for native mode), the wrapper also normalizes declared scalar string properties:

• %String / %RawString scalar properties that come back as None from the native proxy are returned as Python ""
• non-string properties are left unchanged
• collection-valued properties are left unchanged
• arbitrary method return values are left unchanged

Connect modes

iris.dbapi.connect() accepts mode="auto" | "embedded" | "native".

• mode="embedded": forces embedded SQL backend via %SQL.Statement
• mode="native": forces native DB-API backend via the official module iris.dbapi
• mode="auto":
  • if explicit remote arguments are provided (hostname, port, namespace, etc.), uses native
  • if iris.runtime.dbapi is already bound, reuses that DB-API connection
  • otherwise uses embedded (%SQL.Statement) only when runtime policy is not native
  • if iris.runtime is configured for native mode without a bound DB-API connection, raises an error instead of silently falling back to embedded
  • raises an error if embedded runtime is not available

Native resolution uses the official module path iris.dbapi (not intersystems_iris.dbapi).

mode is optional for DB-API.

• With explicit remote arguments (hostname, port, namespace, username, password, etc.), DB-API infers native.
• With iris.runtime.configure(dbapi=conn), DB-API auto mode reuses the bound native connection.
• Without remote arguments or a bound runtime DB-API connection, DB-API auto mode uses embedded unless iris.runtime is explicitly in native mode.

Examples

Embedded mode:

import iris

conn = iris.dbapi.connect()
cur = conn.cursor()
cur.execute("SELECT Name FROM Sample.Person")
rows = cur.fetchall()
cur.close()
conn.close()

Native mode:

import iris

conn = iris.dbapi.connect(
		mode="native",
		hostname="localhost",
		port=1972,
		namespace="USER",
		username="SuperUser",
		password="<password>",
)
cur = conn.cursor()
cur.execute("SELECT 1")
print(cur.fetchone())

Auto mode with explicit remote arguments (routes to native):

import iris

conn = iris.dbapi.connect(
		hostname="localhost",
		port=1972,
		namespace="USER",
		username="SuperUser",
		password="<password>",
)

Auto mode with a runtime-bound native DB-API connection:

import iris

conn = iris.dbapi.connect(
		mode="native",
		hostname="localhost",
		port=1972,
		namespace="USER",
		username="_SYSTEM",
		password="SYS",
)
iris.runtime.configure(dbapi=conn)

same_conn = iris.dbapi.connect(mode="auto")
assert same_conn is conn

Runtime independence

iris.dbapi.connect() is independent from iris.runtime by default.

Calling iris.dbapi.connect(...) does not auto-bind a connection into iris.runtime.dbapi. If you need runtime-managed DB-API binding, bind it explicitly with iris.runtime.configure(dbapi=conn). Once bound, iris.dbapi.connect(mode="auto") reuses that connection instead of creating a new one.

Bind a virtual environment to embedded python in IRIS

You can also bind or unbind an virtual environment to embedded python in IRIS. Here is an example:

bind_iris

Output:

(.venv) demo ‹master*›$ bind_iris
INFO:iris_utils._find_libpython:Created backup at /opt/intersystems/iris/iris.cpf.fa76423a7b924eb085911690c8266129
INFO:iris_utils._find_libpython:Created merge file at /opt/intersystems/iris/iris.cpf.python_merge
up  IRIS              2024.3.0.217.0    1972   /opt/intersystems/iris

Username: SuperUser
Password: ***
IRIS Merge of /opt/intersystems/iris/iris.cpf.python_merge into /opt/intersystems/iris/iris.cpf
IRIS Merge completed successfully
INFO:iris_utils._find_libpython:PythonRuntimeLibrary path set to /usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/Python
INFO:iris_utils._find_libpython:PythonPath set to /demo/.venv/lib/python3.11/site-packages
INFO:iris_utils._find_libpython:PythonRuntimeLibraryVersion set to 3.11

You may have to put your admin credentials to bind the virtual environment to the embedded python in IRIS.

In windows, you must restart the IRIS.

Unbind a virtual environment from embedded python in IRIS

unbind_iris

Output:

(.venv) demo ‹master*›$ unbind_iris
INFO:iris_utils._find_libpython:Created merge file at /opt/intersystems/iris/iris.cpf.python_merge
up  IRIS              2024.3.0.217.0    1972   /opt/intersystems/iris

Username: SuperUser
Password: ***
IRIS Merge of /opt/intersystems/iris/iris.cpf.python_merge into /opt/intersystems/iris/iris.cpf
IRIS Merge completed successfully
INFO:iris_utils._find_libpython:PythonRuntimeLibrary path set to /usr/local/Cellar/python@3.11/3.11.10/Frameworks/Python.framework/Versions/3.11/Python
INFO:iris_utils._find_libpython:PythonPath set to /Other/.venv/lib/python3.11/site-packages
INFO:iris_utils._find_libpython:PythonRuntimeLibraryVersion set to 3.11

Troubleshooting

You may encounter the following error, here is how to fix them.

No module named 'pythonint'

This can occur when the environment variable IRISINSTALLDIR is not set correctly. Make sure that the path is correct.

IRIS_ACCESSDENIED (-15)

This can occur when the service callin is not enabled. Make sure that the service callin is enabled.

IRIS_ATTACH (-21)

This can occur when the user is not the same as the iris owner. Make sure that the user is the same as the iris owner.

irisbuiltins.SQLError: ddtab+82^%qaqpsq

This error can occur when the required libraries are not found. You can fix this by copying the necessary libraries to the Python framework directory:

cp /opt/intersystems/iris/bin/libicudata.69.dylib /usr/local/Cellar/python@3.11/3.11.13/Frameworks/Python.framework/Versions/3.11/Resources/Python.app/Contents/MacOS/
cp /opt/intersystems/iris/bin/libicuuc.69.dylib /usr/local/Cellar/python@3.11/3.11.13/Frameworks/Python.framework/Versions/3.11/Resources/Python.app/Contents/MacOS/