IVA TPU Python API

Main entities

TPUDevice

TPUDevice is a device handle

TPUProgram

TPUProgram contains TPU instructions and weigths data

TPUInference

TPUInference contains input/output data

Example

import asyncio
import numpy as np
from iva_tpu import TPUDevice, TPUProgram, TPUInference

from iva_applications.resnet50 import image_to_tensor
from iva_applications.imagenet import tpu_tensor_to_classes
from PIL import Image

image = Image.open('ILSVRC2012_val_00000045.JPEG')
tensor = image_to_tensor(image)

device = TPUDevice()
program = TPUProgram("resnet50.tpu") #default TPUProgramInfo is totally fine
device.load_program(program)
inference = TPUInference(program)
inference.load_tensors([tensor])
status_future = device.load_inference(inference)  # device returns future for inference status
event_loop = asyncio.get_event_loop()
status = event_loop.run_until_complete(status_future)
assert status.is_success # check that there is no errors during inference
output = inference.get_tensors() # get results
tpu_tensor_to_classes(output[0], top=1)

TPU Dictionary interface

...
program = TPUProgram("resnet50.tpu")
inference = TPUInference(program)
inference.load_tensors_dict({"Placeholder:0": tensor})
...
assert status.is_success
output = inference.get_tensors_dict()
tpu_tensor_to_classes(output["logits:0"], top=1)

TPU Blocking interface

status = device.load_inference_sync(inference) #would block until completion

TPU Raw buffer examples

import asyncio
from iva_tpu import TPUDevice, TPUProgram, TPUInference
program = TPUProgram("omega_program_dnn_quant_3.0.0.tpu")
device = TPUDevice()
device.load_program(program)
inference = TPUInference(program)

with open("f.bin", "rb") as f:
    buf=f.read()

inference.load_raw([buf])
asyncio.get_event_loop().run_until_complete(device.load_inference(inference))
outputs = inference.get_raw()

for i in range(3):
  o = outputs[i]
  with open(f"o{i}.bin", "wb") as f:
    f.write(o)

Release files for iva-tpu 11.2.6

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for iva-tpu 11.2.6
File	Size	Uploaded
iva-tpu-11.2.6.tar.gz	2.2 kB	Aug 31, 2021	Details

Release files / iva-tpu-11.2.6.tar.gz

Download URL	iva-tpu-11.2.6.tar.gz
Size	2.2 kB
Tags	Source
SHA-256 checksum How to use checksums	`885cb9ef75a87fb83dcf988d8e8664477a4bccca5f058feaede2130dc94a63ff`
BLAKE2b-256 checksum How to use checksums	`64ee0c5f2b71f8fc94771a638dc1300dd9a9c6e530701734ff405249111a3a26`
Upload date	Aug 31, 2021
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/3.4.1 importlib_metadata/3.7.3 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.59.0 CPython/3.6.8`