VEDA TensorFlow

VEDA TensorFlow is a library to add device support for the NEC SX-Aurora TSUBASA into TensorFlow using the Pluggable Device API.

Python Versions Maintenance

Release Notes

Version	Comment
v7	Added TF v2.13.* support Added TF v2.12.* support Fixed <v2.10.* support
v6	Added TF v2.11.* support Added TF v2.10.* support Upgraded to VEDA CPP API
v5	Added TF v2.9.* support
v4	Added BroadcastTo operation Increased `host_memory_allocate` alignment to be 64, as lower values keep failing in `isAligned()`
v3	Bugfixes for loss functions Added missing optimizers: SGD, Adadelta, Adagrad, Adam, and Adamax Fixed possible segfault in PluggableDevice `host_memory_allocate`
v2	Minor changes to enable TF v2.7.1 and v2.8.0 Fixed vedaInit error checking to ignore if already initialized
v1	Initial Release

F.A.Q.

I get the error message: "Internal: platform is already registered with name: "NEC_SX_AURORA"

This error is caused by the combination of RH-Python38 package and using a VirtualEnv. Due to improper checking for symlinks in TensorFlow the device support library gets loaded and initialized twice causing this error message.

You can use the following workaround as long as the bug is not resolved in TensorFlow.

# BEGIN BUGFIX
import sys
import os

sys.path = list(set(os.path.realpath(p) for p in sys.path))

import site
getsitepackages = site.getsitepackages
def getsitepackages_(prefixes=None):
    return list(filter(lambda x: 'lib64' not in x, getsitepackages(prefixes)))
site.getsitepackages = getsitepackages_
# END BUGFIX

import tensorflow
...

I get the error message "tensorflow.python.framework.errors_impl.InvalidArgumentError: 'visible_device_list' listed an invalid Device id '1' but visible device count is 1"

This is a known problem within TF due to TF throws: "'visible_device_list' listed an invalid Device id" when using non-GPU PluggableDevices when using CUDA and VE devices at the same time. The VE devices get added to list of GPUs, ultimately creating invalid devices indices.

Either you need to manually patch your TF installation (see the TF issue), or use VEDA_VISIBLE_DEVICES=100 or CUDA_VISIBLE_DEVICES= to disable either the CUDA or VE devices.

Release files for veda-tensorflow 2.13.0.post7

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distribution (wheel)

Table of built distributions (wheels) for veda-tensorflow 2.13.0.post7
File	Interpreter	ABI	Platform	Reset
veda_tensorflow-2.13.0.post7-py3-none-manylinux_2_24_x86_64.whl 316.1 kB Aug 4, 2023	Python 3	none	Linux glibc 2.24+ x86-64	Details

Release files / veda_tensorflow-2.13.0.post7-py3-none-manylinux_2_24_x86_64.whl

Download URL	veda_tensorflow-2.13.0.post7-py3-none-manylinux_2_24_x86_64.whl
Size	316.1 kB
Tags	Linux glibc 2.24+ x86-64 Python 3
SHA-256 checksum How to use checksums	`e699a5827e4790ed5470c2591dc2b1a42313f4c9ced16a4a03e80e3acb471747`
BLAKE2b-256 checksum How to use checksums	`5d56b9976d9ad28febbb641d1c882f1613a4674ef895804da5a7898f4829803c`
Upload date	Aug 4, 2023
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/4.0.2 CPython/3.10.6`