Tool for generating App Inventor extension skeletons from Tensorflow.js models
MIT App Inventor TFJS Extension Generator
The aim of this tool is to make it easier to generate the scaffolding needed to use a Tensorflow.js model in App Inventor.
- java 8
- ant 1.10
- python 3
Install the App Inventor TFJS extension generator using pip:
pip install appinventor-tfjs
Create an extension prototype for Posenet:
python -m appinventor.tfjs posenet edu.mit.appinventor.ai.posenet.PosenetExtension
The output of this command will be a new directory called PosenetExtension. Within this directory, you will find a fresh git clone of the App Inventor extension template repository. The directory will have the following structure:
build.xml lib ├─ android │ ├─ android.jar │ ├─ appcompat-v7-28.0.0.jar │ └─ dx.jar ├─ ant-contrib │ └─ ant-contrib-1.0b3.jar ├─ appinventor │ └─ AndroidRuntime.jar │ └─ AnnotationProcessors.jar └─ deps README.md src └─ edu └─ mit └─ appinventor └─ ai └─ posenet ├─ assets │ ├─ app.js │ ├─ group1-shard1of2.bin │ ├─ group1-shard2of2.bin │ ├─ index.html │ ├─ model-stride16.json │ ├─ posenet.min.js │ ├─ tf-converter.min.js │ ├─ tf-core.min.js │ └─ VERSIONS └─ PosenetExtension.java
Of those files, the ones under
src are most interesting. Briefly:
PosenetExtension.java- Boilerplate extension code for a TFJS extneion in App Inventor. You will want to customize it to provide model-specific behavior, such as interpreting the results before passing information back to the blocks layer.
group-*.bin- These are the weights at each level of the model, pulled from the TFJS model repository. The number of files will vary based on the size of the model.
VERSIONS- The VERSIONS file contains a key-value mapping the different npm modules to the versions that were retrieved. There should be one entry per min.js file.
usage: python -m appinventor.tfjs [-h] [--scope SCOPE] model_name class_name Create a TensorFlow.js-based extension for MIT App Inventor. positional arguments: model_name class_name optional arguments: -h, --help show this help message and exit --scope SCOPE
model_name argument names the Tensorflow.js model of interest. A list of pretrained models is available on GitHub. For example, if you are interested in trying the MobileNet model, you would specify
mobilenet as the
class_name argument specifies a fully qualified Java class name that will be used for the extension. For example, a MobileNet extension for App Inventor might have the fully qualified class name
com.example.tfjs.mobile.MobileNetExtension. The extension generator will create this class and any intermediate packages for you.
--scope SCOPE argument allows you to import models from npm packages that are not under the
@tensorflow-models namespace (effectively, if
--scope is not specified it is the same as
You will need to create a virtual environment and install the dependencies by running. We provide instructions for macOS below.
Create a virtualenv
brew install pyenv echo "eval \"\$(pyenv init -)\"" >> ~/.bash_profile echo "eval \"\$(pyenv virtualenv-init -)\"" >> ~/.bash_profile source ~/.bash_profile
Create a python environment using pyenv and activate it
pyenv install 3.6 pyenv virtualenv 3.6 appinventor3 pyenv activate appinventor3
pip install -r requirements.txt pip install .
This software is made available under the Apache Software License 2.0. You are welcome to contribute pull requests to this project via GitHub.
Copyright 2020 Massachusetts Institute of Technology
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Hashes for appinventor_tfjs-0.1.4-py3-none-any.whl