upydev 0.1.5

Command line tool for wireless Micropython devices

uPydev

Command line tool for wireless Micropython devices

uPydev is an acronym of 'Micropython device', and it is intended to be a command line tool to make easier the development, prototyping and testing process of devices based on boards running Micropython.

It is a command line tool for 'wireless Micropython devices' since it make use of the WebREPL protocol to provide communication with and control of the device.

*uPydev is built on top of other tools/scripts which are:

The core is 'webrepl_client.py ' : a Terminal WebREPL protocol as seen in this WebREPL pull request by @Hermann-SW

Other tools are:

'webrepl_cli.py' for the file transfer protocol (from the WebREPL repo of micropython) (modified and named 'upytool')

'esptool.py' to flash the firmware into esp boards

'mpy-cross' to compile .py scripts into .mpy files.

Keep in mind that this project is in ALPHA state, sometimes, some commands may not work/return anything

Features:

• Command line wireless communication/control of micropython devices.
• Terminal WebREPL protocol
• Custom commands to automate communication/control
• Command line autocompletion

Getting Started

First be sure that the WebREPL daemon is enabled and running see webrepl-a-prompt-over-wifi and

webrepl-web-browser-interactive-prompt

Requirements:

WebREPL enabled

Python modules (automatically installed using pip):

argcomplete (for command line autocompletion)

prompt_toolkit (for new WebREPL Terminal implementation)

mpy-cross

esptool

python-nmap

netifaces

requests

upydevice

Tested on:

MacOS X (Mojave 10.14.5-6)

Raspbian GNU/Linux 9 (stretch) (through ssh session)

For Raspbian pip install mpy-cross seems to fail, so to install upydev without mpy-cross do:

$ git clone https://github.com/Carglglz/upydev.git
[...]
$ cd upydev
$ sudo pip3 install . --no-deps -r rpy_rqmnts.txt

Then to install upydevice see instructions in upydevice repo

upy Boards:

Esp32 Huzzah feather

Esp8266 Huzzah feather

Installing using pip:

$ pip install upydev

Quick Reference:

Help:

In CLI do :

$ upydev -h

Configurate uPy Device:

1st step: Configurate upy device target and password:

• To save configuration in working directory:

  $ upydev config -t [UPYDEVICE IP] -p [PASSWORD]

  example:

  $ upydev config -t 192.168.1.58 -p mypass

• To save configuration globally use -g option:

  $ upydev config -t [UPYDEVICE IP] -p [PASSWORD] -g

  example:

  $ upydev config -t 192.168.1.58 -p mypass -g

upydev will use local working directory configuration unless it does not find any or manually indicated with -g option.

---

uPydev Usage:

Usage:

$ upydev [Mode] [options] or upydev [upy command] [options]

This means that if the first argument is not a Mode keyword or a upy command keyword it assumes it is a 'raw' upy command to send to the upy device

Example: Mode

$ upydev put -f dummy.py

Example: uPy command

$ upydev info

Example: Raw commands

$ upydev "my_func()"

$ upydev 2+1

$ upydev "import my_lib;foo();my_var=2*3"

---

uPydev Mode/Tools:

• config : to save upy device settings (see -p, -t, -g), so the target and password arguments wont be required any more

• put : to upload a file to upy device (see -f, -s , -dir, -rst; for multiple files see -fre option)

• get : to download a file from upy device (see -f , -dir, -s; for multiple files see -fre option)

• sync : for a faster transfer of large files (this needs sync_tool.py in upy device) (see -f, -s and -lh; for multiple files see -fre option)

  > sync_tool.py is under upyutils directory

• d_sync: to recursively sync a folder in upydevice filesystem use -dir to indicate the folder (must be in cwd), use '-tree' to see dir structure, or '-s sd' to sync to an Sd card mounted as 'sd'

• cmd : for debugging purpose, to send command to upy device ; (see -c, -r, -rl);

  • Examples:

  $ upydev cmd -c "led.on()"

  $ upydev cmd -r "print('Hello uPy')"

  $ upydev cmd -rl "function_that_print_multiple_lines()"

  • tip: simple commands can be used without quotes; but for commands with parenthesis or special characters use quotes, for example: 'dummy_func()' ; use double quotes "" when the command includes a string like this example: "uos.listdir('/sd')"

• wrepl :to enter the terminal WebREPL; CTRL-x to exit, CTRL-d to do soft reset To see more keybinding info do CTRL-k (Added custom keybindings and autocompletion on tab to the previous work see: Terminal WebREPL for the original work)

• srepl : to enter the terminal serial repl using picocom, indicate port by -port option (to exit do CTRL-a, CTRL-x) (see: Picocom for more information)

• ping : pings the target to see if it is reachable, CTRL-C to stop

• run : just calls import 'script', where 'script' is indicated by -f option (script must be in upy device or in sd card indicated by -s option and the sd card must be already mounted as 'sd');

  ​ Supports CTRL-C to stop the execution and exit nicely.

• install : install libs to '/lib' path with upip; indicate lib with -f option

• mpyx : to froze a module/script indicated with -f option, and save some RAM, it uses mpy-cross tool (see mpy-cross for more information)

• timeit: to measure execution time of a module/script indicated with -f option.

  This is an adapted version of timed_function

• fw: to list or get available firmware versions, use -md option to indicate operation: - to list do: "upydev fw -md list -b [BOARD]" board should be 'esp32' or 'esp8266' (web scraping from micropython downloads page ) - to get do: "upydev fw -md get [firmware file]" (uses curl) - to see available serial ports do: "upydev fw -md list serial_ports"

• flash: to flash a firmware file to the upydevice, a serial port must be indicated to flash do: "upydev flash -port [serial port] -f [firmware file]"

• see: to get specific command help info indicated with -c option

• find: to get a list of possible upy devices. Scans the local network to find devices with port 8266 (WebREPL) open. Use -n option to perform n scans (A single scan may not find all the devices)

• diagnose: to make a diagnostic test of the device (sends useful commands to get device state info)

• errlog: if 'error.log' is present in the upydevice, this shows the content (cat('error.log')).

  If 'error.log' in sd use -s sd (This command needs upysh installed, do upydev install -f upysh)

• stream_test: to test download speed (from device to host). Default test is 10 MB of random bytes are sent in chunks of 20 kB and received in chunks of 32 kB. To change test parameters use -chunk_tx , chunk_rx, and -total_size.

• sysctl: to start/stop a script without following the output. To follow initiate wrepl/srepl as normal, and exit with CTRL-x (webrepl) or CTRL-A,X (srepl) TO START: use -start [SCRIPT_NAME], TO STOP: use -stop [SCRIPT_NAME]

• log: to log the output of a upydevice script, indicate script with -f option, and the sys.stdout log level and file log level with -dslev and -dflev (defaults are debug for sys.stdout and error for file). To log in background use -daemon option, then the log will be redirected to a file with level -dslev. To stop the 'daemon' log mode use -stopd and indicate script with -f option. 'Normal' file log and 'Daemon' file log are under .upydev_logs folder in $HOME directory, named after the name of the script. To follow an on going 'daemon' mode log, use -follow option and indicate the script with -f option.

• make_group: to make a group of boards to send commands to. Use -f for the name of the group

  ​ and -devs option to indicate a name, ip and the password of each board. (To store the group settings globally use -g option)

• mg_group: to manage a group of boards to send commands to. Use -G for the name of the group and -add option to add devices (indicate a name, ip and the password of each board) or -rm to remove devices (indicated by name)

GROUP COMMAND MODE (-G option):

To send a command to multiple devices in a group (made with make_group command) use -G option

Usage: upydev [command] -G [GROUP NAME]

To target specific devices within a group add -devs option as -devs [DEV_1 NAME] [DEV_2 NAME]

upydev will use local working directory configuration unless it does not find any or manually indicated with -g option

GROUP COMMAND PARALLEL MODE (-GP option):

To send a command at the same time to multiple devices in a group (made with make_group command) use -GP option.

**Be aware that not all the commands are suitable for parallel execution (wrepl for example)

Usage: upydev [command] -GP [GROUP NAME]

To target specific devices within a group add -devs option as -devs [DEV_1 NAME] [DEV_2 NAME]

upydev will use local working directory configuration unless it does not find any or manually indicated with -g option

---

uPydev Commands:

uPy commands are organized in:

• General: These commands should work 'out of the box' in any Micropython running board with WebREPL daemon enabled.

• Wifi utils : This commands make easier to save/load wifi configuration (STA and AP ) and connect to an access point or enable its own (needs wifiutils.py in upydevice, see upyutils directory)

• SD: These commands need sdcard.py in the upy device, and a sd module/shield at least.

• INPUT: These commands need a specific sensor module and the appropriate script in the upydevice (All these scripts are under upyutils directory)

  • ADC: commands that make use of the ADCs from the board, or an external ADC module (ADS1115) (for external module needs 'ads1115.py' and 'init_ADS.py')
  • IMU: commands that make use of the LSM9DS1 module, although other IMU modules could be easily implemented (needs 'lsm9ds1.py' and 'init_MY_IMU.py')
  • WEATHER: commands that make use of the BME280 module, although other weather sensor modules could be easily implemented (needs 'bme280.py' and 'init_BME280.py')
  • POWER: commands that make use of the INA219 module.(needs 'ina219.py' and 'init_INA219.py')

• OUTPUT: These commands use the DAC or PWM of the board, some needs an actuator module (buzzer or motor driver and a motor) at least and the appropriate script in the upydevice.

  • DAC: to generate an analog signal (dc value, sine wave or square wave at the momment) (needs 'dac_signal_gen.py')
  • BUZZER: to drive a buzzer with PWM (needs 'buzzertools.py')
  • DC MOTOR: to control a DC motor (needs a motor driver and the appropriate script) (needs 'dcmotor.py')
  • SERVO: to drive a servo motor (needs 'servo.py')
  • STEPPER MOTOR: to drive stepper motor (needs a motor driver and 'stepper.py')

• NETWORKING:

  • MQTT: commands to connect to a broker, subscribe to topic, publish and receive messages (needs 'mqtt_client.py')
  • SOCKETS: commands to start client/server socket and send/receive messages (needs 'socket_client_server.py')
  • UREQUEST: commands to make http requests, and get json or text output

• PORT/BOARD SPECIFIC COMMANDS:

  • battery : if running on battery, gets battery voltage (esp32 huzzah feather)
  • pinout : to see the pinout reference/info of a board, indicated by -b option, to request a single or a list of pins info use -po option (currently just esp32 huzzah feather)
  • specs : to see the board specs, indicated by -b option (currently just esp32 huzzah feather)
  • pin_status: to see pin state, to request a specific set use -po option

---

Extensive explanation:

For an extensive explanation and commands demo see Documentation.

Addiontal Scripts for some commands:

The commands that need additional scripts in the upy device are under the uPyutils folder.

For more info see upyutils_docs.

---

USEFUL DEVELOPER TOOLS: (Under upyutils folder)

• upylog: MicroPython logging module with time format (predefined) and log to file support.
• upynotify : module with NOTIFIER class to notify events with beeps and blinks.

---

HOW TO:

• ATOM/VSC INTEGRATION (PLATFORMIO TERMINAL)

  • ATOM:

    To do this go to Atom Settings --> Packages --> Then search for platformio-ide-terminal and click on Settings. Here go to Custom Texts section: (There are up to 8 "custom texts" or commands that can be customised) These custom text will be pasted an executed in the Terminal when called. And this can be done with keybindings or key-shortcuts. For example:

    • To automate upload the current file:

      In Custom text 1 write: upydev put -f $F

    • To automate run the current file:

      In Custom text 2 write: upydev run -f $F

    • To automate open the wrepl:

      In Custom text 3 write: upydev wrepl

    • To automate diagnose:

      In Custom text 4 write: upydev diagnose

    Now configure the Keybindings, to do this go to Settings --> Keybindings --> your keymap file

    Then in keymap.cson add: (This is an example, the key combination can be changed)

    'atom-workspace atom-text-editor:not([mini])':
      'ctrl-shift-d': 'platformio-ide-terminal:insert-custom-text-4'
      'ctrl-cmd-u': 'platformio-ide-terminal:insert-custom-text-1'
      'ctrl-cmd-x': 'platformio-ide-terminal:insert-custom-text-2'
      'ctrl-cmd-w': 'platformio-ide-terminal:insert-custom-text-3'
    

    Save the file and now when pressing these key combinations should paste the command and run it in the Terminal.

  • VSC:

    Using tasks and adding the shortcut in keybinds.json file for example:

    Task:

    "version": "2.0.0",
        "tasks": [
            {
                "label": "upydev_upload",
                "type": "shell",
                "command": "upydev",
                "args": ["put", "-f", "${file}"],
                "options": {
                    "cwd": "${workspaceFolder}"
                },
                "presentation": {
                    "echo": true,
                    "reveal": "always",
                    "focus": true,
                    "panel": "shared",
                    "showReuseMessage": true,
                    "clear": false
                },
                "problemMatcher": []
            }, ]
    

    Keybinding.json

    {
            "key": "ctrl+cmd+u",
            "command": "workbench.action.tasks.runTask",
            "args": "upydev_upload"
        }
    

• DEBUG :

  *RECOMMENDATION:

  Since upydev is based in a wireless protocol connection, in order to succes sending upydev commands make sure that there is a reliable connection between the host and the device* **and that the wifi signal strength (rssi) in the device is above -80 ** (below -80 performance could be inconsistent)

  A 'Reliable' connection means that there is no packet loss (use ping or upydev ping command to check)

  See https://en.wikipedia.org/wiki/Received_signal_strength_indication and

  https://wiki.teltonika.lt/view/Mobile_Signal_Strength_Recommendations.

  *TRACKING PACKETS:

  To see if "command packets" are sent and received or lost use wireshark and filter the ip of the device

  * SEE WHAT'S GOING ON UNDER THE HOOD:

  *(THIS NEEDS THAT THE HOST AND THE DEVICE TO BE CONNECTED BY USB)

  In a terminal window open a 'serial repl' with upydev srepl --port /dev/tty.[USBPORT] command

  In another window use upydev normally. Now in the terminal window with the serial repl you can see what commands are sent.