Skip to main content

Command line interface for Exosite platform.

Project description

Exoline: Exosite Command Line
=============================

Exoline is command line tool for working with the Exosite [One Platform](http://exosite.com/products/onep).

Installation
------------

To install the latest released version of exoline from PyPI:

```bash

$ pip install exoline
```

Alternatively, you can install from source:

```bash

$ git clone git://github.com/exosite/exoline
$ cd exoline
$ python setup.py install
```

Depending on your Python environment, you may need to run the above install commands as sudo. [virtualenvwrapper](http://virtualenvwrapper.readthedocs.org/en/latest/) is a great way to manage Python environments.

Usage
-----

```

$ exo --help
Exosite RPC API Command Line Interface
Provides command line access to the Remote Procedure Call API:
https://github.com/exosite/api/tree/master/rpc

Usage:
exo [--help] [options] <command> [<args> ...]

Commands:
read Read data from a resource.
write Write data at the current time.
record Write data at a specified time.
create Create a resource from a json description passed on stdin,
or using reasonable defaults.
listing List the RIDs of a client's children.
info Get metadata for a resource in json format.
update Update a resource from a json description passed on stdin.
map Add an alias to a resource.
unmap Remove an alias from a resource.
lookup Look up a resource's RID based on its alias or cik.
drop Drop (permanently delete) a resource.
flush Remove all time series data from a resource.
usage Display usage of One Platform resources over a time period.
tree Display a resource's descendants.
script Upload a Lua script
spark Show distribution of intervals between points.
copy Make a copy of a client.
diff Show differences between two clients.
ip Get IP address of the server.
data Read or write with the HTTP Data API.
spec Determine whether a client matches a specification (beta)

Options:
--host=<host> OneP URL. Default is $EXO_HOST or m2.exosite.com
--port=<port> OneP port. Default is $EXO_HOST or 80
--httptimeout=<sec> HTTP timeout [default: 60]
--https Enable HTTPS (deprecated, HTTPS is default)
--http Disable HTTPS
--debug Show info like stack traces
--debughttp Turn on debug level logging in pyonep
--discreet Obfuscate RIDs in stdout and stderr
-h --help Show this screen
-v --version Show version

See 'exo <command> --help' for more information on a specific command.
```

Examples
--------

Here are a few things you can do with Exoline.

* Show a tree view of a client

```bash
$ exo --discreet tree 5de0cfcf7b5bed2ea7a801234567890123456789
Dev client cik: 5de0cfcf7b5bed2ea7a801234567890123456789 (aliases: (see parent))
├─device1 client cik: 970346d3391a2d8c703a01234567890123456789 (aliases: [u'device1'])
└─device2 client cik: e95052ab56f985e6807d01234567890123456789 (aliases: [u'device2'])
└─json string dataport rid: 82209d5888a3bd1530d201234567890123456789 (aliases: [u'json'])

```

* Upload a Lua script

```bash

$ exo script translate_gps.lua e469e336ff9c8ed9176bc05ed7fa40daaaaaaaaa
Updated script RID: 6c130838e14903f7e12d39b5e76c8e3aaaaaaaaa
```

* Monitor output of a script

```bash

$ exo read e469e336ff9c8ed9176bc05ed7fa40daaaaaaaaa translate_gps.lua --follow
2013-07-09 11:57:45,line 2: Running translate_gps.lua...
2013-07-09 12:00:17,"line 12: New 4458.755987,N,09317.538945,W
line 23: Writing 4458.755987_-09317.538945"
2013-07-09 12:15:41,"line 12: New 4458.755987,N,09317.538945,W
line 23: Writing 4458.755987_-09317.538945"
```

* Write raw data

```bash

$ exo write e469e336ff9c8ed9176bc05ed7fa40daaaaaaaa gps-raw --value=4458.755987,N,09317.538945,W
```

* Record a bunch of data without timestamps

```bash

$ cat myrawgps | exo record e469e336ff9c8ed9176bc05ed7fa40daaaaaaaaa gps-raw -
```

* Dump data from multiple dataports to CSV

```bash

$ time ./exo.py read 2ca4f441538c1f2cc8bfaaaaaaaaaaaaaaaaaaaa gas temperature humidity event --start=5/1/2013 --end=8/1/2013 --chunkhours=24 > alldata.csv

real 1m58.377s
user 0m10.981s
sys 0m0.506s

$ wc -l alldata.csv
316705 alldata.csv
```

* Make a copy of a device

```bash

$ exo copy e469e336ff9c8ed9176bc05ed7fa40daaaaaaaaa ed6c3facb6a3ac68c4de9a6996a89594aaaaaaaa
cik: c81e6ae0fbbd7e9635aa74053b3ab6aaaaaaaaaa
```

* Create a new client or resource

```bash

$ ../exoline/exo.py create ad02824a8c7cb6b98fdfe0a9014b3c0faaaaaaaa --type=dataport --format=string --name=NewString
rid: 34eaae237988167d90bfc2ffeb666daaaaaaaaaa
```

* Show differences between two clients

```bash

$ exo copy 3ae52bdd5280d7cb96a2077b0cd5aaaaaaaaaaaa 5de0cfcf7b5bed2ea7a802ebe0679baaaaaaaaaa
cik: cc080a86b1c9b53d5371e0fa793faaaaaaaaaaa
$ exo diff 3ae52bdd5280d7cb96a2077b0cd5aaaaaaaaaaaa cc080a86b1c9b53d5371e0fa793f1daaaaaaaaaa
$ exo create cc080a86b1c9b53d5371e0fa793f1aaaaaaaaaaa --type=dataport --format=float --name=Humidity
rid: 6a8974d3d7d1f0ffd28385c90a1bebaaaaaaaaaa
$ ../exoline/exo.py diff 3ae52bdd5280d7cb96a2077b0cd5dbaaaaaaaaaa cc080a86b1c9b53d5371e0fa793f1daaaaaaaaaa
{
"<<RID>>": {
"aliases": {
"<<RID>>": [
"temp"
]
},
"basic": {
"subscribers": 0,
"type": "client"
},
"children": {
"<<RID>>": {
+ "basic": {
+ "subscribers": 0,
+ "type": "dataport"
+ },
+ "children": {},
+ "comments": [],
+ "description": {
+ "format": "float",
+ "meta": "",
+ "name": "Humidity",
+ "preprocess": [],
+ "public": false,
+ "retention": {
+ "count": "infinity",
+ "duration": "infinity"
+ },
+ "subscribe": null
+ },
+ "shares": [],
+ "subscribers": [],
+ "tags": []
+ },
+ "Temperature.f2a40b81cb677401dffdc2cfad0f8a266d63590b": {
"basic": {
"subscribers": 0,
"type": "dataport"
},
"children": {},
"comments": [],
"description": {
"format": "float",
"meta": "",
"name": "Temperature",
"preprocess": [],
"public": false,
"retention": {
"count": "infinity",
"duration": "infinity"
},
"subscribe": null
},
"shares": [],
"subscribers": [],
"tags": []
}
},
"comments": [],
"counts": {
"client": 0,
- "dataport": 1,
? ^
+ "dataport": 2,
? ^
"datarule": 0,
"dispatch": 0
},
"description": {
"limits": {
"client": "inherit",
"dataport": "inherit",
"datarule": "inherit",
"disk": "inherit",
"dispatch": "inherit",
"email": "inherit",
"email_bucket": "inherit",
"http": "inherit",
"http_bucket": "inherit",
"share": "inherit",
"sms": "inherit",
"sms_bucket": "inherit",
"xmpp": "inherit",
"xmpp_bucket": "inherit"
},
"locked": false,
"meta": "",
"name": "MyDevice",
"public": false
},
"shares": [],
"subscribers": [],
"tagged": [],
"tags": []
}
}
```

See the HTTP requests and responses being made by pyonep:

```bash

$ exo --debughttp --discreet read <cik> temperature
DEBUG:pyonep.onep:POST /api:v1/rpc/process
Host: m2.exosite.com:80
Headers: {'Content-Type': 'application/json; charset=utf-8'}
Body: {"calls": [{"id": 70, "procedure": "read", "arguments": [{"alias": "temperature"}, {"sort": "desc", "selection": "all", "limit": 1, "endtime": 1376943416, "starttime": 1}]}], "auth": {"cik": "2ca4f441538c1f2cc8bf01234567890123456789"}}
DEBUG:pyonep.onep:HTTP/1.1 200 OK
Headers: [('date', 'Mon, 19 Aug 2013 20:16:53 GMT'), ('content-length', '54'), ('content-type', 'application/json; charset=utf-8'), ('connection', 'keep-alive'), ('server', 'nginx')]
Body: [{"id":70,"status":"ok","result":[[1376819736,24.1]]}]
2013-08-18 04:55:36,24.1
```


Environment Variables
---------------------

For convenience, several command line options may be replaced by environment variables.

* EXO\_HOST: host, e.g. m2.exosite.com. This supplies --host to exo and --url for exodata.
* EXO\_PORT: port, e.g. 80. Currently this only applies to exo, not exodata.


CIK Shortcuts
-------------

Store your commonly used CIKs in a file:

```bash

$ printf "keys:\n" > ~/.exoline
$ printf " foo: 2ca4f441538c1f2cc8bf01234567890123456789\n" >> ~/.exoline
$ exo read foo temperature
2013-08-18 04:55:36,24.1
```

Help
----

For help, run each command with -h from the command line.


Usage as a Library
------------------

Exoline can be directly imported and used in Python as a library. There are two patterns
for doing this. First, you can call `exo.run` with whatever arguments you would have
passed on the command line, plus an optional string stdin parameter.

```python

from exoline import exo

result = exo.run(['exo',
'script',
'scripts/myscript.lua',
'ad02824a8c7cb6b98fdfe0a9014b3c0faaaaaaaa'])

print(result.exitcode) # 0
print(result.stdout) # Updated script RID: c9c6daf83c44e44985aa724fea683f14eda71fac
print(result.stderr) # <no output>
```

It's also possible to use Exoline's wrapper for the pyonep library, which covers a lot of
Exoline's functionality.

```python

from exoline import exo

rpc = exo.ExoRPC()

rpc.upload_script(ciks=['ad02824a8c7cb6b98fdfe0a9014b3c0faaaaaaaa'],
filename='scripts/myscript.lua')
```


Issues/Feature Requests
-----------------------

If you see an issue with exoline or want to suggest an improvement, please log it [here](https://github.com/exosite/exoline/issues).


Test
----

To run the tests, install the packages in test/requirements.txt, and then type:

```bash

$ cd test
$ source test.sh
```

TODO
----

- copy command should support taking input from stdin (which would be the ouput of info --recursive)
- investigate copying resources with public: True to destination with public: False (One Platform error: restricted)
- clarify what the first and second parameter to copy need to be. Maybe require explicit --device= and --portal= They could be anything, but 95% of the time the first param will be a device CIK, the second a portal CIK.
- --name parameter to copy command so names don't conflict
- --desconly parameter for info command, to show info as json so it can be piped to create
- add raw command, taking full RPC json from stdin
- Make the info command take multiple rids (or stdin)
- delete serial number when dropping device
- add --howmany option to create command
- tab completion for commands and shortcuts
- add create command shorthand: "exo create float foo" to create a dataport of format float with alias and name both set to foo
- add create clone support
- add test for --tz option



History
=======

0.5.0 (2013-11-21)
------------------

- remove --counts option to tree command
- remove storage option to the info command

0.4.3 (2013-11-19)
------------------

- make second parameter to exo.cmd optional
- restore std* so stdout is visible after calling exo.cmd()

0.4.2 (2013-11-13)
------------------

- spec command support for units and json format validation
- example spec file

0.4.1 (2013-11-11)
------------------

- add activate command
- fix spec message for dataport format differences
- add documentation of spec command yaml syntax
- fix data write to handle urlencode characters (e.g. %)

0.4.0 (2013-10-30)
------------------

- use https by default, specify --http for http
- fix issue where read --follow could not be piped to other commands due to stdout buffering
- show commands in a consistent order in 'exo --help'
- show command summaries in 'exo --help'

0.3.6 (2013-10-29)
------------------

- read command defaults to reading all dataports/datarules if no RIDs are specified
- listing command outputs valid JSON

0.3.5 (2013-10-28)
------------------

- reuse connection to speed up API calls

0.3.4 (2013-10-10)
------------------

- default to utc if local timezone can't be determined
- fix timezone bug in read output

0.3.3 (2013-10-4)
-----------------

- decode scripts as utf-8 for spec command

0.3.2 (2013-10-4)
-----------------

- remove plugin dependency on script install location

0.3.1 (2013-10-1)
-----------------

- fix install issue

0.3.0 (2013-9-30)
-----------------

- add plugin framework
- update tree output, incl. sort by client name
- add spec command as a plugin (beta)
- make listing default to all resource types
- timezone support for read command

0.2.6 (2013-9-19)
-----------------

- fixed update command

0.2.5 (2013-8-26)
-----------------

- record reads csv on stdin
- fixed read --sort=asc
- fixed --follow order when multiple values come within the polling window

0.2.4 (2013-8-19)
-----------------

- fixed combination of --debughttp and --discreet

0.2.3 (2013-8-19)
-----------------

- --debughttp shows http requests & responses
- --discreet hides ciks/rids
- documented usage as library

0.2.2 (2013-8-16)
-----------------

- --header option for read command

0.2.1 (2013-8-15)
-----------------

- cik lookup in ~/.exoline
- support ISO8601 dates for read
- copy comments

0.2.0 (2013-8-13)
-----------------

- tree is faster for large portals
- --level option for tree
- copy checks limits when possible (when not set to 'inherit')
- improve json format for info --recursive

0.1.3 (2013-8-9)
----------------

- set up for automated testing in jenkins
- --include and --exclude flags for info
- info and listing commands output json when using --pretty
- --recursive flag for script command
- fixed regression in read --follow

0.1.2 (2013-7-31)
-----------------

- added --port option
- added --chunkhours option to break up large reads

0.1.1 (2013-7-30)
-----------------

- fixed --httptimeout
- show model and serial number from metadata in tree output

0.1.0 (2013-7-24)
-----------------

- read from multiple data sources
- copy command (make a copy of a client)
- diff command (compare clients)
- --recursive option for info

0.0.33 (2013-7-19)
------------------

- support python 2.6

0.0.32 (2013-7-19)
-----------------

- lookup command looks up RID of CIK if no alias is passed
- fixed exception


0.0.31 (2013-7-18)
------------------

- updated to use pyonep 0.7.0
- added usage command

0.0.30 (2013-7-16)
------------------

- fixed regression in tree

0.0.29 (2013-7-16)
------------------

- fixed pyonep reference

0.0.28 (2013-7-16)
------------------

- usage command
- Better test coverage

0.0.27 (2013-7-14)
------------------

- Support uploading script to multiple CIKs
- Added code coverage for tests
- read --intervals shows the distribution of
delays between points

0.0.26 (2013-7-12)
------------------

- Fixed https port

0.0.25 (2013-7-12)
------------------

- Added --https flag

0.0.24 (2013-7-12)
------------------

- Added raw read format

0.0.23 (2013-7-12)
------------------

- Made <rid> optional for all commands
- Added root node detail output to tree

0.0.22 (2013-7-11)
------------------

- Bumped up version requirement for pyonep

0.0.21 (2013-7-11)
------------------

- Fixed tree output for devices with expired status
- Hide KeyboardInterrupt exception except when --debug set

0.0.20 (2013-7-10)
------------------

- Fixed script command

0.0.19 (2013-7-10)
------------------

- Fixed README.md

0.0.18 (2013-7-10)
------------------

- Help for individual commands, git style
- Fixed regression in 0.0.17 affecting all commands taking <rid>
- record-backdate is now record with --interval


0.0.17 (2013-7-09)
------------------

- Handle keyboard interrupt gracefully for read --follow
- Added example usage in README.md
- Fixed read --follow when dataport has no data

0.0.16 (2013-7-08)
------------------

- Support passing alias for <rid>
- Make read return latest value by default

0.0.15 (2013-7-08)
------------------

- script upload

0.0.14 (2013-7-07)
------------------

- tests for create, read, write

0.0.13 (2013-7-03)
------------------

- record, unmap, lookup commands, better/stronger/faster tree

0.0.12 (2013-6-27)
------------------

- Show OnePlatform library exceptions nicely

0.0.11 (2013-6-27)
------------------

- Changed defaults for tree

0.0.10 (2013-6-27)
------------------

- flush command

0.0.9 (2013-6-26)
-----------------

- Added format to tree output


0.0.8 (2013-6-26)
-----------------

- Added units to tree output, support writing negative numeric values

0.0.7 (2013-6-23)
-----------------

- Time series data write and read commands, with --follow option


0.0.6 (2013-6-23)
-----------------

- RID lookup and bulk drop commands


0.0.5 (2013-6-21)
-----------------

- Install two command line scripts: exo, exodata


0.0.4 (2013-6-18)
-----------------

- Complete Exosite Data API
- Subset of Exosite RPC API

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

exoline-0.5.0.tar.gz (50.0 kB view hashes)

Uploaded source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page