This is a pre-production deployment of Warehouse, however changes made here WILL affect the production instance of PyPI.
Latest Version Dependencies status unknown Test status unknown Test coverage unknown
Project Description
See https://github.com/openworm/tracker-commons
# Python implementation of Tracker Commons

This project contains source code in Python that implements the WCON data format for tracking data.

*When the Python specification and the behavior of the Scala implementation differ, and it is not obvious which one is correct, the behavior of the Scala implementation should be presumed to be authoritative.*

### Installation

`wcon` is [registered with the Python Package Index](https://pypi.python.org/pypi/wcon/) so just enter this command from any shell:

```
pip install wcon
```

Any problems? Visit the more detailed [installation guide](INSTALL.md).

### Usage

```
import wcon

# Open a worm file, convert it to canonical form, and save it
# (actually it's automatically converted to canon before
# saving, but here we do so explicitly)
w = wcon.WCONWorms.load_from_file('tests/minimax.wcon')
canon_w = w.to_canon
canon_w.save_to_file('test.wcon', pretty_print=True)

# From a string literal:
from io import StringIO
w2 = WCONWorms.load(StringIO('{"units":{"t":"s","x":"mm","y":"mm"}, '
'"data":[]}'))

# WCONWorms.load_from_file accepts any valid WCON, but .save_to_file
# output is always "canonical" WCON, which makes specific choices about
# how to arrange and format the WCON file. This way the functional
# equality of any two WCON files can be tested by this:

w1 = WCONWorms.load_from_file('file1.wcon')
w2 = WCONWorms.load_from_file('file2.wcon')

assert(w1 == w2)

# or:

w1.save_to_file('file1.wcon')
w2.save_to_file('file2.wcon')

import filecmp
assert(filecmp.cmp('file1.wcon', file2.wcon'))

w3 = w1 + w2 # Merge the two. An exception is raised if the data clashes
```

### Formal API

- Class `WCONWorms`
- methods
- `load_from_file`
- [class method]
- parameters:
- `JSON_path`, a `str`, the path of the file
- `save_to_file`
- parameters:
- `JSON_path`, a `str`, the path of the file
- `compress_file`, a boolean, whether to compress the file
- `pretty_print`, a boolean, whether to render the output on multiple lines
- `to_canon`
- [property]
- returns: a copy of this object but in canonical form
- `__add__`
- [use `+`]
- Merges WCONWorms objects together. If the worm IDs or time periods are disjoint, or if the data agrees, this method works. If not, an exception is thrown.
- `__eq__`
- [use `==`]
- Return a boolean indicating whether two WCONWorms objects are the same, after conversion of all quantities to canonical units. Compares both data and metadata.
- attributes
- `units`: dict
- May be empty, but is never None since 'units' is required
to be specified.
- `metadata`: dict
- If 'metadata' was not specified, metadata is None.
- The values in this dict might be nested into further dicts or other
data types.
- `data`: Pandas DataFrame or None
- If 'data' was not specified, data is None.
- All the worms are merged into one DataFrame
- Expensive and memory-intensive for objects containing many worms, so try to avoid using this. Use `data_as_odict` instead.
- [Note: `files`, if present in the input, is not persisted unless the `.load` factory method is used.]
- `num_worms`: int
- `worm_ids`: list
- `data_as_odict`: OrderedDict of pandas DataFrames, keyed by worm ID
- This is the native representation of the data in this object, and thus the fastest to load. Try to use this instead of `data`.
- Class `MeasurementUnit`
- Note: this class does not need to be used publicly, but it can be if desired.
- consequently it can be ommitted from a public API
- methods
- `create`
- [class method]
- Factory method
- parameter: `unit_string`, a `str`, the unit expression (e.g. `"mm"` or `"cm/s"` or `"C"`)
- returns: an instance of this class
- `to_canon`
- transforms `v` from original units to canonical units
- parameter: `v` (a `float`)
- returns: float
- `from_canon`
- the inverse of `to_canon`
- attributes
- `unit_string`: str
- The original string (e.g. `"m/s^2"`)
- `canonical_unit_string`: str
- The canonical form for all units within the original string (e.g. `"mm/s^2"`)

### Custom WCON objects

Any top-level key other than the basic:

- files
- units
- metadata
- data

...are ignored. It is convenient, but not required, to follow the convention of beginning custom fields with the prefix `"@"`. Handling custom objects requires subclassing `WCONWorms`.


### WCON parser: proof of concept

Thanks to the Python libraries `json` and `jsonschema`, it is relatively trivial to parse and validate a WCON file. Here's an example of how one might accomplish this, without even using the `wcon` package:

import json, jsonschema

# The WCON schema
with open("wcon/wcon_schema.json", "r") as wcon_schema_file:
schema = json.loads(wcon_schema_file.read())

# Our example WCON file
JSON_path = '../../tests/minimax.wcon'
with open(JSON_path, 'r') as infile:
serialized_data = infile.read()

# Load the whole JSON file into a nested dict.
w = json.loads(serialized_data)

# Validate the raw file against the WCON schema
jsonschema.validate(w, schema)

With the above code we end up with a nested dictionary `w` containing everything that was serialized in the `minimax.wcon` file.

### The `wcon` Python package

Using this `wcon` Python package, something similar can be accomplished:

import wcon

w = wcon.WCONWorms.load_from_file('../../tests/minimax.wcon')

Here, instead of being a nested dictionary, `w` is a `WCONWorms` object that is more powerful. Here are some of the additional things that can be accomplished with the `WCONWorms` object:

- The WCON file is validated not just against the WCON schema, but also to ensure units are valid, that every data key has a corresponding unit, and that every data segment has "aspects" of the same length. (e.g. if a skeleton at time `1.3` has 45 `x`-coordinates, it should also have 45 `y`-coordinates. This condition is not expressible in a JSON schema but it is validated programatically by the WCONWorms initializer.
- Units are expressed as `MeasurementUnit` objects, which can be compared with other such objects, to verify that "mm" and "millimetres" refer to the same units, for instance. (see the below section for more details)
- WCONWorms objects can have their data be converted into canonical units, and then saved again.
- WCONWorms objects can be loaded from multiple files and combined together, via the `"files"` object.
- Worm data recorded in multiple "tracks", or elements, in the `"data"` object, can have such tracks merged.
- Worm data can be extracted in a Pandas DataFrame format for easier downstream processing, since the dimensions of the data have been placed into one two-dimensional array, rather than in a nested array.
- WCONWorms can be subclassed by labs implementing "special features", in two places: ("type 1") top-level objects starting with `"@"` or ("type 2") objects within individual `"data"` array items starting with `"@"`.


### MeasurementUnit

The WCON format requires a `"units"` object, where you specify in what units your quantities are being measured. `WCONParser` represents these units internally as `MeasurementUnit` objects. With `MeasurementUnit`, you can convert from any supported unit expression to the canonical one:

>>> MeasurementUnit.create('m')
MeasurementUnit, original form: 'm' canonical form: 'mm'
>>> u = MeasurementUnit.create('m')
>>> u.to_canon(1)
1000.0
>>> u.from_canon(100)
0.1
>>> u = MeasurementUnit.create('F')
>>> u.to_canon(72)
22.222222222222221
>>> u = MeasurementUnit.create('m/min')
>>> u.canonical_unit_string
'mm/s'
>>> u.to_canon(5)
83.33333333333334
>>> u = MeasurementUnit.create('m^2')
>>> u.to_canon(1)
1000000.0

You can also check the equality of various unit expressions. For example, all of these expressions will evaluate to `True`:

MeasurementUnit.create('mm') == MeasurementUnit.create('millimetre')
MeasurementUnit.create('Mm') == MeasurementUnit.create('megametre')
MeasurementUnit.create('mm') != MeasurementUnit.create('Mm')
Release History

Release History

1.1.0

This version

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

1.0.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.3.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.3.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.2.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.2.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.6

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.5

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.4

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.3

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.2

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

Download Files

Download Files

TODO: Brief introduction on what you do with files - including link to relevant help section.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
wcon-1.1.0-py3-none-any.whl (34.7 kB) Copy SHA256 Checksum SHA256 py3 Wheel Apr 17, 2016
wcon-1.1.0.tar.gz (29.1 kB) Copy SHA256 Checksum SHA256 Source Apr 17, 2016

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS HPE HPE Development Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting