pylv200

Pull images out of a running Olympus/Evident cellSens experiment on the LV200.

cellSens does not finalise its image container until a run ends, but the pixel data is already on disk in the _<experiment>_ folder as .ets tile files, and a small .vsi written when the run starts names the channels. pylv200 reads both and writes an ImageJ hyperstack covering everything acquired so far, so you do not have to end a multi-day recording to look at it.

It is strictly read-only: it never writes, renames or locks anything inside the experiment folder, so it is safe to run mid-acquisition.

When a run does end, cellSens writes the finished .vsi, and Bio-Formats converts that to OME-TIFF. pylv200 is for the run that has not finished — or has not survived. --ome-tiff writes the same kind of file that conversion produces, so a stack pulled mid-run and one converted afterwards are interchangeable downstream.

Install

pip install pylv200
pip install pylv200[compressed]   # only if your tiles turn out compressed

On a microscope PC, install nothing. A standalone Windows build carries Python, numpy, tifffile and the image codecs inside a single 46 MB file. It is built locally rather than committed, so it is not in a fresh clone. Copy the one file across, run it, delete it when done — nothing is registered, no PATH is changed, and no administrator rights are needed. Rebuild it with:

pyinstaller --onefile --console --name pylv200 --collect-all imagecodecs src/pylv200/__main__.py

Use

pylv200 list left-lv200                          # what is on that microscope
pylv200 plan left-lv200:ois351A --last-n 24      # what a pull would fetch
pylv200 sync left-lv200:ois351A -o D:\pulled     # fetch what is new

command	does
setup	run on the microscope: find the data, open one read-only door
probe <source>	reachable? roots resolve? anything being written?
login <source>	hand a credential to Windows; this package never sees it
logout <source>	forget a source's stored credential
rescan <source>	find an output folder that moved, without a lab visit
sources	the microscopes this machine knows about
status [<target>]	what is reachable, what a watcher is doing, what has been written
scans <target>	which timepoints a run holds, and when each was acquired
list [<source>]	experiments on a source, --live to mark the running ones
find <pattern>	which experiment is that?
plan <target>	what a pull would fetch, and what is already held
pull <target>	fetch once, writing every selected timepoint
sync <target>	fetch only what is new, using the ledger
preview <target>	a few tiles per position, to see which one you want
watch <target>	poll on an interval and collect what is new
manifest <folder>	summarise a previous pull
preset	show or save a named recipe
gui	the desktop app: choose a position by looking at it

A target is source:experiment. A bare experiment name works when only one microscope is configured; with two it says so rather than guessing, because guessing puts one experiment's channel names on another's pixels. A path is taken as itself:

pylv200 pull "D:\Experiments\_MyRun_" --dry-run

--json goes before the subcommand and puts a machine-readable payload on stdout, with everything else on stderr:

pylv200 --json list left-lv200 | jq '.experiments[].name'

Exit codes: 0 success, 1 nothing to do or a fault, 2 usage or unreachable. probe and sync are meant for a scheduled task, so "there was nothing new" is 1 and distinguishable from both success and failure.

pull --dry-run first, and check the reported geometry — channel count, timepoint count, frame size — against the experiment you actually set up.

A run in progress has no chunk table. cellSens writes the table that locates every tile only when it closes the file, so a live acquisition has zeroes where the table pointer and count belong. pylv200 detects this and rebuilds the index by walking the fixed tile stride instead, reporting recovered by walking the tile stride when it does. That path needs --channels N to be correct, because without a table there is nothing recording which tile belongs to which channel — tiles are laid down channel-fastest, N per timepoint. Get --channels wrong and the frames will be dealt into the wrong stacks.

Pulling from a run that is still going, use --safe-copy. The chunk table is written at the end of the file and rewritten as the file grows, so it can move while the tool is reading. --safe-copy parses a stable snapshot instead. The tool checks whether the newest part is still growing and reminds you if it is.

Windows will not refresh a file's size or timestamp in a directory listing while cellSens holds it open, so a part that looks days old in Explorer may be the one being written to right now. Never use the listed timestamp to decide which part is live.

flag (on pull / sync)	what it does
--last-n N	the N most recent timepoints
--from-t T / --to-t T	an explicit window, numbered as the file numbers it
--channels N	override the channel count read from the file header
--pixel-size	microns per pixel, for spatial calibration
--interval	seconds between timepoints; derived from the run's own timestamps otherwise
--channel-names	override the names read from the .vsi, and pin them for later pulls
--composite	overlay channels in ImageJ's default colours instead of grayscale
--split-channels	one TIFF per channel instead of one composite
--ome-tiff	a lossless LZW OME-TIFF, matching a Bio-Formats conversion
--include-last	include the final tile (dropped by default, it may be mid-write)
--axes tx,ty,z,c,t	override the axis assignment if the guess is wrong
--ndims N	force the chunk-table dimension count
--dtype uint16	force pixel type
--safe-copy	copy each .ets to temp before parsing
--mirror DIR	keep a local byte copy and fetch only what the run has added
--preset NAME	apply a saved recipe; the command line still wins
--no-pin	do not remember --channel-names, and forget any already pinned

Setting up a microscope, once

Run pylv200 setup on the LV200 itself. It walks every fixed drive and recognises cellSens data by the first four bytes of a file, never by a folder name — the output folder is a user setting and the two microscopes disagree about it — then reports what is stopping a share from working:

pylv200 setup                    # look, change nothing
pylv200 setup --create-share     # print the plan
pylv200 setup --create-share --yes

It creates nothing without --yes, writes the undo commands to lv200-setup-undo.txt before the first change, and prints a source block to paste on the analysis machine. Then, once, from there:

pylv200 login left-lv200
pylv200 probe left-lv200

The account it creates is an ordinary user, and it refuses to build the shares if that account is ever in Administrators: a broad read-only share is safe only because the account cannot read what an administrator could. Its password is typed into Windows' own prompt, so it never reaches pylv200 — nothing here can leak what it never receives.

If cellSens is later pointed somewhere else on the same machine, pylv200 rescan left-lv200 finds it again without a visit.

Pulling over the network

Tiles are fetched by seeking to each offset rather than reading the file end to end, so restricting the time axis restricts what crosses the wire. On a nine-day run at half-hour intervals, --last-n 24 pulls the last day: 50 MB instead of 875 MB. The reported frame interval is unaffected, because it is derived from the whole file's timestamps before the selection is applied.

plan reads only headers and tile indexes — a few kilobytes — so it is the cheap way to see what a transfer would cost before committing to it:

pylv200 plan left-lv200:ois351A --last-n 24 -o D:\pulled
  ois351A_20260814
  4 position(s), 417 timepoint(s), 4 channel(s)
  channels        C0 BioLuminescence  C1 BF  C2 GFP  C3 RFP
  selection       t=393..416
  already held    393 timepoint(s) in D:\pulled
  to fetch        24 timepoint(s), 50.3 MB

Look before you pull

Eleven positions of nearly a gigabyte each, named ois351A and ois3518, tell you nothing about which one holds the field of view you care about.

pylv200 preview left-lv200:ois351A -o D:\thumbs

It reads a handful of tiles — kilobytes, not gigabytes — and writes one thumbnail per position and channel plus a contact sheet: positions down, channels across, each column naming its channel and whether that name was read from the .vsi or pinned by hand.

The display range is computed once per channel across every position on the sheet, not per image. A per-image stretch makes an empty position look exactly as bright as a full one, which is the precise question the sheet exists to answer.

Photon-starved channels are averaged over a few frames, because a single bioluminescence frame is mostly shot noise and the cells appear only as it averages. The contrast is a percentile stretch, not a full-range map: a full-range map renders a luminescence frame black, which reads as "no data" rather than "few photons". --stretch LO HI changes it.

Previews need matplotlib: pip install pylv200[preview]. Nothing else in pylv200 does, and importing it never pulls in a display toolkit.

Watching a run instead of remembering to pull it

pylv200 watch left-lv200:ois351A -o D:\pulled

It polls, collects what is new, and holds it until there is enough to be worth writing. Over a nine-day acquisition that is the difference between remembering to pull and simply having the data — and if cellSens clears its scratch when a run ends, between capturing a crashed run and losing it.

flag	what it does
-i N	seconds between polls. Defaults to half the run's own frame interval
--batch-frames N	hold until N new timepoints have accumulated (default 4)
--batch-after 7d	write anyway once the oldest waiting frame is that old
--once	poll once and stop
--flush	write whatever is held right now, then stop
--status	what the watcher for this output has done, and whether it is still running

--batch-frames is not cosmetic. A composite hyperstack is rewritten whole when a timepoint is added, so polling a large experiment every half hour without it rewrites gigabytes 48 times a day. A held timepoint leaves nothing behind — no file and no ledger entry — so holding can never lose one.

--batch-after is measured from when a timepoint was acquired, not from when the watcher started, so restarting the watcher does not restart the wait.

A dropped share, a rebooted microscope and a sleeping laptop all look the same and are all recovered from, with backoff. An expired credential is told apart from an unreachable host, because the fix for it is pylv200 login and not the network.

Every poll is recorded in pylv200-watch.log beside the output, including the ones that did nothing — a watcher that silently died is worse than no watcher, because you believe you have the data. pylv200 watch <target> -o <dir> --status reads it back and exits non-zero if the log has gone quiet.

If you would rather not leave a terminal open, run it from a Scheduled Task:

$action  = New-ScheduledTaskAction -Execute "pylv200" `
    -Argument 'watch left-lv200:ois351A -o D:\pulled --once --batch-frames 4'
$trigger = New-ScheduledTaskTrigger -Once -At (Get-Date) `
    -RepetitionInterval (New-TimeSpan -Minutes 15)
Register-ScheduledTask -TaskName "pylv200 ois351A" -Action $action `
    -Trigger $trigger -RunLevel Limited

--once exits 0 when it wrote, 1 when there was nothing new, and 2 when the source was unreachable, so the task's own history is a second record of whether it is working.

Adding to a stack instead of rewriting it

A watcher that rewrote its output on every poll would move the whole stack 48 times a day to gain one timepoint. It does not: new planes are written into the existing file in place, and only the new tiles cross the wire.

The trade is that an appended frame cannot be un-appended, where a rewrite simply replaces the file. --no-append opts out, and --include-last --no-append is the conservative pair. Appending falls back to a rewrite on its own whenever the file is not the earlier part of this same stack - a different geometry, a different frame interval, different channel names - so the reason to ask is caution, not correctness.

It applies to the default composite hyperstack. --split-channels and --ome-tiff still rewrite.

The window

pylv200 gui                       # or pylv200-gui, or python -m pylv200.gui
pylv200 gui left-lv200:ois351A    # opening straight onto one run

The app exists for one job the command line does badly: choosing which field of view you want. Eleven positions called ois351A and ois3518 are eleven names that say nothing, so the app draws a grid of thumbnails instead and you click the pictures. Reading them costs a handful of tiles per field, not a pull.

It has no pull of its own. Every button turns the panel into a pylv200 command, hands it to the same parser the terminal uses, and calls the same function — so the app and the command line cannot write different files, and the confirmation dialog shows you the exact command before it runs one. Copy that command and it is a Scheduled Task.

The panel is PyIncucyte's, subject for subject: an output folder, the channels this run's own .vsi names, a time window, a layout, the switches, the watch cadence, a mirror, and the calibration. Everything on it is one field of the recipe, so the whole panel saves as a preset and reloads.

Two consequences worth knowing. Anything the command line cannot express is not on the panel: the time window is counted in timepoints, never in hours, because --last-n and --from-t are what exist. And Stop takes effect between fields of view — a field already being written is finished, so the ledger and the disk never disagree about what you have.

Tk is imported only by this command. pylv200 list on a headless microscope needs no display, and a test asserts it.

Keeping a local copy that grows with the run

--mirror DIR keeps a byte copy of each .ets in DIR. Tile data is appended at a fixed stride, so a second run fetches only what the acquisition has added since the first:

pylv200 sync left-lv200:ois7041 --mirror "D:\mirror" -o "D:\pulled"

  mirroring to D:\mirror (232.8 MB to fetch)
    ois7041.ets  +232.8 MB of 232.8 MB
                                        ... and an hour later:
  mirroring to D:\mirror (16.8 MB to fetch)
    ois7041.ets  +16.8 MB of 249.6 MB

The tiles are then read from the copy, so a re-run costs nothing over the wire. What is left behind matters more than the speed: during acquisition the tiles are the only copy of the run that exists, because cellSens writes nothing else until it ends.

It is not a backup. It is a copy of a live scratch file and inherits whatever the source has, including corruption. When a copy looks wrong, the mirror can re-read sampled stretches from the source and compare.

sync into the same folder twice does no work the second time:

  nothing new: all 111 selected timepoint(s) are already in D:\pulled

A ledger beside the output records which timepoints it holds, and how many tiles the source had for each when it was written. So a frame the acquisition has since added to is fetched again, one it has not touched is not, and re-running with different channel names or a different layout is never mistaken for work already done. pull ignores the ledger and writes everything selected; deleting an output file is enough to make it come back.

The sidecars and the timestamps are still read from the source, where they live: a copy's modification time is when it was copied, and using it would make the frame interval whatever the copy took.

What is read, what is derived, and what is not known

Channel identity is read, not guessed. The .vsi written beside the tiles when the run starts carries the channel names in acquisition order, along with the experiment name, the camera, the objective and each channel's filter-wheel positions. On a real run: BioLuminescence, BF, GFP, RFP, Hamamatsu ImagEM X2 9100-23B, LUCPLFLN 40x.

Three more things come out of the .ets itself:

	where it comes from
channel count	a length-prefixed [X, Y, C, Z, T] array in the sub-header tail. On a live part the T entry is stale, written once at creation, but C is correct
timepoint count	the tiles actually present
frame interval	the run's start (from the .vsi) to its last write, divided by the timepoints written

One thing is not known: microns per pixel. The .vsi's XResolution is its thumbnail's screen resolution, not the microscope's, and no value in the metadata could be tied to a spatial calibration with any confidence. So pylv200 reports the objective and leaves the calibration empty rather than inventing one — a wrong micron-per-pixel silently rescales every measurement downstream. Pass --pixel-size if you know it.

The pixel statistics are still computed and printed, as a cross-check on the names rather than a source of them: bioluminescence barely correlates frame to frame, brightfield is the brightest and near-identical, and nothing in the pixels distinguishes GFP from RFP. Every value in the manifest records where it came from, so six months later a name read off the .vsi and one the pixels were asked to guess at are still tellable apart.

Pulling only some of it

pylv200 sync left-lv200:ois351A -o D:\pulled --positions ois351A ois3518
pylv200 sync left-lv200:ois351A -o D:\pulled --channels 0 2

Both drop tiles before they cross the wire — a tile is fetched by seeking to its offset, so what is not asked for is never transferred. Pulling one channel of a four-channel run moves roughly a quarter of the bytes.

Channel names follow the selection by index, so --channels 0 2 on BioLuminescence, BF, GFP, RFP writes a two-channel stack labelled BioLuminescence, GFP — not the first two names. The manifest records the original indexes alongside them.

--channels used to mean "how many channels the header should report". That is now --channel-count. Asking for a channel the file does not have says so and names the other flag rather than writing a stack with the wrong shape.

From Python

import pylv200

with pylv200.connect() as lv200:
    result = lv200.fetch(output="./run-01", last_n=2)

connect() takes a source name from pylv200 setup, a folder, or nothing at all when there is only one saved source. What comes back writes the same bytes as pylv200 sync - the command line calls the same function. lv200.pull(...) is the same method under the verb everything else in this package uses.

plan = lv200.plan("ois351A", last_n=24)        # what a pull would fetch
lv200.preview("ois351A").save("./thumbs")      # a few tiles per position
watcher = lv200.watch("ois351A", output="./run-01", batch_after="7d")
print(watcher.pending_frames, watcher.hold_description)
watcher.stop(wait=True, flush=True)            # write what it was holding

A watcher left to itself pulls through the client, so a poll that wrote something hands the whole result to on_result - not just the paths:

def outline(poll):                             # poll.wrote is the paths
    for image in poll.result.files:            # poll.result is a PullResult
        print(image.path, image.axes, image.complete,
              [(c.index, c.name, c.source) for c in image.channel_refs])

lv200.watch("ois351A", output="./run-01", on_result=outline)

A recipe is one object, and it round-trips:

options = pylv200.ExportOptions(start_from="-48h", layout="ome")
options.save("nightly.json")
print(options.cli_command(target="left-lv200:ois351A"))
# pylv200 sync left-lv200:ois351A --start-from -48h --ome-tiff

Channels are written grayscale

These are grayscale detector channels, so the stack is written in ImageJ's grayscale display mode and no lookup tables are assigned. An ImageJ composite would otherwise colour them red/green/blue/grey purely by index, implying a relationship between channels that does not exist. --composite opts back in.

Output

One composite ImageJ hyperstack per position, (T, Z, C, Y, X), opening in Fiji as a multichannel hyperstack you can scrub. Past ImageJ's 4 GB hyperstack ceiling it writes OME-TIFF instead, which Fiji opens through Bio-Formats; --ome-tiff chooses that outright. --split-channels gives one file per channel.

Beside the stacks it writes pylv200-manifest.json and pylv200-index.csv recording what was written, how each derived value was known, and which source files it came from. pylv200 manifest <folder> reads them back.

Each file entry is written to be read on its own, without joining anything back to the header:

{
  "path": "D:\\pulled\\Per2_Torin1_ois7041.tif",
  "axes": "TZCYX",
  "channels": [{"index": 1, "name": "BioLuminescence", "source": "vsi",
                "source_index": 0},
               {"index": 2, "name": "GFP", "source": "vsi",
                "source_index": 2}],
  "frame_count": 393, "missing": 0,
  "complete": false, "interval_s": 1380.0, "pixel_size_um": null
}

index counts from one, the way ImageJ and Fiji count, and describes this file — a pull that caught a partly acquired newest frame holds fewer channels than the run has, and an index taken from the run would name a plane that is not there. source_index is the run's own numbering, which is what a --channels selection is written in. source says how the name was known: vsi, oex, override or inferred.

complete is false while cellSens is still writing — the acquisition has no chunk table until it closes — so a downstream step knows to come back for this stack rather than treating it as final. Alongside it in the header, include_last states the policy that produced the frame count: the newest tile may be half written, so it is dropped unless --include-last asks for it, and 393 frames of a 394-frame run is a decision rather than a fault.

As a library:

from pylv200 import LV200Client, ExportOptions

client = LV200Client(r"D:\Experiments")
for experiment in client.experiments(live=True):
    print(experiment.name, [p.name for p in experiment.positions])

plan = client.plan("MyRun", ExportOptions(start_from="-24f"))
print(plan.timepoints, plan.bytes_to_read)

from pylv200 import parse_ets, assign_axes, build_stacks

meta  = parse_ets("D:/Experiments/_MyRun_/frame_t.ets")
axes  = assign_axes(meta, channels=2)
built = build_stacks(meta, axes)
built["data"]                             # (T, Z, C, Y, X) composite
built["stacks"]                           # {channel: view into that array}

How it finds the data

.ets field offsets shift between cellSens versions, so nothing is hardcoded. The parser scans the volume header for a pointer that lands on the ETS magic, then scores every candidate chunk-table position against every dimension count from 2 to 8, keeping the reading where tile byte counts agree with each other, divide evenly into whole bytes per pixel, and where the tile coordinates form a dense regular grid. A wrong guess fails all three tests.

For a run still in progress the header over-declares how many tiles exist, so records are read past the declared count and stop at the first run of invalid ones. Incomplete timepoints are dropped rather than half-filled.

Datasets are found the same way — by the first four bytes of a file, never by a folder name. A directory qualifies on the first valid header rather than on every file in it, which is what makes list usable over a share.

Coming from PyIncucyte

pylv200 is the sister package to PyIncucyte, and the two are deliberately driven the same way. Where the idea is shared the name is shared; where the instrument differs, so does the noun.

PyIncucyte	pylv200
pyincucyte vessels	pylv200 list (vessels also works)
pyincucyte download -v 38 -o out	pylv200 sync <source>:<run> -o out (download also works)
pyincucyte scans	pylv200 scans - same name (timepoints also works)
--wells A1-D6	--positions ois351A ois3518 - fields of view, not plate wells
--channels phase,green	--channels 0 2 - channel indexes, named from the .vsi
--layout time_channel_stack	--layout composite - the default; a run here is always a time series
IncucyteClient.from_saved()	pylv200.connect()
client.fetch(vessel=38, ...)	client.fetch(positions=["ois351A"], ...)
pyincucyte scan-now	nothing - pylv200 never commands the microscope
--unmix, --calibrate, --green-lut	nothing - no device coefficients, and a display LUT would corrupt what downstream analysis measures

Typing one of the absent commands says why it will not exist rather than printing a usage block.

Status

Alpha. The parser is validated against synthetic .ets files covering complete and truncated acquisitions, and against five real acquisitions up to 875 MB, but has not been confirmed against every cellSens version in the wild. Run pull --dry-run and sanity-check the geometry before trusting the pixels. Reports of files it misreads are welcome.

Tests

pip install -e .[test]
pytest

Licence

MIT.