file-tag-parser

Package

A tag-based filename parser for imaging data.

This library is designed to make parsing of arbitrary filenames attainable and searchable via a pandas-based database. Each filename can then be interpreted and/or manipulated for whatever processing and analysis is desired.

At the moment, the library is solely dedicated to the interpretation of complex filenames using a "tagging system", to streamline their grouping and loading. This will likely be expanded in the future as needed.

Below are the details for how the file tag parser works. This allows the program to easily handle filename structures from different devices and groups, by extracting "tags" from the filename that have meaning in the context of data analysis.

Filename strings and tags

Filename tags are highly useful for grouping datasets for processing and analysis. Each of the tags below will be extracted from the filename if it is contained within braces.

Example filenames and tags

Example 1:

For an image filename with the following format:

8675309_OD_1776_850nm_AVG.tif

and tag string:

{IDnum}_{Eye}_{VidNum}_{Modality}_AVG.tif

The program would store the following tags in the pandas database:

IDNum = "8675309"
Eye = "OD"
VidNum = "1776"
Modality = "850nm"

Example 2:

For an video filename with the following format:

hellothere_OD_1861_split_det_favorite.avi

and tag string:

{IDnum}_OD_{VidNum}_{Modality}_favorite.avi

The program would store the following tags in the pandas database:

IDNum = "hellothere"
VidNum = "1861"
Modality = "split_det"

Example 3: Ignoring tags.

If your filenames have changing numbers that should be grouped together but file-tag-parser doesn't have a tag for them, they can be blank (ignored) fields. For example:

Video 1: hellothere_OD_1861_split_det_vanilla.avi Video 2: hellothere_OD_1861_confocal_chocolate.avi

and tag string:

{IDnum}_OD_{VidNum}_{Modality}_{}.avi

file-tag-parser would store the following tags for use in other analysis steps:

Video 1:

IDNum = "hellothere"
VidNum = "1861"
Modality = "split_det"

Video 2:

IDNum = "hellothere"
VidNum = "1861"
Modality = "confocal"

Example 4: Optional fields

In some cases, for example in query location files, files may take on a form that sometimes includes modifiers and other times does not. Optional fields automatically strip out common "separator" characters, (e.g. -, _, , etc.) Optional fields are delineated in a tag string as follows:

{Fieldname:s?}

So for query location files:

Query Locations 1: hellothere_OD_1861_split_det_vanilla_coords.csv

Query Locations 2: hellothere_OD_1861_split_det_vanilla_subset_coords.csv

and tag string:

{IDnum}_OD_{VidNum}_{Modality}_{}_{QueryLoc:s?}coords.csv

file-tag-parser would store the following tags in the database:

Query Locations 1:

IDNum = "hellothere"
VidNum = "1861"
Modality = "split_det"
QueryLoc = ""

Query Locations 2:

IDNum = "hellothere"
VidNum = "1861"
Modality = "split_det"
QueryLoc = "subset"