Skip to main content

felix' python data analysis suite

Project description

fepydas

fepydas ("felix' python data analysis suite") is a Python 3 library for analysing and fitting optical-spectroscopy and photophysics data: photoluminescence (PL), PL excitation (PLE), Raman, time-resolved PL (TCSPC decays), external quantum efficiency (EQE) and spatially-resolved spectrum maps.

It gives you a small, unit-aware data model plus ready-made readers, fitters and plotters, so a typical analysis is just: load a file → manipulate the dataset → fit → plot/export.

from fepydas.constructors.datareaders.Custom import ASCII_Spectrum
from fepydas.workers.Fit import GaussianFit

spectrum = ASCII_Spectrum("measurement.txt")   # -> a Spectrum
spectrum.cutAxis(500, 600)                      # keep 500-600 nm
spectrum.normalize()                            # scale peak to 1

fit = GaussianFit()
fit.initializeAutoFromSpectrum(spectrum)        # guess starting parameters
fit.fitSpectrum(spectrum)                        # run the fit

spectrum.toPlot().save("spectrum.png")          # save a plot
spectrum.export("spectrum.txt")                 # save tab-separated data

Table of contents

  1. Installation
  2. Core concepts
  3. Loading data
  4. Working with spectra and series
  5. Units, axes and calibration
  6. Fitting
  7. Time-resolved decays (IRF convolution)
  8. Spectrum maps
  9. PLE measurements
  10. Decomposition and clustering
  11. Plotting
  12. Exporting and saving
  13. Batch processing whole directories
  14. Worked examples
  15. API map
  16. Requirements

Installation

pip install fepydas

Python 3 is required (3.9+ recommended, since numpy >= 2.0 is a dependency). All other dependencies (scipy, matplotlib, lmfit, scikit-learn, pyamg, sympy, pathos, tqdm, ptufile) are installed automatically. See Requirements.

To work from a checkout instead:

git clone https://git.tu-berlin.de/nippert/fepydas
cd fepydas
pip install -e .

Core concepts

fepydas is organised in three layers. You usually only touch the top two.

Layer Package What it does
Constructors fepydas.constructors Read files into datasets, and build plots.
Workers fepydas.workers Fit, write ASCII, batch-load directories.
Data model fepydas.datatypes The unit-aware containers everything is built on.

The data model

  • Data / Data1D / Data2D / Data3D — a numpy array (.values) plus a DataType (.datatype) and optional .errors. The dimensionality matches the structure: a single spectrum is Data1D, a stack of spectra is Data2D, a spatial cube is Data3D.
  • DataType — a quantity name + a Unit (e.g. "Wavelength" in nm). Predefined ones live in fepydas.datatypes (WL_nm, E_eV, COUNTS, TIME_ns, TEMP_K, ...).
  • Unit — wraps a sympy unit so values can be compared and converted (including the special wavelength ↔ energy case, with the Jacobian applied to intensities).

Datasets

A Dataset pairs an x-axis with data. The hierarchy specialises this:

Dataset
├── Spectrum                       one spectrum (axis + Data1D)
└── SpectrumSeries                 many spectra sharing an axis, indexed by a "key" axis
    ├── BinnedSpectrumSeries       a series rebinned along its key axis
    ├── JoinSpectrumSeries         several separate spectra stacked into a series
    ├── MergeSpectraSeries         several series concatenated
    └── SpectrumSeriesWithCalibration
        ├── PLE                    emission spectra keyed by excitation wavelength
        └── BaseSpectrumMap
            ├── SpectrumMap        dense rectangular map (spectrum at every pixel)
            └── SparseSpectrumMap  map where only some pixels were measured

Spectrum
└── SpectrumWithIRF                a decay histogram bundled with its instrument response
    └── SpectraWithCommonIRF       several decays sharing one IRF

Map                                a single scalar value per map pixel (e.g. a fit result)

Most dataset methods mutate in place and return None (e.g. cutAxis, normalize, convertToEnergy). Methods that derive a new object return it (e.g. integrate, average, collapse, toPlot).

The typical workflow

ds = SomeReader(path)            # constructors.datareaders.* -> a dataset
ds.cutAxis(...); ds.normalize()  # in-place dataset operations
fit = SomeFit(); fit.initializeAutoFromSpectrum(ds); fit.fitSpectrum(ds)
ds.toPlot().save("plot.png")     # constructors.Plots
ds.export("data.txt")            # workers.ASCIIWriter

Loading data

All readers are plain functions that return a dataset. Pick the one that matches your file.

Generic / in-memory (fepydas.constructors.datareaders.Generic)

from fepydas.constructors.datareaders.Generic import (
    GenericXY, GenericXMultiY, ASCII, Binary,
)
import numpy

# Build a Spectrum from arrays you already have. Units are abbreviation strings.
x = numpy.linspace(400, 700, 300)
y = numpy.exp(-((x - 550) ** 2) / 50)
spectrum = GenericXY(x, "Wavelength", "nm", y, "Counts", "")

# Build a SpectrumSeries from a shared axis and a 2D stack of y-data.
series = GenericXMultiY(x, "Wavelength", "nm", y2d, "Counts", "",
                        keys, "Temperature", "K")

# Read a raw numeric table (thin numpy.genfromtxt wrapper).
table = ASCII("data.txt", delimiter="\t", skip_header=0)

# Reload anything previously saved with .saveBinary()
obj = Binary("dataset.bin")

TCSPC histograms (Generic + Proprietary)

from fepydas.constructors.datareaders.Generic import Histogram, HistogramWithIRF

decay = Histogram("decay.phu")                      # .phu/.thi/.sdt/.ptu -> Spectrum
pair  = HistogramWithIRF("decay.sdt", "irf.sdt")    # -> SpectrumWithIRF

Histogram auto-detects the format from the extension: .phu (PicoHarp), .thi (TimeHarp), .sdt (Becker & Hickl), .ptu (PicoQuant TTTR). For large .ptu files the reader is chunked/low-RAM and takes options (forwarded through **kwargs): channel, bin_width_s, all_channels, number_of_bins, use_memmap, show_progress, progress_callback, ... See the Histogram/PTUHistogram docstrings.

Lab-specific ASCII formats (fepydas.constructors.datareaders.Custom)

Function Returns Use for
ASCII_Spectrum(file) Spectrum a single spectrum (wavelength/counts)
ASCII_TimeSeries(file) SpectrumSeries spectra over time
ASCII_RamanSeries(folder) SpectrumSeries one Raman spectrum per file in a folder
ASCII_Gelbes(file) / ASCII_TempSeries_Gelbes(file) / ASCII_TimeSeries_Gelbes(file) Spectrum / SpectrumSeries the "Gelbes" lab format
ASCII_SSTRPL(file) SpectrumSeries steady-state-transient PL
ASCII_EQE(file) SpectrumSeries EQE current series (background-subtracted)
ASCII_PLE(file, cal=None, resp=None) PLE PLE map with optional calibration/response
Mapscan(file) / Mapscan2(file) SpectrumMap binary spectrum maps (float64 / uint16)

LabVIEW exports (fepydas.constructors.datareaders.FromLabview)

from fepydas.constructors.datareaders.FromLabview import Automatic

ds = Automatic("measurement.plm")   # inspects the header and returns the right type

Automatic detects the measurement type from the file header and returns a Spectrum, SpectrumSeries, PLE, SparseSpectrumMap or Map (dark-frame correction and tracked power are applied automatically when present).


Working with spectra and series

These operations are available on Spectrum and (where it makes sense) SpectrumSeries. They mutate the dataset in place unless noted.

# --- cropping ---
spectrum.cutAxis(500, 600)          # keep an axis value range
spectrum.cutAxisByIdx(50, 3000)     # keep an axis index range
series.cutKeys(10, 300)             # keep a key range (series only)

# --- intensity operations ---
spectrum.normalize()                # scale max to 1
series.normalize(individual=True)   # scale each spectrum to its own max
series.subtractBaseline()           # remove a flat baseline (no dark spectrum available)
spectrum.antiBloom(45000, width=2)  # mask CCD blooming above a threshold
spectrum.divideBySpectrum(reference)

# --- reductions (these RETURN new datasets) ---
integral = series.integrate()       # integrated intensity vs key  -> Spectrum
avg      = series.average()         # mean spectrum                -> Spectrum
total    = series.collapse()        # summed spectrum              -> Spectrum
peakpos  = series.maximum()         # peak axis position vs key    -> Spectrum

# --- selecting / combining ---
series.filterByIntegral(0.5)        # drop weak spectra (< 50% of the strongest integral)
hdr = series.highDynamicRange()     # merge differently-exposed spectra -> Spectrum

identifyPeaks(threshold=10, width=10) returns index ranges around detected peaks (used by the calibration helpers).


Units, axes and calibration

Predefined data types

Import from fepydas.datatypes: WL_nm, RS_cm (Raman shift), E_eV, COUNTS, NUMBER, TIME_s, TIME_ns, TIME_ps, TEMP_K, ANGLE_au, I_mA, P_mW, POS_um, POS_mm. Build your own with generateDataType("Power", "mW") (the unit string is parsed by unitFromAbbrev).

Wavelength ↔ energy

spectrum.convertToEnergy("eV")   # converts the axis AND applies the Jacobian to intensities

This works on Spectrum, SpectrumSeries and the maps. At the Data1D level you can also call axis.transformTo(E_eV) for any compatible unit.

Vacuum correction

spectrum.applyVacuumCorrection(temperature=20, pressure=101325, humidity=0, co2=610)

Converts air wavelengths to vacuum wavelengths using the refractive index of moist air.

Spectral calibration

If a calibration spectrum (e.g. a lamp with known lines) is attached (PLE, the map types, or anything that is a SpectrumSeriesWithCalibration):

ds.calibrate(references=[None, 435.8335, 404.6565])  # known positions for detected peaks

Standalone, you can fit a calibration curve from measured vs reference line positions:

from fepydas.workers.Fit import Calibration

cal = Calibration(measured_positions, nist_positions)   # fits on construction; saves a plot
calibrated_axis = cal.apply(other_axis.values)

Fitting

Every fit model is a Fit subclass. The pattern is always: create → initialise parameters → fit → use the result.

from fepydas.workers.Fit import GaussianFit

fit = GaussianFit()
fit.initializeAutoFromSpectrum(spectrum)   # guess bg, I, x0, fwhm from the data
# (optional) constrain parameters before fitting:
fit.parameters["x0"].min = 419
fit.parameters["x0"].max = 421
fit.fitSpectrum(spectrum)                   # or fit.fit(x, y)

print(fit.result.params)                    # lmfit result
model_curve = fit.evaluate(spectrum.axis.values)

Available models

Class Function fitted Key parameters
GaussianFit, LimitedGaussianFit Gaussian peak bg, I, x0, fwhm
LorentzianFit, SpectralLine Lorentzian peak bg, I, x0, fwhm
LinearFit, CalibrationFit a*x + b a, b
QuadraticFit, QuadraticCalibrationFit a*x² + b*x + c a, b, c
AutomaticCalibration quadratic axis calibration (fits peaks → references on construction)
ExponentialFit single exponential decay bg, I, x0, tau, rise
BiExponentialFit, TriExponentialFit 2-/3-component decay I_n, tau_n, ...
BiExponentialTailFit 2-component tail (no rise) bg, I_1, tau_1, I_2, tau_2

You can set parameters manually instead of initializeAuto*:

from fepydas.workers.Fit import Fit
from lmfit import Parameters

def model(x, a=1, b=0):
    return a * x + b

fit = Fit(model)
p = Parameters()
p.add("a", value=1, min=0)
p.add("b", value=0)
fit.initializeParameters(p)
fit.fitSpectrum(spectrum)

Fitting a whole series in parallel

fit = GaussianFit()
fit.initializeAutoFromSpectrum(series.average())
series.fit(fit)                               # fits every spectrum (uses a process pool)

Windows note: parallel fitting forks a process pool. On Windows, guard your script with if __name__ == "__main__": so child processes don't re-run the whole module.

Saving fit reports

fit.saveReport("fit.txt")          # parameter table + statistics
fit.saveLMfitReport("lmfit.txt")   # the raw lmfit fit_report

FitSimple — fitting with native lmfit models

For workflows built directly on lmfit.Model, FitSimple offers parameter guessing, optional parallel fitting and dynamic bound propagation:

from fepydas.workers.Fit import FitSimple
from lmfit.models import GaussianModel

fs = FitSimple(GaussianModel(), spectrum_or_series)
fs.initguess()                         # or fs.setinitparams(params)
fs.fit(multiprocessing=True)           # y_err=..., bounds_tolerance=..., cores=... all supported

Time-resolved decays (IRF convolution)

For TCSPC lifetime fitting, load the decay together with its instrument response function and fit a model convolved with the IRF:

from fepydas.constructors.datareaders.Generic import HistogramWithIRF
from fepydas.constructors.Plots import SpectrumWithIRFPlot
from fepydas.workers.Fit import ExponentialFit

s = HistogramWithIRF("Signal.sdt", "InstrumentResponseFunction.sdt")
s.normalize()

fit = ExponentialFit()
fit.initializeAutoFromSpectrum(s)
fit.parameters["rise"].value = 0.01
fit.parameters["rise"].vary = False
fit.convolutedFit(s.axis.values, s.data.values, s.IRF.values)

plot = SpectrumWithIRFPlot(s)
plot.addLine(fit.evaluateConvolution(s.axis.values, s.IRF.values), label="Convolution Fit")
plot.setYLog(); plot.legend()
plot.save("ConvolutionFit.png")

Spectrum maps

Maps store a spectrum at every spatial pixel. SpectrumMap is a dense rectangular grid; SparseSpectrumMap (e.g. from a masked LabVIEW scan) stores only measured pixels.

from fepydas.constructors.datareaders.Custom import Mapscan
from fepydas.workers.Fit import GaussianFit
from fepydas.datatypes.Data import DataType

if __name__ == "__main__":           # required for parallel fitting on Windows
    m = Mapscan("Example.map")
    m.cutAxis(290, 360)              # restrict the spectral window

    m.integrate().toPlot().save("integrated.png")   # integrated intensity image
    m.maximum().toPlot().save("peakpos.png")        # peak-position image
    m.average().toPlot().save("avg_spectrum.png")   # mean spectrum

    # fit every pixel and turn a fit parameter into a map
    fit = GaussianFit()
    fit.initializeAutoFromSpectrum(m.average())
    m.fit(fit)
    posMap = m.fitParameterToMap(fit, "x0", DataType("Position", "nm"))
    posMap.toPlot(zRange=[330, 350]).save("fitted_position.png")

Maps also support clustering-based segmentation:

clusters = m.fitMixture(n=2, nmf=3)               # spectral clustering (optionally NMF-reduced)
m.getClusterMapIdx(clusters).toPlot().save("clusters.png")
m.getClusterIntegratedSpectra(clusters).toPlot().save("cluster_spectra.png")

integrate(), maximum() and fitParameterToMap(...) return Map objects (one value per pixel), which plot as false-color images via toPlot(log=..., zRange=...).


PLE measurements

PLE is a series of emission spectra keyed by excitation wavelength, with extra correction steps. A full PLE workflow (from the PLE example):

from fepydas.constructors.datareaders.Custom import ASCII_PLE

ple = ASCII_PLE("PLE.txt", "Calibration.cal", "IntensityRef.dat")
ple.toPlot().save("01_raw.png")

ple.calibrate([None, 435.8335, 404.6565])     # calibrate the detection axis
ple.applyVacuumCorrection()
ple.subtractBaseline()
ple.applyLampResponseCorrection()              # divide by the lamp response
ple.calibrateLamp()                            # calibrate the excitation axis from the lamp line
ple.export("CalibratedPLE.txt")

ple.integrate().toPlot().save("integrated.png")
ple.normalize(individual=True)
ple.toPlot().save("normalized.png")            # false-color contour (log by default)

Decomposition and clustering

SpectrumSeries (and maps) can be decomposed into spectral components using scikit-learn (NMF / PCA / LDA) — useful for separating overlapping signals or removing a background.

# find / inspect components
decomp, components = series.getDominatingComponents(algo="LDA", n=3)
components.toPlot().save("components.png")
strongest = series.getDominatingComponent(algo="PCA")     # -> Spectrum

# remove a background learned from a reference, then apply it to the signal
nmf = reference.removeDominatingComponents(n=10)   # fit + subtract, returns the decomposition
signal.removeComponents(nmf)                        # subtract the same components

# project each series onto a fitted decomposition
weights = series.getProjection(decomp)              # -> SpectrumSeries of component weights

Plotting

Datasets expose toPlot() (and sometimes toScatterPlot()), which return a plot object. Call .save(path) to write an image, .show() to open a window, or styling methods first.

plot = series.toPlot()           # SpectrumSeriesPlot (overlaid, colored by key)
plot.setYLog()                   # log y-axis
plot.setXRange(400, 700)         # zoom x
plot.addLine(extra.data.values, "extra")   # overlay another curve
plot.legend()
plot.save("series.png", dpi=300, bbox_inches="tight")   # kwargs go to matplotlib savefig
Dataset toPlot() returns Notes
Spectrum SpectrumPlot (line) / SpectrumScatterPlot toScatterPlot() for points
SpectrumSeries SpectrumSeriesPlot toPlot(polar=True, cmap=..., norm=...)
Map / SpectrumMap.integrate() etc. ContourMapPlot toPlot(log=..., zRange=...)
PLE ContourPlot toPlot(log=True)

You can also construct plot classes directly from fepydas.constructors.Plots, e.g. MultiSpectrumScatterPlot([a, b], ["A", "B"]) to overlay spectra with different axes, or SpectrumScatterPlotWithFit(spectrum, fit) to show data and fit together. Common styling methods on every plot: setXRange, setYRange, setXLog, setYLog, legend, show, save.


Exporting and saving

spectrum.export("spectrum.txt")     # tab-separated text (X + Y columns, with names/units)
series.export("series.txt")         # shared X plus one column per key
map.export("map.txt")               # one column per y position

spectrum.saveBinary("spectrum.bin") # pickle the whole object
# reload with Binary("spectrum.bin")

For fit output alongside the data, use the writers in fepydas.workers.ASCIIWriter:

from fepydas.workers.ASCIIWriter import SpectrumWithFitWriter
SpectrumWithFitWriter("plot.txt", spectrum, fit, withIRF=True)   # writes data + fit (+ IRF)

Batch processing whole directories

DirectoryLoader applies one reader to every file in a folder and lets you run the same operation on all of them:

from fepydas.workers.DirectoryLoader import DirectoryLoader
from fepydas.constructors.datareaders.Custom import ASCII_Gelbes
from fepydas.datatypes.Dataset import JoinSpectrumSeries

loader = DirectoryLoader(ASCII_Gelbes)
loader.loadFromDirectory("data/", lambda name: name.endswith(".dat"))
loader.callFunction("cutAxisByIdx", 50, 3000)      # call .cutAxisByIdx(50, 3000) on each

series = JoinSpectrumSeries(loader.getList(), interpol=True, names=loader.getKeys())

Rebinning a series

from fepydas.constructors.Binner import LinearBinner   # also LogBinner, InverseBinner
from fepydas.datatypes.Dataset import BinnedSpectrumSeries

binned = BinnedSpectrumSeries(series, LinearBinner(), num_bins=200)

Worked examples

The examples/ directory contains complete, runnable scripts (each with its own data files). Run a single one from its folder, e.g. cd examples/PLE && python3 PLE.py, or run them all with cd examples && python3 RunExamples.py.

Example Demonstrates
PLE/ Full PLE pipeline: calibration, vacuum correction, lamp-response correction, contour plots
Binning/ Temperature series, Arrhenius transform, BinnedSpectrumSeries, integration
EQE/ EQE analysis with custom fit functions and SpectrumScatterPlotWithFit
HighDynamicRange/ Joining differently-exposed spectra into one HDR spectrum
Mapscan/ & NewMapscan/ Spectrum maps: integration, per-pixel fitting, clustering
Histogram_IRF_Convolution_Fitting/ TCSPC decay fitting with IRF convolution
PQ_Histograms/ Reading PicoHarp/TimeHarp histograms
SSTRPL/ Component removal/decomposition and exponential fitting
RamanAngleDecomposition/ Raman angle series, LDA decomposition, polar projection plots

API map

Import Contents
fepydas.datatypes DataType, generateDataType, predefined types (WL_nm, E_eV, ...)
fepydas.datatypes.Data Data1D/2D/3D, Unit, DataType, Transformation, VacuumCorrection, Response
fepydas.datatypes.Dataset Spectrum, SpectrumSeries, PLE, SpectrumMap, SparseSpectrumMap, Map, SpectrumWithIRF, ...
fepydas.constructors.datareaders.Generic GenericXY, ASCII, Histogram, HistogramWithIRF, Binary, ...
fepydas.constructors.datareaders.Custom ASCII_Spectrum, ASCII_PLE, ASCII_EQE, Mapscan, ...
fepydas.constructors.datareaders.Proprietary PicoHarpHistogram, TimeHarpHistogram, BeckerHicklHistogram, PTUHistogram
fepydas.constructors.datareaders.FromLabview Automatic
fepydas.constructors.Plots SpectrumPlot, SpectrumSeriesPlot, ContourPlot, ContourMapPlot, ...
fepydas.constructors.Binner LinearBinner, LogBinner, InverseBinner
fepydas.workers.Fit Fit, GaussianFit, ExponentialFit, Calibration, FitSimple, ...
fepydas.workers.ASCIIWriter MultiSpectrumWriter, SpectrumWithFitWriter
fepydas.workers.DirectoryLoader DirectoryLoader

Every class and function carries a detailed docstring (visible as a tooltip in most editors) describing its parameters, options and return value.


Requirements

  • Python 3 (3.9+ recommended)
  • numpy ≥ 2.0, scipy, matplotlib, lmfit, scikit-learn, pyamg, sympy ≥ 1.13.3, pathos, tqdm, ptufile

These are installed automatically by pip install fepydas.


fepydas is developed at TU Berlin. Bug reports and contributions: https://git.tu-berlin.de/nippert/fepydas.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

fepydas-0.0.85-py3-none-any.whl (88.2 kB view details)

Uploaded Python 3

File details

Details for the file fepydas-0.0.85-py3-none-any.whl.

File metadata

  • Download URL: fepydas-0.0.85-py3-none-any.whl
  • Upload date:
  • Size: 88.2 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.7.17

File hashes

Hashes for fepydas-0.0.85-py3-none-any.whl
Algorithm Hash digest
SHA256 46a4b4067e607d5f9b96409da687f094412920303955d8a12c5ba8731233303a
MD5 dacff51f9fc88be939747d21d2a9e3a3
BLAKE2b-256 b1e072e0c034a5cecb3c3a0c96a6804d508198b34ccc7306317b69ff85a68696

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page