spa-ds 2.0.8

Data science toolkit for spectroscopic profiling data analysis.

spa-ds (originally qsi-tk)

Data science toolkit for spectroscopic profiling signals/data.

Installation

pip install spa-ds

Contents

This package is a master library containing various previous packages published by our team.

module	sub-module	description	standalone pypi package	publication
spa.io	spa.io.load	File I/O, Dataset loading		Provides 40+ open datasets. 15+ with publications
	spa.io.aug	Data augmentation, e.g., generative models		Data aug with deep generative models. e.g., " variational autoencoders, generative adversarial networks, autoregressive models, KDE, normalizing flow models, energy-based models, and score-based models. "
	spa.io.pre	Data preprocessing, e.g., window filter, x-binning, baseline removal.		Enhanced data preprocessing with novel window function in Raman spectroscopy: Leveraging feature selection and machine learning for raspberry origin identification [J]. Spectrochimica Acta Part A: Molecular and Biomolecular Spectroscopy. 2024. doi: 10.1016/j.saa.2024.124913
spa.vis		Plotting
spa.cs		compressed sensing	cs1	Adaptive compressed sensing of Raman spectroscopic profiling data for discriminative tasks [J]. Talanta, 2020, doi: 10.1016/j.talanta.2019.120681 Task-adaptive eigenvector-based projection (EBP) transform for compressed sensing: A case study of spectroscopic profiling sensor [J]. Analytical Science Advances. Chemistry Europe, 2021, doi: 10.1002/ansa.202100018 Compressed Sensing library for spectroscopic profiling data [J]. Software Impacts, 2023, doi: 10.1016/j.simpa.2023.100492 Secured telemetry based on time-variant sensing matrix – An empirical study of spectroscopic profiling, Smart Agricultural Technology, Volume 5, 2023, doi: 10.1016/j.atech.2023.100268 Variational Auto-Encoder based Deep Compressed Sensing on Raman Spectroscopy [J]. Smart Agricultural Technology. 2025
spa.fs
	spa.fs.nch_time_series_fs	channel alignment for e-nose; multi-channel e-nose/e-tongue data fs with 1d-laplacian conv kernel		基于电子鼻和一维拉普拉斯卷积核的奶粉基粉产地鉴别,2024,doi: 10.13982/j.mfst.1673-9078.2024.5.0299
	spa.fs.glasso	Structured-fs of Raman data with group lasso		Cheese brand identification with Raman spectroscopy and sparse group LASSO [J], Journal of Food Composition and Analysis, 2025, doi: 10.1016/j.jfca.2025.107371
	spa.fs.mt	Multi-task feature selection for yogurt fermentation analysis		Studying yogurt fermentation dynamics using multi-task feature selection, 2025, 2nd-round review
spa.kernel	spa.kernel.*	Implementation of 31 atom kernel types	ackl	Analytical chemistry kernel library for spectroscopic profiling data, Food Chemistry Advances, Volume 3, 2023, 100342, ISSN 2772-753X, https://doi.org/10.1016/j.focha.2023.100342.
	spa.kernel.mkl	Multi-kernel learning; PSO-MKL, GA-MKL		In progress
spa.dr	spa.dr.metrics	Dimensionality Reduction (DR) quality metrics	pyDRMetrics, wDRMetrics	pyDRMetrics - A Python toolkit for dimensionality reduction quality assessment, Heliyon, Volume 7, Issue 2, 2021, e06199, ISSN 2405-8440, doi: 10.1016/j.heliyon.2021.e06199.
	spa.dr.mf	matrix-factorization based DR	pyMFDR	Matrix Factorization Based Dimensionality Reduction Algorithms - A Comparative Study on Spectroscopic Profiling Data [J], Analytical Chemistry, 2022. doi: 10.1021/acs.analchem.2c01922
spa.cla	spa.cla.metrics	classifiability analysis	pyCLAMs, wCLAMs	A unified classifiability analysis framework based on meta-learner and its application in spectroscopic profiling data [J]. Applied Intelligence, 2021, doi: 10.1007/s10489-021-02810-8 pyCLAMs: An integrated Python toolkit for classifiability analysis [J]. SoftwareX, 2022, doi: 10.1016/j.softx.2022.101007
	spa.cla.ensemble	homo-stacking, hetero-stacking, FSSE	pyNNRW	Spectroscopic Profiling-based Geographic Herb Identification by Neural Network with Random Weights [J]. Spectrochimica Acta Part A: Molecular and Biomolecular Spectroscopy, 2022, doi: 10.1016/j.saa.2022.121348 Geographical origin identification of dendrobium officinale based on NNRW-stacking ensembles. Machine Learning with Applications [J]. 2024. doi: 10.1016/j.mlwa.2024.100594
	spa.cla.kernel	kernel-NNRW
	spa.cla.nnrw	neural networks with random weights
spa.regress		Regression algorithms, e.g., GW-KNNR (Gaussian-weighted K-nearest neighbor regressor).		Quantification of Cow Milk in Adulterated Goat Milk Using Raman Spectroscopy and Machine Learning[J]. Microchemical Journal, 2025, doi: 10.1016/j.microc.2025.114319
spa.pipeline		General data analysis pipelines.		Building an Information Infrastructure of Spectroscopic Profiling Data for Food-Drug Quality and Safety Management [J]. Enterprise Information Systems, 2019, doi: 10.1080/17517575.2019 Machine learning-assisted MALDI-TOF MS toward rapid classification of milk products[J]. Journal of Dairy Science, 2024, doi:10.3168/jds.2024-24886
spa.gui		Web-based apps. e.g., `python -m spa.gui.chaihu` will launch the app for bupleurum origin discrimination.		Rapid Raman Spectroscopy Analysis Assisted with Machine Learning: A Case Study on Radix Bupleuri[J], Journal of the Science of Food and Agriculture, 2024. doi:10.1002/jsfa.14012

刚提的两项设置（我上一条只做了一半） 恢复默认：Service 方法已加，但还没接 API 端点和前端按钮。 语言/画幅等限定下拉：未做，仍是自由文本。

B. 产品/平台层——大块完全没做 Export Presets（抖音/YouTube/B站 多尺寸批量导出）——未做。 Safety & Review / 内容审核——未做。 Model Routing 未接线：Settings 里 ScriptModel/ImageModel/VideoModel/VoiceModel 字段能存，但生成时并没真正按它路由到不同模型（provider 仍读 env/MediaOptions）。

C. 引擎模块——首版已跑通，深度未做 ③ Generation Router：算出并记录路线，但没有真正差异化的生成路径（多关键帧/首帧/图生视频只是标签，未落到不同流程）。 ④ Reference Director：只有 prompt 优先级 + 禁止项文本；没有参考图 embedding / IP-Adapter 传图做人脸强一致。 ⑤ Lineart/专业分镜：结构字段有了，没有真正的线稿渲染 / ControlNet 结构锁定。 ⑥ QC Gate：规则检查，没有真实画面漂移检测（需视觉模型）。 ⑦ Repair Path：映射到重生成策略，没有局部重绘/局部重算。 ⑧ Dependency Graph：能看角色→镜头引用，没有"改角色→自动批量重算受影响镜头"的传播。 ⑩ Experiment Run：会记录，没有 A/B 实验编排 / ReRank 训练集导出。 ⑪ Delivery Package：MP4+BGM+SRT+manifest，缺封面帧、工程包(zip)、审阅态、资产清单打包。 D. 皇冠层——规则引擎已落地，其余未做 Director RAG（影视教材/摄影/剪辑知识检索）——未做。 真实 Video Critic 模型——当前是启发式占位打分，非真实模型。 E. 生成层仍是占位/默认省成本 视频：默认走 FFmpeg 伪运镜；真 text2video（Wan2.2）已接但默认关。 图像一致性：只靠 prompt+seed，无真正 img2img/人脸锁定（默认）。 mflux 本地图像：因磁盘不足未启用；Ollama 本地 LLM 非默认（现用 DeepSeek）。 Cost Estimator：给的是调用次数，不是真实金额（无单价表）。 一句话 11 个引擎模块 + 皇冠规则引擎都有可运行首版；没做的主要是：产品化平台层（鉴权/权限/支付/存储/发布集成/审核）+ 各引擎的"真模型/真渲染"深度 + Director RAG + 真实 Critic 训练。