clinical-cad 0.1.0

Robust Clinical Association Displacement (CAD) analysis with statistical normalization

clinical-cad

Clinical Association Displacement (CAD) and Weighted Association Error (WAE) analysis for evaluating lexical fairness and demographic bias in clinical text (e.g., radiology reports).

Compare a reference corpus (e.g., ground-truth reports) with model predictions to measure how word associations with demographic groups (M/F) shift, and compute fairness metrics.

---

What it does

CAD (Clinical Association Displacement)

CAD measures how strongly words are associated with male vs. female in the reference corpus, and how those associations change in the prediction corpus. It produces:

• Log-odds ratios and z-scores for each word (variance-normalized for cross-model comparability)
• Displacement = change in association from reference to prediction
• Category labels: female-associated, male-associated, or neutral (based on significance thresholds)
• Sweep plots showing how metrics vary with different displacement thresholds

WAE (Weighted Association Error)

WAE summarizes displacement as a single fairness metric: the weighted average of squared z-scores (displacement). Lower WAE = less bias. It reports:

• Overall WAE and 95% bootstrap confidence intervals
• WAE by subgroup: female-associated words vs. male-associated words
• Three weighting schemes: total (ref+pred counts), ref (reference only), pred (prediction only)

---

Installation

pip install clinical-cad

From source:

git clone https://github.com/yourusername/clinical-cad
cd clinical-cad
pip install -e .

---

Quick start

Minimal run (reference CSV, prediction CSV, output dir):

cad --reference_csv ref.csv --prediction_csv pred.csv

Output goes to ./cad_output by default.

Custom output directory:

cad --reference_csv ref.csv --prediction_csv pred.csv --output_dir ./my_results

Demo run (no data needed; generates toy CSVs and runs full pipeline):

cad --demo

---

Input CSV format

Both CSVs must have:

1. A text column (default: findings for reference, predicted_report for predictions)
2. A demographic label column with values M or F (default: PatientSex; auto-detects patient_sex, sex, etc.)

Example:

PatientSex	findings / predicted_report
M	No acute cardiopulmonary process.
F	Normal heart size. No pleural effusion.

---

Output

All files are written to --output_dir:

• JSON summary: association stats, WAE metrics, bootstrap CIs
• CSV tables: term-level associations, displacements, categories
• Plots: category distribution, WAE bar chart, displacement sweep

---

Parameter reference

Input files

Parameter	Default	Description
--reference_csv	(required)	Path to reference corpus CSV.
--prediction_csv	(required)	Path to prediction CSV.
--output_dir	./cad_output	Directory for all output files.
--reference_text_column	findings	Column name for reference text.
--prediction_text_column	predicted_report	Column name for prediction text.
--label_column	PatientSex	Demographic column (M/F). Auto-detects common names.
--reference_label_column	—	Override label column for reference CSV only.
--prediction_label_column	—	Override label column for prediction CSV only.

CAD & association parameters

Parameter	Default	Description
--min_freq	1	Minimum token frequency to include.
--alpha	0.1	Dirichlet smoothing for log-odds (reduces noise for rare words).
--significance_level	0.0455	P-value for strong association boundary (two-tailed).
--neutral_significance	0.317	P-value for neutral band; higher = more words considered neutral.
--displacement_significance	0.01	P-value for displacement significance; z threshold derived from this.
--displacement_threshold_z	—	Override z threshold directly (ignores displacement_significance if set).
--min_abs_log_odds	0	Minimum |log-odds| to treat as non-neutral.
--top_k	50	Number of top terms in summary tables.

WAE parameters

Parameter	Default	Description
--wae_weight	total	Weight source: total (ref+pred), ref (reference counts), pred (prediction counts).
--bootstrap_samples	1000	Bootstrap samples for WAE confidence intervals.

Tokenization & filtering

Parameter	Default	Description
--stopwords_path	—	File with extra stopwords (one per line). Built-in list used otherwise.
--no_stopword_removal	False	Disable stopword removal.
--include_bigrams	False	Include bigrams in vocabulary.
--vocab_path	—	Restrict vocabulary to words in this file.
--chunksize	100000	Chunk size for streaming large CSVs.

Sweep plot parameters

Parameter	Default	Description
--sweep_disp_start	0	Start z for displacement sweep.
--sweep_disp_stop	3	Stop z for displacement sweep.
--sweep_disp_step	0.1	Step size for displacement sweep.
--sweep_p_start	0.001	Start p-value for sweep.
--sweep_p_stop	1	Stop p-value for sweep.
--sweep_p_step	0.05	Step size for p-value sweep.

Diversity metrics (optional)

Parameter	Default	Description
--diversity_sample_size	2000	Max documents to sample for diversity.
--diversity_truncate_words	150	Truncate docs to this many words.
--diversity_self_bleu_samples	500	Samples for Self-BLEU approximation.
--diversity_random_seed	0	Random seed for sampling.
--diversity_semantic_backend	auto	Backend: auto, sentence_transformers, or tfidf.

Misc

Parameter	Default	Description
--demo	False	Run on built-in demo data (no CSVs needed).

---

Python API

from clinical_cad import parse_args, run_pipeline

args = parse_args()
# Or build args manually, then:
run_pipeline(
    args.reference_csv,
    args.reference_text_column,
    args.prediction_csv,
    args.prediction_text_column,
    args.label_column or "PatientSex",
    args.output_dir,
    args,
)

---

Publishing to PyPI

1. Install: pip install build twine
2. Update pyproject.toml (authors, URLs).
3. Build: python -m build
4. Upload: twine upload dist/* (or --repository testpypi for Test PyPI).

---

License

MIT