Floor-300 diagnosis-set expansion + synonym canonicalization

Date: 2026-08-13

Expands the model’s diagnosis set from 25 to 74 classes (every distinct clinical diagnosis with ≥300 images in dataset v1, per client decision) and fixes a silent 10% training-data loss caused by synonym spellings.

Label set: labels/floor300.txt (74 classes)

The current 25 production classes plus 49 additions, selected from the v1 manifest primary-diagnosis distribution at a 300-image floor. Curation rules:

• Catch-alls excluded even where they clear the floor (“dermatitis uns” 35,468, “allergic rash”, “rash”, “nail dis uns”, …) — they stay in the other-pool.
• Synonym duplicates excluded — each concept keeps one canonical string (see below).
• Subtypes folded into parents (psoriasis vulgaris/guttate → psoriasis, contact dermatitis-irritant → contact dermatitis, infected tinea pedis → tinea pedis).
• “seborrheic keratosis” qualifies only after synonym merging (252 + 180 = 432).

Synonym map (ddtrain/datasets/synonyms.py, dataset.synonym_map: true)

The v1 ETL applied only the legacy replace_dict_yd; the manifest still holds multiple spellings of the same disease, and DermaDetectDataset label matching is exact-string. Result: 38–40k images (10.3% of v1) silently landed in the other-pool instead of their class — “pityriasis versicolor” (7,422) vs the “tinea versicolor” class, “atopic dermatitis” (7,335) vs “eczema uns”, “seborrhea” (3,854) vs “seborrheic dermatitis”, etc. This affected the 25-class production training identically.

SYNONYM_MAP (alias → canonical, ~60 entries; follows the dermatologist-approved legacy DiseaseCorrecter.replace_dict_macc where one exists) is applied to diagnosis and diagnoses_all in DermaDetectDataset.__init__ before the principal/other split, behind a new dataset.synonym_map config flag (default false — existing configs reproduce their exact datasets). Verified on the real manifest: +40,578 images (+17.6%) reach a named class (270,755 direct vs 230,177 exact-match). Tests: tests/test_synonyms.py (5 tests, incl. a no-chained-aliases invariant; full suite 43 passed / 6 skipped).

Runs

• floor300syn-convnextv2b-s17 (devbox, 2× Pascal, DataParallel fp16) — ConvNeXt-V2-Base 224px baseline, configs/floor300_convnextv2_base.yaml. 216,493 principal + 11,394 other train samples, 75 outputs. In progress. (An earlier floor300-convnextv2b-s17 without the synonym map was killed ~1.5 epochs in, superseded.)
• floor300-medsiglip-s17 (configs/floor300_medsiglip.yaml) — the deployable candidate: MedSigLIP-448 + trimmed questionnaire, identical recipe to Round-1 gen2a-medsiglip, only label_file/synonym_map changed. Requires a RunPod H100 (Pascal can’t train 448 ViTs): use scripts/runpod_train.sh (create → provision → train → fetch → destroy), which replaces the never-committed Round-1 Terraform env. Blocked on RUNPOD_API_KEY + HF_TOKEN + fresh AWS creds at time of writing.

Expected effects

• Coverage of clinical cases roughly +16 points vs the 25-class set (46.5% → ~62% of dermatologist diagnosis mentions, before synonym gains).
• Classes near the floor (~15 classes at 300–450 images) will have noisy per-class validation (~30–60 val images); judge on PR curves, not top-1.
• The “other” pool is cleaner (no more same-morphology synonym images teaching “none of the above”), which should sharpen the abstain signal.