Commit graph

1 commit

Author SHA1 Message Date
2453fb48ba feat(dataset): add row-level exclusion curation for labeled volumes
Introduce src/exclusions.py (rules: total TV > 1e6 mm3, single target
< 100 mm3, non-brain NTUH keys) and scripts/make_tv_slices.py, a
first-stage curation script that classifies every labeled row right
after the source fetch and writes TV-slice QA figures to
data/tv_slices/{inclusion,exclusion}/.

05_build_splits.py re-applies the same rules after split assignment:
kept rows retain their split, excluded subjects still count in the
subject-level shuffle boundaries, and the audit trail is written to
data/manifests/excluded.jsonl. Label stats run in a
ProcessPoolExecutor (32 workers).

Move proc_row into src/common.py and add labeled_source_rows() so the
curation stage and the split builder share one processed-row view.

Document the 143-row exclusion audit and the interim 10-epoch
benchmark results in BENCHMARKING.md; update the README data layout
and script tables.
2026-10-05 16:53:56 +08:00