Furen Xiao
e7721760f2
- Updated `05_monai_run_iterative.py` to include legacy mode and improved configuration options for training. - Introduced `UNetBAligned` class in `monai_common.py` for deep supervision and instance normalization. - Modified `train_transform` and `make_dataloader` in `monai_common.py` to support B-aligned data protocol with Z-score normalization and augmentation. - Added new loss functions `nn_dc_and_ce` and `b_dc_and_ce` in `losses.py` for improved training performance. - Enhanced `Unet3D` architecture in `unet3d.py` to support leaky ReLU and deep supervision. - Created new augmentation functions in `augment_b.py` for Pipeline B's training, including rotation, scaling, and noise. - Implemented new training scripts `run_b_100ep_stages.sh` and `run_c_bloss_100ep_after_b.sh` for structured training and evaluation. - Updated dataset handling in `dataset.py` to accommodate new augmentation and normalization strategies.
231 lines
No EOL
15 KiB
Markdown
231 lines
No EOL
15 KiB
Markdown
# Registration Pipeline Benchmark — CT → T1c (M6) — Affine
|
||
|
||
Companion to [REGISTRATION_BENCHMARK_RIGID.md](REGISTRATION_BENCHMARK_RIGID.md)
|
||
and [REGISTRATION_BENCHMARK_DEFORMABLE.md](REGISTRATION_BENCHMARK_DEFORMABLE.md):
|
||
the **affine** (12-DOF) variant of the same benchmark. The rigid benchmark
|
||
asked "which toolkit aligns the head best with 6 DOF?"; this one asks "how
|
||
much does the full 12-DOF affine buy on top of rigid — and do the toolkits'
|
||
affine workflows still rank the same way?" — on the same 10 M6 CT→T1c test
|
||
cases, the same unified contract, and the same metrics.
|
||
|
||
Run date: 2026-10-03, 13:50–14:40 local (single run, 6 workers, 24 threads,
|
||
2 workers per GPU; ~50 min wall, dominated by `impact`'s ~6 min phase).
|
||
10 M6 test cases, 3× NVIDIA A100 80GB PCIe, 96-core host, conda env
|
||
`longitudinal-313`.
|
||
|
||
## Unified contract
|
||
|
||
Identical to the rigid benchmark: the returned transform `T` must satisfy
|
||
`sitk.Resample(ct, t1c, T, ...)` = CT on the T1c grid, i.e. **T maps T1c →
|
||
CT coordinates**. fixed = T1c, moving = CT, full volumes (no head box, no
|
||
mask, no crop, no pad), each toolkit's native automatic transform
|
||
initialization, CT clipped to [0, 80] HU only for the pipelines where the
|
||
rigid A/B showed clipping helps (niftyreg, synthmorph).
|
||
|
||
One change vs the rigid benchmark: every pipeline estimates a **full affine
|
||
(12-DOF)** transform instead of a rigid (6-DOF) one — each toolkit's native
|
||
affine workflow (or, for the antspyx rows, a stock linear-only preset).
|
||
|
||
## The nine pipelines (affine configurations)
|
||
|
||
| pipeline | affine configuration | metric |
|
||
|---|---|---|
|
||
| **antspyx** | antspyx stock `Affine` preset (the toolkit's native affine): Affine[0.25], 2100×1200×1200×10 iters, shrink 6×4×2×1, smoothing 3×2×1×0, center-of-mass auto init | Mattes MI (32 bins, 20 % regular sampling) |
|
||
| **ants_trsaa** | antspyx stock `TRSAA` preset: Translation[1] → Rigid[1] → Similarity[1] (each 2000×2000×0 iters, 4×2×1, 2×1×0) → Affine[1] × 2 (each 40×20×0, 4×2×1, 2×1×0), COM init — the full TRSA cascade | Mattes MI (32 bins), all stages |
|
||
| **ants_reproa** | `antsRegistrationSyNRepro[a]`: Rigid[0.1] + Affine[0.1] (each 1000×500×250×100, 8×4×2×1, 3×2×1×0vox), COM init — the linear-only half of the Repro workflow | GC (1/1/Regular/0.25), both stages |
|
||
| **ants_quickreproa** | `antsRegistrationSyNQuickRepro[a]`: Rigid[0.1] + Affine[0.1] (each 1000×500×250×0 — no full-res level —, 8×4×2×1, 3×2×1×0vox), COM init | GC (1/1/Regular/0.25), both stages |
|
||
| **elastix** | AffineTransform (12-DOF, the rigid benchmark's map with Euler→Affine), ASGD 2000 iters × 3 resolutions, CenterOfGravity auto init | Mattes MI (32 bins), 5000 samples/level |
|
||
| **impact** | the ImpactReg `MR_CT_TS` reference map with **AffineTransform** in place of EulerTransform (same MIND + TotalSegmentator static features, 4 scales 400/300/200/200 iters, 2000 samples, same CoG-init / 0.1 sample-gate adaptations) | IMPACT semantic (L1 + Dice) + Mattes MI |
|
||
| **niftyreg** | reg_aladin **full affine** (no `-rigOnly`), 3-level pyramid, 15 TLS iters/level | NCC block matching (init) + TLS |
|
||
| **simpleitk** | two-stage (the toolkit's canonical affine workflow): Euler3D stage (2000 iters/level, pyramid [4,2,1], identity start) → AffineTransform stage (same budget) initialized to reproduce the rigid stage exactly — a single-stage affine from identity collapses its scale on these pairs | Mattes MI (32 bins), both stages |
|
||
| **synthmorph** | `affine` matrix model (the toolkit's native 12-DOF deep model) | anatomy-aware contrast-invariant network (pretrained) |
|
||
|
||
The `ants_trsaa` / `ants_reproa` / `ants_quickreproa` rows are the linear-only
|
||
(`[a]`) antspyx presets — the same stage configurations the
|
||
`antsRegistrationSyNRepro*.sh` / `antsRegistrationSyNQuickRepro*.sh`
|
||
workflows use for their Rigid + Affine head.
|
||
|
||
## Test cases
|
||
|
||
Same 10 as the rigid / deformable benchmarks (first 10 M6 labelled cases of
|
||
`m6.discover()`, all with a GTV label):
|
||
`m6_22GD5VL2_20230724`, `m6_22IRRKU6_20240115`, `m6_23XIMO4M_20230222`,
|
||
`m6_24V62QQM_20230223`, `m6_24V62QQM_20231017`, `m6_24V62QQM_20240820`,
|
||
`m6_26TYQI3C_20221027`, `m6_2BBN5XA4_20250411`, `m6_2C4TKI6H_20240506`,
|
||
`m6_2CDNN7BH_20240102`.
|
||
|
||
(`m6_22GD5VL2_20230724` has an incomplete CT head — a known data issue,
|
||
not a pipeline failure; it is scored as ct_incomplete in every pipeline.)
|
||
|
||
## Metrics (identical for every pipeline)
|
||
|
||
- **Mattes MI** (nats) between the warped CT and the T1c over the MR head
|
||
(largest connected component of the T1c mask) — achieved alignment, higher
|
||
= better.
|
||
- **IMPACT loss** (vboussot/ImpactLoss; pretrained TotalSegmentator
|
||
`TS/M730` deep features — weighted L1 over the two deepest layers, full
|
||
volumes; the same scorer as the other benchmarks) between the warped CT
|
||
and the T1c — semantic-anatomy similarity, lower = better.
|
||
- **Registration wall time** (seconds, excludes I/O + scoring).
|
||
|
||
Per-case JSONs additionally retain the production skull-ring QA (gap +
|
||
concentricity, and the acceptance verdict) for reference.
|
||
|
||
### Result
|
||
|
||
Alignment is scored by the achieved **Mattes MI** (nats, warped-CT ~ T1c
|
||
over the MR head; higher = better) **and** the **ImpactLoss** semantic score
|
||
(warped-CT ~ T1c over the full volumes; lower = better), on the same
|
||
unclipped scoring volumes as the other benchmarks, so all rows are
|
||
comparable on the same scales and against the rigid / deformable numbers.
|
||
`niftyreg` and `synthmorph` clip the CT (the moving image) to [0, 80] HU
|
||
*before registration* (per the rigid benchmark's clipping A/B); the others
|
||
register the CT as-is. All nine transforms are affine (12-DOF).
|
||
|
||
All nine pipelines registered 9/9 cases (the 10th is the known
|
||
incomplete-CT data issue, unscored in every pipeline). **Bold** = best for
|
||
the column (MI higher is better, ImpactLoss lower is better).
|
||
|
||
| pipeline | toolkit | registered | s/reg (med) | ImpactLoss (med/mean) | MI (med/mean, nats) |
|
||
|---|---|---|---|---|---|
|
||
| **antspyx** | ANTsPy 0.6.3 (`Affine` preset) | 9 | 12.6 | 0.156 / 0.155 | **0.660 / 0.660** |
|
||
| **ants_trsaa** | ANTsPy 0.6.3 (`TRSAA` preset) | 9 | 30.4 | 0.156 / 0.155 | 0.658 / 0.659 |
|
||
| **ants_reproa** | ANTsPy 0.6.3 (`Repro[a]`) | 9 | 63.1 | 0.160 / 0.159 | 0.525 / 0.514 |
|
||
| **ants_quickreproa** | ANTsPy 0.6.3 (`QuickRepro[a]`) | 9 | 28.4 | 0.161 / 0.159 | 0.523 / 0.514 |
|
||
| **elastix** | ITK-Elastix 0.25.4 | 9 | 12.3 | 0.156 / 0.156 | 0.651 / 0.645 |
|
||
| **impact** | Docker `vboussot/elastix_impact` (Elastix 5.2.0 + IMPACT, affine MIND+TS reference map) | 9 | 215.5 | 0.159 / 0.158 | 0.623 / 0.567 |
|
||
| **niftyreg** | NiftyReg 2.1.1 (reg_aladin full affine, Docker + CUDA) | 9 | 19.8 | 0.167 / 0.170 | 0.278 / 0.330 |
|
||
| **simpleitk** | SimpleITK 2.5.6 | 9 | 23.3 | 0.156 / 0.156 | 0.654 / 0.641 |
|
||
| **synthmorph** | Docker `freesurfer/synthmorph` (TensorFlow, pretrained, `affine` model) | 9 | 27.1 | 0.167 / 0.171 | 0.282 / 0.320 |
|
||
|
||
Per-case MI (nats, unclipped score; **bold** = best for the case):
|
||
|
||
| case | antspyx | ants_trsaa | ants_reproa | ants_quickreproa | elastix | impact | niftyreg | simpleitk | synthmorph |
|
||
|---|---|---|---|---|---|---|---|---|---|
|
||
| m6_22GD5VL2_20230724 | incomplete-CT | incomplete-CT | incomplete-CT | incomplete-CT | incomplete-CT | incomplete-CT | incomplete-CT | incomplete-CT | incomplete-CT |
|
||
| m6_22IRRKU6_20240115 | **0.655** | 0.647 | 0.547 | 0.546 | 0.638 | 0.391 | 0.278 | 0.654 | 0.282 |
|
||
| m6_23XIMO4M_20230222 | **0.661** | 0.660 | 0.519 | 0.521 | 0.656 | 0.657 | 0.611 | 0.660 | 0.520 |
|
||
| m6_24V62QQM_20230223 | **0.710** | 0.708 | 0.568 | 0.569 | 0.684 | 0.708 | 0.254 | **0.710** | 0.252 |
|
||
| m6_24V62QQM_20231017 | 0.628 | **0.635** | 0.425 | 0.423 | 0.622 | 0.623 | 0.473 | 0.619 | 0.479 |
|
||
| m6_24V62QQM_20240820 | 0.670 | **0.673** | 0.459 | 0.464 | 0.670 | 0.659 | 0.121 | 0.545 | 0.122 |
|
||
| m6_26TYQI3C_20221027 | **0.719** | 0.717 | 0.525 | 0.523 | 0.675 | 0.676 | 0.593 | 0.699 | 0.570 |
|
||
| m6_2BBN5XA4_20250411 | **0.590** | 0.588 | 0.505 | 0.507 | 0.568 | 0.360 | 0.069 | **0.590** | 0.071 |
|
||
| m6_2C4TKI6H_20240506 | **0.646** | 0.645 | 0.546 | 0.542 | 0.636 | 0.451 | 0.241 | 0.635 | 0.256 |
|
||
| m6_2CDNN7BH_20240102 | **0.660** | 0.658 | 0.535 | 0.534 | 0.651 | 0.575 | 0.327 | 0.656 | 0.328 |
|
||
|
||
Per-case ImpactLoss (semantic, unclipped score; **bold** = best (lowest)
|
||
for the case):
|
||
|
||
| case | antspyx | ants_trsaa | ants_reproa | ants_quickreproa | elastix | impact | niftyreg | simpleitk | synthmorph |
|
||
|---|---|---|---|---|---|---|---|---|---|
|
||
| m6_22GD5VL2_20230724 | incomplete-CT | incomplete-CT | incomplete-CT | incomplete-CT | incomplete-CT | incomplete-CT | incomplete-CT | incomplete-CT | incomplete-CT |
|
||
| m6_22IRRKU6_20240115 | 0.1514 | 0.1516 | 0.1527 | 0.1528 | **0.1509** | 0.1607 | 0.1667 | 0.1513 | 0.1666 |
|
||
| m6_23XIMO4M_20230222 | 0.1565 | 0.1564 | 0.1598 | 0.1599 | 0.1565 | 0.1564 | 0.1585 | **0.1563** | 0.1616 |
|
||
| m6_24V62QQM_20230223 | **0.1466** | 0.1468 | 0.1527 | 0.1529 | 0.1470 | 0.1468 | 0.1670 | **0.1466** | 0.1672 |
|
||
| m6_24V62QQM_20231017 | 0.1519 | **0.1517** | 0.1599 | 0.1599 | 0.1527 | 0.1533 | 0.1587 | 0.1520 | 0.1581 |
|
||
| m6_24V62QQM_20240820 | 0.1560 | **0.1559** | 0.1618 | 0.1617 | 0.1564 | 0.1570 | 0.1840 | 0.1599 | 0.1843 |
|
||
| m6_26TYQI3C_20221027 | **0.1563** | 0.1570 | 0.1604 | 0.1615 | 0.1615 | 0.1657 | 0.1621 | 0.1585 | 0.1624 |
|
||
| m6_2BBN5XA4_20250411 | 0.1617 | 0.1617 | 0.1622 | 0.1622 | **0.1593** | 0.1662 | 0.1845 | 0.1617 | 0.1849 |
|
||
| m6_2C4TKI6H_20240506 | **0.1559** | 0.1561 | 0.1612 | 0.1613 | 0.1561 | 0.1594 | 0.1759 | **0.1559** | 0.1766 |
|
||
| m6_2CDNN7BH_20240102 | 0.1591 | 0.1590 | 0.1609 | 0.1617 | **0.1587** | **0.1587** | 0.1740 | **0.1587** | 0.1741 |
|
||
|
||
### Interpretation
|
||
|
||
- **The four intensity tools form a statistical tie, and affine barely
|
||
moves them.** `antspyx` (0.660), `ants_trsaa` (0.658), `simpleitk`
|
||
(0.654) and `elastix` (0.651) all sit within 0.009 nats of each other on
|
||
the median (ImpactLoss medians 0.156), and per-case MI bests split
|
||
antspyx 5/9, trsaa 2/9, simpleitk 2/9 (ties). Against the rigid
|
||
benchmark's per-pipeline medians, affine buys only
|
||
`antspyx` 0.652 → 0.660 (+0.008), `simpleitk` 0.647 → 0.654 (+0.007),
|
||
`elastix` 0.646 → 0.651 (+0.005) — the M6 head is essentially rigid
|
||
between sessions, so the extra 6 DOF mostly absorb inter-session
|
||
anisotropic scaling (≤ a few %), not misalignment.
|
||
- **`ants_trsaa` is the full TRSA cascade for free quality — and it is not
|
||
free.** Translation → Rigid → Similarity → Affine → Affine (5 linear
|
||
stages) reaches 0.658, i.e. the same as the single-stage `Affine` preset
|
||
(0.660) at 2.4× the wall time (30.4 s vs 12.6 s): the cascaded 6-DOF and
|
||
9-DOF stages just re-solve what the one 12-DOF stage finds.
|
||
- **`ants_quickreproa` ≈ `ants_reproa` quality at 45 % of the time**
|
||
(0.523 vs 0.525 MI; 28.4 s vs 63.1 s): dropping the full-resolution rigid
|
||
level costs nothing. But both are clearly the weakest of the *top-tier
|
||
eligible* antspyx presets — the GC-based Rigid+Affine head (the linear
|
||
half of the Repro T1w-T1w workflow) locks a cross-modality-weak pose, the
|
||
same story as the Repro family in the deformable benchmark (0.530–0.549
|
||
MI there).
|
||
- **`impact`'s affine run degrades vs its own rigid.** Median 0.623
|
||
(mean 0.567) vs 0.652 for rigid: the semantic 12-DOF landscape has more
|
||
local minima, and the single CenterOfGravity start falls into two of them
|
||
(`m6_22IRRKU6` 0.391, `m6_2BBN5XA4` 0.360 — both 0.645 / 0.572 in the
|
||
rigid run). It also roughly doubles in wall time (215.5 s vs 111.2 s)
|
||
for the extra 6 DOF.
|
||
- **`niftyreg` full affine does not fix its gross-pose failures** (0.278 vs
|
||
0.292 rigid; the two repositioned cases still at 0.069 / 0.121): reg_
|
||
aladin's block-matching + TLS is a local optimizer either way, and the
|
||
output affine is still flipped by the orientation gate on 8/9 cases.
|
||
- **`synthmorph`'s `affine` model confirms the rigid-benchmark
|
||
investigation** (0.282 vs 0.277; within ±0.03 per case): the linear part
|
||
of its output is a near-pure rotation — the extra 6 DOF are essentially
|
||
unused, and the pretrained matrix head's gross-pose estimation weakness
|
||
stands for 12 DOF as for 6.
|
||
- **The deformable stage is still worth it.** The best affine median here
|
||
is 0.660 (`antspyx`); the best deformable in the companion benchmark is
|
||
0.753 (`ants_rigid_synonly`; 0.723 for the stock `SyN` preset) — the
|
||
deformable stage adds +0.06 to +0.09 MI on top of the best affine, which
|
||
is fine-tissue alignment, not gross pose.
|
||
- **Production skull-ring QA (reference only, kept in the per-case JSONs):**
|
||
acceptance 4/9 `antspyx` / `ants_reproa` / `ants_quickreproa` /
|
||
`elastix` / `simpleitk`, 3/9 `ants_trsaa`, 2/9 `impact`, 1/9 `niftyreg` /
|
||
`synthmorph`. Concentricity: `ants_reproa` tightest mid-gap (5.6 mm),
|
||
`niftyreg` / `impact` / `synthmorph` worst on the rotational term
|
||
(23.5–25.3 mm std) — the same weak correlation with perceived alignment
|
||
noted in the other benchmarks, so it is reported for reference only.
|
||
|
||
## Transform conventions (affine-specific)
|
||
|
||
- **antspyx linear-only presets write one merged affine per consecutive
|
||
linear run.** The `TRSAA` cascade (5 linear stages) comes back as a single
|
||
*binary* `0GenericAffine.mat` that SimpleITK cannot parse; the pipeline
|
||
recovers it by point-probing the transform (exact for affine maps,
|
||
verified to NCC 1.0 against ANTs' own `warpedmovout` on the pilot cases).
|
||
`Repro[a]` / `QuickRepro[a]` (Rigid + Affine) merge the same way.
|
||
- **`niftyreg`'s `-aff` file** is documented as `Affine * Reference =
|
||
Floating`; the `fix_orientation` gate resolves the direction empirically
|
||
and inverts it on 8/9 M6 cases (as in the rigid benchmark).
|
||
- **`elastix` / `impact`** return the full combination transform (auto-init
|
||
folded in) in the contract direction (`auto_inverted: False`); the impact
|
||
affine read-back is `TransformParameters.0.itk.txt` (parsed by
|
||
SimpleITK, with a manual 12-parameter fallback).
|
||
- **`simpleitk`** returns the two-stage composite (rigid-init affine o
|
||
optimized affine) condensed to a single AffineTransform by point mapping.
|
||
- **`synthmorph`**'s `-T` LTA (LINEAR_RAS_TO_RAS 4×4) is the contract
|
||
transform directly, direction-checked by the usual gate.
|
||
|
||
## Reproducing
|
||
|
||
```bash
|
||
source /opt/conda/etc/profile.d/conda.sh && conda activate longitudinal-313
|
||
cd /mnt/b4/xfr/git26/longitudinal
|
||
|
||
# all nine affine pipelines (results go to results/bench_reg/bench_registration_affine/;
|
||
# 2026-10-03 run, 6 workers, ~50 min wall dominated by impact's ~6 min phase)
|
||
python scripts/bench_registration.py --affine --pipelines "antspyx ants_trsaa ants_reproa ants_quickreproa elastix impact niftyreg simpleitk synthmorph" --workers 6 --threads 24 --qa-images
|
||
|
||
# single pipeline / case subset
|
||
python scripts/bench_registration.py --affine --pipelines antspyx simpleitk --n-cases 3
|
||
|
||
# combine the per-pipeline JSONs into summary.json / summary.md / plot
|
||
python scripts/bench_registration.py --affine --merge
|
||
```
|
||
|
||
## Visual QA images
|
||
|
||
`--qa-images` writes a `checkreg`-style PNG per (pipeline, case) to
|
||
`results/bench_reg/bench_registration_affine/qa/<pipeline>__<case>.png` (9 per
|
||
pipeline; the incomplete-CT case is skipped) — same format as the other
|
||
benchmarks: a 3×2 grid of the mid-axial / mid-coronal / mid-sagittal slices
|
||
through the head center, **left column** checkerboard blend of the warped CT
|
||
vs the T1c (clean registration = seamless mosaic), **right column** additive
|
||
color overlay (CT red, T1c green; agreement = yellow, misregistration =
|
||
red/green fringing). CT is brain-windowed 0–80 HU, T1c 2–98 % head window;
|
||
the title carries the pipeline, case key and the achieved Mattes MI. |