
NIMBLE Benchmark Console
These are targets and gates, not results. No figure is published here until its phase gate passes on a locked hold-out set, reported with a 95% confidence interval. Everything currently reads AWAITING because the pipelines that would produce it are still in build.
Gates passed
0 / 6
diagnostic + quantification
System metrics
0 / 5
uniqueness proof
Modalities
5
in one pipeline, planned
Cohort
520
subjects · 3 timepoints
Split
subject
grouped, enforced in code
To v1.0
28 wk
10 phases
Evaluation scope
- Subjects
- 520
- planned
- Timepoints
- 3
- per subject
- Split
- subject
- grouped, enforced
- Hold-out
- locked
- 1 open / release
Planned class balance
- Cognitively normal18035%
- Mild cognitive impairment20038%
- Alzheimer's disease14027%
Modality coverage
- T1 MRI100%03
- fMRI62%03
- PET71%03
- EEG44%04
- Clinical100%05
Share of the planned cohort expected to carry each modality. Fusion must handle the missing remainder explicitly.
Build schedule
- 00Repo remediation
- 01Core contracts
- 02Benchmark harness
- 03Imaging pipeline
- 04EEG pipeline
- 05Fusion + models
- 06Simulation
- 07Hardware / live
- 08Console
- 09v1.0 release
Methodology guards
Subject-level splits
02No subject in train and test, across any timepoint. Asserted in code.
Locked hold-out
02Opened once per release. All tuning on a separate development set.
Bootstrap CI
02Every figure published with a 95% confidence interval.
Provenance record
01Git sha, config hash, library versions and RNG seed on every run.
Benchmark register
| ID | Task | Target | Phase | Gate |
|---|---|---|---|---|
| BM-01 | AD vs Cognitively NormalAccuracy | 88–93% | 05 | AWAITING |
| BM-02 | MCI vs NormalAccuracy | 75–85% | 05 | AWAITING |
| BM-03 | Progression — converter vs stableAccuracy | 70–78% | 05 | AWAITING |
| BM-04 | EEG-based AD detectionAccuracy | 80–90% | 04 | AWAITING |
| BM-05 | Hippocampal volumeVolume error | < 4% | 03 | AWAITING |
| BM-06 | Segmentation qualityDice coefficient | ≥ 0.85 | 03 | AWAITING |
All acceptance bands
- BM-01 AD vs Cognitively NormalAWAITING0target 88–93%100%
- BM-02 MCI vs NormalAWAITING0target 75–85%100%
- BM-03 Progression — converter vs stableAWAITING0target 70–78%100%
- BM-04 EEG-based AD detectionAWAITING0target 80–90%100%
- BM-05 Hippocampal volumeAWAITING0target < 4%8%
- BM-06 Segmentation qualityAWAITING0.60target ≥ 0.851.00
The cyan rule on each band is the floor a release must clear. A measured value appears only once its phase gate passes; until then the band stays empty rather than showing a provisional number.
Task analysis
Metric
Accuracy
Target
88–93%
Gate phase
05
Status
AWAITING
Multi-modal fusion; AUC must clear 0.90 on the locked hold-out.
Acceptance band
- Gated regions
- Lateral ventricles
- Amygdala
- HippocampusLEAD
- Entorhinal cortexLEAD
- Lead regions carry the volumetry gate — hippocampal volume error under 4% against FreeSurfer, Dice above 0.85.
System & uniqueness metrics
- SYS-0101 · 07
Source-switch cost
0 code / 1 config line
Architecture quality
- SYS-0205
Multi-modal fusion count
5 modalities
Core uniqueness
- SYS-0307 · 08
End-to-end latency
p50 / p95 published
Real-time readiness
- SYS-0401 · 09
Reproducibility
Byte-identical re-run
Scientific credibility
- SYS-0503 · 09
Full-subject processing
Wall-clock, stated hardware
Practical usability