Leaderboard · version 0.9.7.2

CAFE

CAFE tests time-series foundation models with their weights frozen, on sensor recordings from machines, structures and the natural environment. Each model reads a recording on C (whole when it is longer than 512 samples, up to the model's context; block-averaged to 512 for fixed-length models; stretched to 512 when shorter), the last 512 values on FE and windows of 256 values on A, and returns a vector, and the same small head reads that vector for every model, to classify a recording (C), find abnormal stretches (A) or warn of an event before it happens (FE). A score of 0.5 is chance and 1 is perfect, and the CAFE score is the mean of C, A and FE.

0.9.7.2 did not pass the test fixed beforehand for CAFE 1.0 (Amendment 13 item 9 (applied unchanged by Amendment 15 item 4): 62 resolved pairs with the datasets resampled, 56 needed; A signal-to-noise 4.81, 6 needed), so 0.9.5 stays the CAFE 1.0 candidate. The test · all versions compared.

Main board

2 of 15 ranked models share tier 1, so the order inside that tier is no claim. On A, the no-encoder ZERO row (0.642) scores above 8 of the 15 ranked rows; only Mantis, pj_better_10m ◊, pj_better_3p7m ◊, pjgood-warp-xview-combo-10M ◊, pj_better_1m ◊, MantisV2, NuTime score above it. Without A (C and FE only), tier 1 is pj_better_10m ◊, Chronos-2, Mantis and the ◊ ranks move: pj_better_3p7m ◊ 3 to 4; pjgood-warp-xview-combo-10M ◊ 4 to 5. The board rests on 24 C, 13 A and 15 FE datasets. Main board: one readout per model.

Forgis runs CAFE and developed 5 of the 15 ranked models, marked ◊, with CAFE test scores visible during their development. They get no special colour here. You can take them out of every table with the switch above the board. Their weights are not public, so outsiders cannot rerun them. Forgis staff chose the A additions of 0.9.7.2 with the released 0.9.7 board and its per-dataset A signal-to-noise known (from 1.0 on UKWIND to 6.1 on KUKA), after four candidates had been measured without a model (Amendment 15). Forgis staff chose the 0.9.7 drops and additions with the 0.9.6 board known; six drops were informed by scores (C 3W, bosch, soredd_abc; FE ngafid_hastall, substorm_omni, epa_aqs_pm25), and the two C additions were accepted after a screening that scored a ◊ row (pjgood-warp-xview-combo-10M) against its random-init twin on them (see the card). ◊ rows: Forgis weights, not public checkpoints. Other variants scored on CAFE or PrecursorBench Eval data before each was submitted (lower bounds, see Known limitations): LeSaturn-warppool-factorized-7.4M 43, pjgood-warp-xview-combo-10M 59, pj_better_3p7m and pj_better_10m 64, and pj_better_1m 65.

Task
Readout
Rows
Dataset groups

◊ Forgis row. Kind tells unranked rows apart: baseline, reference, tabular. Click a column to sort. The glossary near the end explains every term.

Is 0.9.7.2 the CAFE 1.0 candidate?

0.9.7.2 resolves 62 of 105 pairs with the datasets resampled, against 61 for 0.9.7, 51 for 0.9.6 and 49 for 0.9.5 and its A signal-to-noise is 4.81 (0.9.7: 4.04). The test of Amendment 13 item 9 (applied unchanged by Amendment 15 item 4) needs at least 56 pairs and an A signal-to-noise of at least 6. 0.9.7.2 fails it (an A signal-to-noise of 4.81 is below 6), so 0.9.5 stays the CAFE 1.0 candidate. In either case CAFE 1.0 also needs two fresh audits in a row with no blocker, and Jonas Petersen's explicit go.

Read the candidacy with this: 0.9.5 contains 5 A datasets that the admission rules of Amendments 13 and 15 reject (CARE_A, CARE_C, RIGS, W3 and SKAB), and it counts ESA_M1/ESA_M2 and SMAP/MSL as separate datasets, which item 2 now pools. Whether 1.0 may be a version that fails its successor's admission rules is for Jonas Petersen to decide before any 1.0 go.

The goal for the A signal-to-noise was 9; it was not met (4.81).

The pair count was fixed before any 0.9.7 cell existed (Amendment 13 item 9); Amendment 15 item 4, written before any 0.9.7.2 cell, applies the same test to 0.9.7.2; the reading of the A signal-to-noise used here was fixed in the PREREG09 note of 2026-11-05 19:02Z, after the first 0.9.7 A cells existed and before any was read (the amendment's literal wording, spread SD over pair noise, is in the table below; the table gives both readings for every version): the 105 pairs of the 15 ranked rows of 0.9.5, on the CAFE score, with the datasets resampled within each task as well as the test material (10,000 draws, Holm), by the same code for every version. Holding the datasets fixed, as the main board does, would narrow the intervals of a bigger version even if its new datasets added nothing; with the datasets resampled, a dataset that orders the models differently from the others widens them. Repeating the resampling with 5 other seeds gives the same count on all 6 seeds (62). Audit 17 repeated it with its own code over 16 seeds and got 61 to 64 (mean 62.1, SD 0.7), so differences of one or two pairs (61 in 0.9.7 against 62 here, 64 without REASSEMBLE) are within Monte Carlo noise; larger differences in the sensitivity tables are not; the pass mark of 56 is not. The A signal-to-noise is the mean absolute gap between two ranked rows on the A score divided by the median bootstrap SD of that gap (separability096.py, as on the 0.9.6 card; PREREG09 note of 2026-11-05 19:02Z); spread SD over pair noise is shown beside it.

pairs resolved, of 105, and A signal-to-noise0.9.50.9.60.9.70.9.7.2
CAFE score, datasets resampled (the measure)49516162
CAFE score, datasets fixed (main board)73738590
of which within seed noise111015
C board, datasets fixed70748080
A board, datasets fixed18376063
FE board, datasets fixed87878787
C board, datasets resampled15212525
A board, datasets resampled46918
FE board, datasets resampled67666767
A signal-to-noise (the test's measure)2.483.004.044.81
A spread SD over pair noise2.232.653.544.50

Sensitivities of the test raised by the audits (reporting only; the board above is the result):

- With every A dataset resampled by recording (the blocks of one recording form one cluster): 63 pairs resolved and an A signal-to-noise of 4.75; the test fails (test097_cr.json). No rank changes. - With KUKA with its first 100 Eval series in hash order instead of 255 (the cap of 100 for several series per asset): 62 pairs resolved and an A signal-to-noise of 4.80; the test fails (test097_k100.json). No rank changes. - With ipshocks_fluxes on the FE board (check 3 lowered to 900 events; PREREG09 notes of 2026-11-06): 60 pairs resolved and an A signal-to-noise of 4.81; the test fails (test097_ips.json). No rank changes.

On the datasets-resampled boards of 0.9.5 and 0.9.7.2 no pair is resolved in opposite directions.

On the datasets-resampled boards of 0.9.6 and 0.9.7.2 no pair is resolved in opposite directions.

No pair of ranked rows is resolved in opposite directions on the main boards of 0.9.6 and 0.9.7.2, on the CAFE score or on any task board.

Read this beside the conflict-of-interest disclosure in What changed in 0.9.7.2: Forgis staff chose the drops and adds of 0.9.7 with the 0.9.6 board known and the A additions of 0.9.7.2 with the released 0.9.7 board and its per-dataset A separability known (Amendment 15), and Forgis develops the ◊ rows.

0.9.7.2 beside 0.9.7

0.9.7.2 is 0.9.7 with the A datasets admitted under Amendment 15 and the tabular reference rows that 0.9.7 marked pending (planned as card revision v0.9.7-r1 and folded into 0.9.7.2); C and FE are as in 0.9.7. The 0.9.7 numbers come from the released files (results/r0970/, the files under data/ at the Hub tag v0.9.7), the 0.9.7.2 numbers from this build. The test of Amendment 13 item 9 (applied unchanged by Amendment 15 item 4) is the same for both.

0.9.70.9.7.2
datasets C · A · FE24 · 10 · 1524 · 13 · 15
A Eval series737992
ranked models1515
tasks in the CAFE scoreC, A, FEC, A, FE
CAFE pairs resolved, main board (datasets fixed)85 (10 within seed noise)90 (15 within seed noise)
CAFE pairs resolved, datasets resampled (the 1.0 measure, of 105)6162
C pairs resolved, datasets resampled2525
A pairs resolved, datasets resampled918
FE pairs resolved, datasets resampled6767
A signal-to-noise (the test's measure)4.044.81
A spread SD over pair noise3.544.50
test of Amendment 13 item 9 (applied unchanged by Amendment 15 item 4)not passednot passed
CAFE 1.0 candidate after the test0.9.50.9.5
CAFE tier 1pj_better_10m ◊pj_better_10m ◊ and Mantis

The test needs at least 56 pairs and an A signal-to-noise of at least 6 (goal 9).

A datasets added in 0.9.7.2 (Amendment 15; dated admission notes in PREREG09): UTAQ (85 Eval series); RCFPOWER (84 Eval series); REASSEMBLE (18 Eval series).

UKWIND changed in 0.9.7.2 (existing, pooled with the later years of the same publisher (Amendment 14 item 2): Kelmarsh and Penmanshiel 2016-2022): 17 → 85 Eval series (12 of the 0.9.7 Eval series kept, 5 fall outside the cap of 100 in hash order, 73 new; UKWIND__K06_b0004, a 0.9.7 Tuning series on which the 0.9.5 readouts were chosen, is now in Eval). Condition as recorded: the 2016 rebuild reproduces all 21 stored 0.8 series byte for byte; 0.8 label rule unchanged; Kelmarsh 2023-2024 left out (each turbine file is 81 overlapping exports, 2.17 M rows for 52,560 timestamps; decided from the file structure before any measurement). Its signal-to-noise was 1.02 in 0.9.7 (the lowest A dataset, known when Amendment 15 was written) and is 3.69 in 0.9.7.2. The mean ◊ lead on it went from +0.0084 to +0.0060. Every row's score on it changes with the material, so its per-dataset scores are not comparable across the two versions.

REASSEMBLE is built from a label-blind partial fetch (zip positions 0 to 78, one failed transfer; 74 recordings after the README known-issue drops, from which the builder's unchanged per-series rules give 18 Eval and 4 Tuning series); a full fetch would roughly double it. It has the lowest A signal-to-noise (1.35); without it 64 pairs are resolved with the datasets resampled (with it 62; a difference within Monte Carlo noise, see the test section). Its cap is 300 although all its series come from one Franka arm; with 18 Eval series neither cap binds.

Tabular reference rows (XGBoost, LightGBM, RealMLP, CatBoost, with their Selected variants; unranked): cells missing on each task board, 0.9.7 → 0.9.7.2: C 36 → 21 (incomplete rows 8 → 6; 24 datasets); A 56 → 26 (incomplete rows 8 → 6; 13 datasets); FE 30 → 25 (incomplete rows 5 → 3; 15 datasets). The missing tabular cells of this run are tasks that ran over the 1-hour per-dataset cap of Amendment 3 (23 tasks: catboost A ESA, catboost A HAI, catboost A HOT, catboost A KUKA, catboost A UAVSEAD, catboost A UKWIND, catboost A UR5E, catboost C nicola15/bdg2_meter6 (selected), catboost FE heatpump_defrost (selected), catboost FE ipshocks_fluxes, catboost FE ipshocks_fluxes (selected), lightgbm FE ipshocks_fluxes (selected), realmlp A KUKA, realmlp FE heatpump_defrost, realmlp FE heatpump_defrost (selected), realmlp FE ipshocks_fluxes, realmlp FE ipshocks_fluxes (selected), xgboost A ESA, xgboost A KUKA, xgboost A UAVSEAD, xgboost A UKWIND, xgboost A UR5E and xgboost FE ipshocks_fluxes (selected)); they were not retried, and the cells of earlier versions that are missing stay missing.

Ranks of the ranked rows, 0.9.7 and 0.9.7.2
modelrank 0.9.7tier 0.9.7CAFE 0.9.7rank 0.9.7.2tier 0.9.7.2CAFE 0.9.7.2places up
pj_better_10m ◊110.7384110.7405=
Mantis420.7326210.7385+2
pj_better_3p7m ◊220.7341320.7367-1
pjgood-warp-xview-combo-10M ◊320.7340420.7363-1
Chronos-2520.7301520.7351=
LeSaturn-warppool-factorized-7.4M ◊630.7275630.7313=
pj_better_1m ◊730.7241740.7264=
MantisV2840.7181840.7230=
NuTime950.7050950.7098=
Timestral-209M1050.69891060.7033=
Timestral-123M1160.69691160.7013=
Timestral-5M1460.69431270.6996+2
Timestral-28M1360.69431370.6987=
Toto-Open-Base-1.01260.69541470.6948-2
TimesFM-2.5-200M1570.66621580.6703=

What changed in 0.9.7.2

Amendment 13 was written into PREREG09 before any 0.9.7 download, build or cell, with the whole 0.9.6 board known. Amendment 15 (0.9.7.2) was written before any 0.9.7.2 build, measurement or cell on a dataset not yet measured and before any model cell on a 0.9.7.2 candidate, with the released 0.9.7 board and its per-dataset A signal-to-noise known (from 1.0 on UKWIND to 6.1 on KUKA). Four candidates (UTAQ, RCFPOWER, PVLAZ, NOBOOM_M202210) had already been measured without a model for 0.9.7.1 on 2026-11-05 (22:37Z to 22:49Z), so it was known that UTAQ and RCFPOWER pass the admission rules. 0.9.7.1 was never released; its candidates were folded into 0.9.7.2 (Amendment 15 item 1). Amendment 15 keeps C and FE as in 0.9.7 and adds A datasets and the tabular reference rows.

Disclosure of Amendment 15 (as written): the released 0.9.7 board and its per-dataset A separability (signal-to-noise per dataset from 1.0 on UKWIND to 6.1 on KUKA; the task value 4.04 is limited mainly by the noise of the datasets with few Eval series) were known. So were the 0.9.7.1 measurements of 2026-11-05 22:37Z to 22:49Z (UTAQ, RCFPOWER, PVLAZ, NOBOOM_M202210; no model was run on them).

The dated notes after them each give their own time and what had been computed or read when they were written; the admission notes came before any 0.9.7 or 0.9.7.2 cell on the datasets they admit (the two C additions had screening scores before that, see New datasets), and some later notes came after 0.9.7 cells or the 0.9.7 board existed (see the provenance table under Rules). Each model keeps the readout chosen for it in 0.9.5; the C head is the one of Amendment 12; A is scored at w = 256, k = 5, centre assignment, as in 0.9.6.

15 datasets of 0.9.6 are not on the 0.9.7.2 board and 9 are new, plus 2 pooled from 0.9.6 datasets. C has 24 datasets, A 13 and FE 15 (0.9.6: 25, 14, 17).

taskdatasetchangereason, as recorded
Cmv06/3wremoved or pooledAmendment 13 item 6
Cmv06/boschremoved or pooledAmendment 13 item 6
Cnicola15/soredd_abcremoved or pooledAmendment 13 item 6
Cnicola15/bdg2_meter6addedrebuilt from the exp-nicola-15 builder; X sha256 equals LOG.md (item 6)
Cnicola15/robo6_rb6addedrebuilt from the exp-nicola-15 builder; X sha256 equals LOG.md (item 6)
AESA_M1removed or pooledpooled into ESA (Amendment 13 item 2)
AESA_M2removed or pooledpooled into ESA (Amendment 13 item 2)
ACARE_Aremoved or pooledAmendment 13 item 3: labels cover most of each series, fewer than 15 Eval series (rule 2(a), (c))
ASMAPremoved or pooledpooled into NASA (Amendment 13 item 2)
AMSLremoved or pooledpooled into NASA (Amendment 13 item 2)
ACARE_Cremoved or pooledAmendment 13 item 3: labels cover most of each series, fewer than 15 Eval series (rule 2(a), (c))
ARIGSremoved or pooledAmendment 13 item 3: 5 Eval series, 7 segments (rule 2(a), (b))
AW3removed or pooledAmendment 13 item 3: median anomaly share 0.39 (rule 2(c)); a time-only score reaches 1.000
ASKABremoved or pooledAmendment 13 item 3: 8 Eval series, label to end of file (rule 2(a))
AESApooled from 0.9.6 datasetsitems 2(a)-(c) hold: 95 Eval, 97 segments, median share 0.032; elapsed-time best 0.611 (gate applies to new datasets only)
ANASApooled from 0.9.6 datasetsitems 2(a)-(c) hold: 37 Eval, 47 segments, median share 0.040; elapsed-time best 0.516 (gate applies to new datasets only)
AUAVSEADaddeditems 2(a)-(c) hold: 101 Eval, 147 segments, median share 0.083; elapsed-time best 0.585 (gate 0.65 holds)
AUR5Eaddeditems 2(a)-(c) hold: 63 Eval, 74 segments, median share 0.191; elapsed-time best 0.616 (gate 0.65 holds)
AKUKAaddeditems 2(a)-(c) hold: 255 Eval, 534 segments, median share 0.108; elapsed-time best 0.519 (gate 0.65 holds)
AUTAQaddeditems 2(a)-(c) hold: 85 Eval, 404 segments, median share 0.027; elapsed-time best 0.560 (gate 0.65)
ARCFPOWERaddeditems 2(a)-(c) hold: 84 Eval, 103 segments, median share 0.010; elapsed-time best 0.513 (gate 0.65)
AREASSEMBLEaddeditems 2(a)-(c) hold: 18 Eval, 44 segments, median share 0.115; elapsed-time best 0.556 (gate 0.65)
FEepa_aqs_pm25removed or pooledAmendment 13 item 5
FEsubstorm_omniremoved or pooledAmendment 13 item 5
FEngafid_hastallremoved or pooledAmendment 13 item 5
FEheatpump_defrostaddeditem 5 gates hold: 53,978 events >= 300; 113 systems >= 30; time since last event 0.659 <= 0.796; time of day 0.566 <= 0.585; best free forecast 0.729 <= 0.85

Datasets of one release or paper are one dataset (Amendment 13 item 2); their series are pooled and the dataset score is the mean over all pooled Eval series: ESA = ESA_M1 + ESA_M2 (95 Eval series); NASA = SMAP + MSL (37 Eval series).

The cap on series per A dataset was raised from 35 (Amendment 13 item 2): 300 for sources whose series are independent physical runs, 100 for sources with several series per asset; Tuning is the first ceil(0.15 m) in hash order, as before.

A datasetcapEval seriesTuning serieskind
ESA1009518existing, pooled ESA_M1 + ESA_M2 (item 2)
NASA100374existing, pooled SMAP + MSL (item 2)
UKWIND1008515existing, pooled with the later years of the same publisher (Amendment 14 item 2): Kelmarsh and Penmanshiel 2016-2022
HOT100509existing, re-selected under cap 100
PREDIST100418existing, re-selected under cap 100
NOBOOM300204existing, re-selected under cap 300 (lists unchanged)
HAI1005811existing, pooled with the other HAI releases (item 4)
UAVSEAD30010118new, firm
UR5E3006312new, firm
KUKA30025545new, second wave
UTAQ1008515new (Amendment 14 item 2, Amendment 15)
RCFPOWER1008415new (Amendment 14 item 2, Amendment 15)
REASSEMBLE300184new (Amendment 14 item 2, Amendment 15)

NOBOOM is recorded with cap 300, although one distillation plant would fall under the cap of 100 for several series per asset; with 20 Eval series neither cap binds, so its list is the same either way.

A candidates not admitted, with the reason recorded before any cell on them (full measurements under Admission):

candidatestatuskindreason
RFLYMADoutnew, conditionalitem 4 elapsed-time gate: 0.731 > 0.65
PMUoutnew, conditionalthe continuous 1 s TransnetBW record (Zenodo 15784548) is one channel (CE frequency) and ends 2024-04-25, covering 23 of 88 cases; P4 needs >= 2 channels and 23 cases give at most 19 Eval series and 19 segments (rule 2(b)); not built
ONBOARDEUoutnew, conditionalitem 4 elapsed-time gate: 0.708 > 0.65
TCABRoutnew, conditionalitem 4 elapsed-time gate: 1.000 > 0.65
BASICoutnew, conditionalcheck 1 / item 4 condition: the flights are software-in-the-loop, not real (Ahmad and Akram, Data in Brief 2024, doi:10.1016/j.dib.2024.110069: 'The BASiC dataset is exclusively derived from simulations conducted in a SITL environment using the ArduPilot platform'); not built
EAWAGoutnew, second waveonly 4 ponds (2D, 3D, 5A, 5B), phase 1, carry the expert 'Anomaly' column, with 5 segments in total: rules 2(a) and 2(b) cannot hold; the licence field is empty (portal: public domain for data outside Swiss copyright)
CAFUC2outnew, second wavecheck 2 (physical label; PREREG09 note of 2026-10-31 ~20:50Z: injected or staged faults pass when the onset is observed in the system): CAFUC2's faults are synthetic signals written into five copies of each recorded flight after the flight; no aircraft experienced them, so the label is not a state of or an event in the physical system. Decided before any cell; only Jonas Petersen can reverse this, in a dated note
GREEKWINDoutnew, second wave2(a): 7 Eval series < 15; 2(b): 8 anomaly segments < 20
CRANFIELDoutnew, second wavethe official download (MATLAB File Exchange #50938) needs a MathWorks sign-in; the other copies (Kaggle, IEEE DataPort) need a login; item 4 / Amendment 13: sets that need a registration are not added
MASToutnew, second waveno precursor-only cut without an elapsed-time leak: a scored block that ends before the current quench carries the precursor label at its end (a rising elapsed-time score reaches 0.933 on synthetic series of that shape); TokaMark ships no disruption labels or quench times
PVLAZout0.9.7.1 candidate (Amendment 14), decided under Amendment 15rule 2(a): 5 Eval series < 15; item 4 elapsed-time gate 0.683 > 0.65
NOBOOM_M202210out0.9.7.1 candidate (Amendment 14), decided under Amendment 15check 3/4: 85 of its 89 runs are NOBOOM runs with the same X and a second label column (21 kept series byte-identical to NOBOOM series; pooled, 2 Eval pairs and 3 Eval-Tuning pairs share X); NOBOOM stays as in 0.9.7
TSB-ADoutscreened, not built (Amendment 15)no source collection passes item 2 and checks 1-4 (simulated or injected events, IT/web/finance metrics, recordings of people, imputed or undocumented labels; work/r097/tsbad_screen0972.md); SMAP and MSL stay in NASA, SWaT needs an access form
LEAD1.0outscreened, not built (Amendment 15)check 4: its meters are the BDG2 electricity meters of the ASHRAE GEPIII subset, the campaign of the C unit nicola15/bdg2_meter6; no licence stated (data behind a Kaggle login)
USGS NWISoutscreened, not built (Amendment 15)check 2: on ice-affected stretches the published discharge is an estimate written in place of the record, so the label marks an estimate or a gap, not a recorded event; no equipment-coded stretch found

The C candidate nicola15/sns_hvcm_b_end had to show a pretraining effect on its own first cells (Amendment 13 item 6): pjgood-warp-xview-combo-10M minus the random-init LeSaturn row (3.7 M parameters, another architecture; not a matched twin of the 10.6 M combo) positive in at least 4 of 5 repeats and a mean of at least +0.02. It was positive in 3 and the mean was +0.0169, so it is out. The gate was not set blind: the exp-nicola-15 screening of the same morning had already scored combo against its random-init twin on this dataset (+0.026 ± 0.021, 4 of 5 repeats; LOG.md), a result that would have passed it. The rule is score-informed; it is computed for every C dataset under Admission.

nicola15/bdg2_meter6 and nicola15/robo6_rb6 were admitted on the sha256 check of item 6 (the rebuilt X equals the one recorded in the screening). That check reads no score, but the screening that accepted them earlier on 2026-11-05 did: it scored pjgood-warp-xview-combo-10M ◊ against its random-init twin on these builds: nicola15/bdg2_meter6 0.649 against 0.597, difference +0.053 ± 0.010, 5 of 5 repeats; nicola15/robo6_rb6 0.603 against 0.501, difference +0.101 ± 0.013, 5 of 5 repeats (exp-nicola-15 LOG.md), so these additions were score-informed too. The combo-minus-twin numbers of 0.9.7 are shown under Admission.

CAFUC2 (general-aviation sorties) is out (2026-11-05T18:56Z): check 2 (physical label; PREREG09 note of 2026-10-31 ~20:50Z: injected or staged faults pass when the onset is observed in the system): CAFUC2's faults are synthetic signals written into five copies of each recorded flight after the flight; no aircraft experienced them, so the label is not a state of or an event in the physical system. Decided before any cell; only Jonas Petersen can reverse this, in a dated note. It would otherwise pass items 2 and 4 (96 Eval series, 286 segments, median share 0.021, elapsed-time best 0.508). Only Jonas Petersen can reverse it, in a dated note.

FE admission is model-free (Amendment 13 item 5): at least 300 events and 30 independent units or days, a time-since-last-event row at most 0.7956 (the highest on the 14 kept FE datasets, nasa_battery_rw), a time-of-day row at most 0.5851 (towie_genbearing_hot), and the best free forecast at most 0.85. The cap on lab-friction sets is three.

Everything else is as in 0.9.6.

Conflict of interest (Amendment 13 item 1, as written before the run): Forgis owns the ◊ rows and Forgis staff chose these rules and drops with the board known. The drops of FE substorm_omni, epa_aqs_pm25 and ngafid_hastall and of C bosch and soredd_abc were informed by scores (separability and combo minus twin) and are score-informed selection in the sense of Amendment 3. The A and 3W drops follow the structural rules of item 2 and the 0.9.5 evidence (3W: Amendment 12 era decision of exp-nicola-15). The thresholds of item 2 (15 series, 20 segments, share 0.3) were placed in gaps seen in the 0.9.6 data (kept datasets reach a share of 0.21; the five removed ones start at 0.35), so they are post-hoc cut-offs, not independent ones. The card shows the 0.9.6 board beside 0.9.7, the effect of every drop and add on every row, and the ◊ lead under each.
Note added 2026-11-06: the 3W drop called structural here was made on combo minus twin (item 6 and the exp-nicola-15 screening), so six drops were score-informed, and the two C additions were accepted after a screening that scored a ◊ row on them.

The mean lead of the ◊ rows over the other ranked rows, on the CAFE score, under each board (reported, part of no rule):

boardmean ◊ row minus mean other ranked row (CAFE)best ◊ row minus best other row (CAFE)
0.9.6 board+0.0262+0.0034
0.9.6 with the drops of 0.9.7 and no additions (sensitivity drops_only)+0.0308+0.0073
0.9.7 board (released)+0.0284+0.0057
0.9.7.2 board+0.0268+0.0020
A task only, 0.9.7 (first on A: pj_better_10m ◊)+0.0264+0.0107
A task only, 0.9.7.2 (first on A: Mantis)+0.0215-0.0004

Every ranked row on the 0.9.6 datasets (computed by the 0.9.7.2 code), on those datasets with the drops of 0.9.7 only (no addition, no pooling), and on 0.9.7.2: CAFE score, rank and tier. The 5 largest gains from the drops alone are exactly the ◊ rows.

model0.9.6 datasetsranktierdrops onlychangeranktier0.9.7.2ranktier
pj_better_10m ◊0.7363110.7480+0.0117110.740511
Mantis0.7329310.7407+0.0078410.738521
pj_better_3p7m ◊0.7347210.7450+0.0103210.736732
pjgood-warp-xview-combo-10M ◊0.7326410.7438+0.0112310.736342
Chronos-20.7301510.7389+0.0087510.735152
LeSaturn-warppool-factorized-7.4M ◊0.7270620.7386+0.0116610.731363
pj_better_1m ◊0.7224720.7315+0.0091720.726474
MantisV20.7192820.7275+0.0083820.723084
NuTime0.7067930.7108+0.0041930.709895
Timestral-209M0.70041030.7062+0.00571030.7033106
Timestral-123M0.69901130.7045+0.00551130.7013116
Timestral-5M0.69651430.7024+0.00591330.6996127
Timestral-28M0.69651330.7023+0.00571430.6987137
Toto-Open-Base-1.00.69721230.7031+0.00581230.6948147
TimesFM-2.5-200M0.66571540.6699+0.00421540.6703158

On the 1.0 measure (pairs resolved with the datasets resampled), the 0.9.6 datasets give 51, the drops alone 58 and 0.9.7.2 62: taking the drops first, of the +11 pairs +7 come from the drops, which were chosen with the 0.9.6 board and its separability table known, and +4 from the additions, the pooling and the regrown A lists (another order could split the gain differently).

The dated notes of 0.9.7 and 0.9.7.2 (PREREG09, after Amendment 13, with Amendment 15)

Note 2026-11-05 18:46Z (0.9.7 admission, wave 0: HOT; Amendment 13 item 8 step 2). Written before any 0.9.7 cell on HOT's new series exists. HOT (Hill of Towie, existing A dataset) re-selected under cap 100 (item 2; several series per turbine) with data097/src/select097.py (= select096 with the per-source cap): 59 candidates after construction, chance computed for all 59, none excluded, 59 accepted, 50 Eval and 9 Tuning (0.9.6: 29 and 6). 24 series are new; 3 old Eval series move to Tuning under the unchanged hash rule. Measured with work/r097/measure097.py (validated: reproduces nulls095 HOT to 1.1e-16): 107 anomaly segments in Eval, median scored anomaly share 0.0142, 50 Eval series with prefix >= 512, elapsed-time best 0.529 (centre). Items 2(a), (b), (c) hold: HOT is in. The board uses the stored 0.9.6 cells for the 26 + 3 old series and new cells for the 24 new ones; the old series are also recomputed by the 0.9.7 tree as a reproduction check. Record: data097/admit097.json (wave 0). No 0.9.7 score has been read.

Note 2026-11-05 18:57Z (0.9.7 admission, wave 1; Amendment 13 item 8 step 2). Written before any 0.9.7 cell on the units it admits. Record: data097/admit097.json, generated by work/r097/admit097.py from the measure097.py files (no model read). In (rules 2(a)-(c) hold; the elapsed-time gate holds for the new ones): ESA (ESA_M1 + ESA_M2 pooled, 95 Eval, 97 segments, median share 0.032, elapsed 0.611), NASA (SMAP + MSL pooled, 37, 47, 0.040, 0.516), UKWIND (17, 41, 0.057, 0.558; unchanged), HOT (50, 107, 0.014, 0.529; cap 100), PREDIST (cap 100: 41 Eval, 8 Tuning, was 29 / 6; 41 segments, 0.051, 0.512; 14 new series, PREDIST__m1_s17_e29 and __m2_s19_e29 move to Tuning), NOBOOM (cap 300; lists unchanged: 20, 37, 0.191, 0.561), UAV-SEAD (new, firm: 101 Eval, 147 segments, 0.083, elapsed 0.585 <= 0.65; real PX4 flights; 25 Hz grid chosen on a label-free count of flights reaching a 512-row prefix), UR5e (new, firm, CC BY-NC 4.0: 63 Eval, 74 segments, 0.191, elapsed 0.616; status, safety, program-node, program-loop, digital IO (bit 5 = the labelling button), speed-scaling and time registers dropped). Gates on the existing lists: measure097 equals nulls095/nulls096 per series (max |diff| 2.2e-16); PREDIST and NOBOOM rebuilt from raw reproduce every 0.9.6 series (X, y, tr, channels; PREDIST's unread ts array differs in unit only, pandas version). Out: Eawag (only 4 ponds carry the expert label, 5 segments: rules 2(a), (b) cannot hold), FAIR-MAST (a precursor-only cut puts the label at the end of every scored block, so elapsed time finds it; TokaMark ships no disruption labels), CAFUC2 (check 2: its faults are synthetic signals written into copies of recorded flights after the flight, so the label is not an event in the physical system; 96 Eval would otherwise pass items 2 and 4; Jonas may reverse this in a dated note). Pending: HAI pooling, RflyMAD, BASiC, TCABR, PMU, OnboardEU, KUKA, Cranfield, Greek wind. No 0.9.7 score has been read.

Note 2026-11-05 19:02Z (reading of Amendment 13 item 9, A signal-to-noise; written before any 0.9.7 score exists). Item 9 defines the A signal-to-noise as "spread SD of the ranked rows over their pair noise, as the 0.9.6 card computes it" and anchors it at "the 3.0 of 0.9.6". The 0.9.6 card's number is the snr field of results/separability096.json (separability096.py: mean absolute point difference over all ranked-row pairs of the task score divided by the median bootstrap SD of the paired difference), which is 2.997 for A; spread SD over pair noise is 0.02110 / 0.00795 = 2.65. The test uses the card's quantity, the one that gives 3.0 (threshold 6, goal 9), computed by separability096.py unchanged on board097; spread SD over pair noise is reported beside it for 0.9.5, 0.9.6 and 0.9.7. No rule is changed.

Note 2026-11-05 19:05Z (deviation from Amendment 13 item 6, order of the SNS HVCM gate cells; disclosed, no rule changed). Item 6 says the gate cells of nicola15/sns_hvcm_b_end (combo-10M and its random twin, 5 repeats) are "computed before any other 0.9.7 cell". They were not: A cells of the units admitted in waves 0 and 1 (HOT, PREDIST, UAV-SEAD, UR5e; Clariden jobs 3867625, 3867627, 3867656) were computed first, because the C unit had not been rebuilt yet (nicola-l4 unreachable). No 0.9.7 score of any kind has been read: the only checks run on those cells are equality checks against stored 0.9.6 cells. The SNS rule is mechanical (combo-10M minus twin positive in at least 4 of 5 repeats and mean at least +0.02) and reads only SNS cells, so the order cannot have informed it. Its gate cells are computed as soon as the unit is rebuilt and accepted by the sha256 check, and its decision is written in a dated note before any other 0.9.7 C cell is read. The card states this deviation.

Note 2026-11-05 19:10Z (0.9.7 C admission; Amendment 13 item 6). VM nicola-l4 was unreachable (SSH host key verification failed; not bypassed), so the three exp-nicola-15 C units were rebuilt from their builders (branch dev/c-candidate-checks, builders byte-identical to the logged commits 885ab0cd0, 99988a280, 28ae362c4). X sha256 (raw float32 bytes of prepared.npz X) equals LOG.md for all three: bdg2_meter6 82e03cf1..., robo6_rb6 935ae4ca..., sns_hvcm_b_end b8dcbe28.... In: nicola15/bdg2_meter6 (CC BY-SA 4.0; 75,409 windows, 1,615 groups, 6 classes), nicola15/robo6_rb6 (MIT; 6,869 windows, 5,352 groups, 6 classes). Pending: nicola15/sns_hvcm_b_end (CC BY 4.0; 3,323 windows, 15 groups, 2 classes) until its combo-minus-twin gate (cgate097.py) is computed; its first gate run lost the twin cell to a missing result directory and is being rerun. No 0.9.7 C cell has been read. Record: data097/admit097.json (C section).

Note 2026-11-05 19:12Z (0.9.7 admission, wave 2; Amendment 13 item 8 step 2). Out: BASiC (item 4 condition: its own paper, Ahmad and Akram, Data in Brief 2024, doi:10.1016/j.dib.2024.110069, states the flights are software-in-the-loop simulations; not built) and RflyMAD real flights (built from the real-flight subset only, non-commercial, 255 Eval series, 405 segments, median share 0.289; fails the elapsed-time gate: rising 0.731 > 0.65, the faults come late in the flights). No 0.9.7 score has been read. Record: data097/admit097.json.

Note 2026-11-05 19:13Z (Amendment 13 item 6: SNS HVCM gate). nicola15/sns_hvcm_b_end gate cells (head H1, 5 repeats; combo-10M from Clariden job 3867712, its random twin from 3867716 after the first twin task failed on a missing result directory): combo minus twin per repeat -0.0187, +0.0161, +0.0473, -0.0146, +0.0546; positive in 3 of 5 (needs 4), mean +0.0169 (needs +0.02). SNS HVCM is out of 0.9.7. C has 24 datasets (25 of 0.9.6 minus 3 plus bdg2_meter6 and robo6_rb6). No other 0.9.7 C cell has been read. Record: results/cgate097.json, data097/admit097.json.

Note 2026-11-05 19:15Z (Amendment 13 item 5, F3 thresholds; computed 18:53:31Z, before any new FE download or build). On the 14 kept FE datasets of 0.9.6, from the stored PrecursorBench v1.4 reference curves (no model involved): the time-since-last-event row (PB v1.4 "event timing") reaches at most 0.7956 (nasa_battery_rw); a pure time-of-day row (hour-of-day harmonics 1 and 2 of the last observed sample; defined on the 9 datasets with a calendar) reaches at most 0.5851 (towie_genbearing_hot). A new FE dataset is admitted only if both rows stay at or below these values, with at least 300 events, at least 30 independent units or days, and the best free-forecast row at most 0.85. File: data097/F3_thresholds097.json (copied from the FE builder's output, written 18:53:31Z; the first FE download started 18:54Z per results/timing097.json).

Note 2026-11-05 19:21Z (0.9.7 admission, wave 3; Amendment 13 item 8 step 2). HAI in (pooled, item 4): HAI 22.04 plus 20.07, 21.03 and 23.05 of the same testbed (CC BY-SA 4.0), built with the 22.04 rules; build_hai.py reproduces all 21 old 22.04 series byte for byte. Two run-level decisions, both made before any HAI cell: (i) "hash order unchanged" (item 2) is read as each source keeping its own hash tag, so pooled HAI is ordered with HAI's tag A08|, which puts the 21 old series exactly in their 0.9.5 Eval/Tuning places (with A083| five old series would have moved); select097.py reads an optional hash_tag per source (default A083|); (ii) HAIEnd 23.05 is left out: it carries the same two 23.05 test recordings and the same label file through a different set of tags, so its series repeat physical attacks already in HAI 23.05 ((only Jonas Petersen can reverse this, in a dated note); with it HAI would have 72 Eval series, 134 segments, elapsed 0.527, also passing). Result: 58 Eval, 11 Tuning, 103 segments, median scored share 0.030, elapsed best 0.533. OnboardEU out: clean single-defect subset chosen by the label text before any build (10 of 25 locations), 85 Eval series, but the elapsed-time gate fails (rising 0.708 > 0.65). No 0.9.7 score has been read.

Note 2026-11-05 19:23Z (0.9.7 admission, wave 4, the final A list, and the A gate threshold; Amendment 13 items 4 and 8; before any 0.9.7 A cell is read). In: KUKA LWR4+ collisions (second wave; Zenodo CC BY 4.0, description MIT): 255 Eval series from 148 recordings, 534 segments, median share 0.108, elapsed best 0.519. Its condition holds as written: the labels are the authors' hand-entered collision onsets (y = 1 on [m - 20, m + 492) as in their example.m), whose counts match the experimenter's independent log in 200 of 206 recordings; the external torque at the marked onset varies (coefficient of variation 0.90) and a threshold is crossed a median 5.5 samples after it, so the labels are not thresholds on the torque. Disclosed: a single threshold of about 1.75 Nm on the external-torque channels finds the collisions with F1 0.92, so this dataset is easy for a level detector (the max-|z| reference row will show it). Out: TCABR (the time-matched precursor build gives 3 series because disruptive and normal shots come from different days; the disruptive-only build has elapsed best 1.000 > 0.65), PMU (the continuous 1 s record is one channel and covers 23 of 88 cases: P4 and rule 2(b) fail; not built), Greek wind farm (7 Eval series, 8 segments: rules 2(a), (b)), Cranfield (the official download needs a MathWorks sign-in). Final 0.9.7 A list (10 datasets, 737 Eval series): ESA 95, NASA 37, UKWIND 17, HOT 50, PREDIST 41, NOBOOM 20, HAI 58, UAV-SEAD 101, UR5e 63, KUKA 255. One flight set (UAV-SEAD) of 10, within the cap of a third. A gate threshold (Amendment 5 rule 5(b), recomputed over the final list as item 4 requires): 0.5540 = mean over the 10 datasets of the highest data-free null (centre assignment; per dataset the largest of the Eval means of elapsed time rising, falling and the random window score, from measure097.py): ESA 0.6109, NASA 0.5162, UKWIND 0.5578, HOT 0.5293, PREDIST 0.5124, NOBOOM 0.5605, HAI 0.5328, UAV-SEAD 0.5851, UR5e 0.6156, KUKA 0.5194. A enters the CAFE score if at least half the ranked rows score above it. File: results/agate097.json. No 0.9.7 A cell has been read.

Note 2026-11-05 19:44Z (0.9.7 FE admission, wave 1; Amendment 13 item 5). In: heatpump_defrost (HeatpumpMonitor.org, CC BY 4.0; event = defrost cycle; inputs = measured temperatures and power; built by data097/src/fe_build_heatpump.py with the PrecursorBench v1.4 rule). Model-free gates (reference rows scored as the other FE reference rows, data097/fe_gates_heatpump_defrost.json): 53,978 events (>= 300), 113 independent installations (>= 30), time since last event 0.659 (<= 0.7956), time of day 0.566 (<= 0.5851), best free-forecast row 0.729, curvature (<= 0.85). IPShocks and PyrNet are still building; each is out if not built and gated by 21:30Z. The FE head for the new unit is the torch head of the dated note 011865766 (0.9.6); its reproduction of the 0.9.5 numpy cells holds through the 0.9.7 tree (ZERO on nasa_battery_rw, all draws equal, DECISIONS097.md), so the FT tasks run without a new gate file. No 0.9.7 FE cell has been read.

Note 2026-11-05 21:59Z (0.9.7 FE admission, wave 2, final FE list; Amendment 13 item 5). In: ipshocks_fluxes (IPShocks catalogue, CC BY 4.0, events = shock arrivals at L1; inputs = ACE and Wind field, plasma and particle fluxes from NASA CDAWeb, 1998-2021, 5-min bins). Gates (computed 19:47Z, data097/fe_gates_ipshocks_fluxes.json): 914 events, 629 distinct shock days (independence counted by day because both spacecraft sit at L1), time since last event 0.490, time of day 0.513, best free forecast 0.695 (current value). Builder choices made after a structural test but before any gate score: 1-hour events, lead times 5 min to 4 h, row thinning as build_ngafid. For heatpump_defrost one builder choice was made after a build and before any score: feed stretches shorter than 1,024 min are set missing. Out: PyrNet (not built and gated by the 21:30Z cutoff). Final FE list: the 14 kept datasets plus heatpump_defrost and ipshocks_fluxes = 16. Disclosed: on two kept FE datasets the best free-forecast row is already above the 0.85 gate that new FE datasets must meet (memristor_jerk 0.882, nasa_battery_rw 0.873; data097/F3_thresholds097.json); the gate applies to new datasets only, so they stay. No 0.9.7 FE cell has been read.

Note 2026-11-05 22:32Z (responses to audits 1 and 2; disclosures; no rule changed). Two fresh audits (work/r097/audits/audit1_rules.md, audit2_code.md) found no blocker. Context: at about 22:10Z a provisional board (the two new FE datasets and two HAI rows still without cells) was built to check the code end to end, so 0.9.7 scores have been visible since then; for that reason no admission rule or decision below is changed now, and every response is a disclosure or a reporting-only sensitivity. 1. KUKA cap (audit 1 R1, audit 2 F8). KUKA was admitted at 19:23Z (before any A cell was read) with cap 300, reading its 206 recordings as independent runs of a robot program, as UR5e's runs were read; item 2 gives 100 to "several series per asset", and KUKA is one robot arm with several blocks per recording (255 Eval series from 148 recordings). The cap stays as admitted. Reported beside the result: the board and the item-9 test with KUKA at its first 100 Eval series in hash order (results/test097_k100.json). 2. Within-dataset independence (audit 1 R2, audit 2 F9). The A bootstrap resamples series, as in 0.9.6 (item 7); blocks of one recording (HAI, KUKA, ESA, UAV-SEAD, HOT, UKWIND) are resampled as if independent except the near-duplicate clusters of 0.9.5. Reported beside the result: the board and the item-9 test with every A dataset resampled by recording (results/test097_cr.json). 3. PyrNet cutoff (audit 1 R3). The 21:30Z cutoff for the FE builds was set by the orchestrator at 19:45Z in a message to the builder (data097/fe_status097.md); it is not in Amendment 13 (whose item 8 has no deadline beyond the plan's budget) and is earlier than the plan's suggested 4-hour phase-1 cut-off (~22:42Z). PyrNet's 4.8 GB download finished at 21:58Z and its builder never ran. PyrNet is not in 0.9.7; it is a 0.9.7.1 candidate. 4. Wording of the 19:02Z note (audit 1 R4). That note says "before any 0.9.7 score exists"; A cells of waves 0 and 1 existed from 18:49Z but none had been read. Read it as "before any 0.9.7 score was read". 5. A_series gate tolerance (audit 1 R6). The gate comparing recomputed old series with their stored cells was exact until PREDIST and HAI were recomputed; it now allows 1e-15 (written 22:16Z, after the 1.1e-16 differences appeared) because select097 recomputed those datasets' chance values (difference about 1e-17). The board uses the stored cells for every old series, so no score depends on it; the bit-for-bit gate of one unchanged unit per task (A_repro, 49/49 exact) is unchanged. 6. FE caveats (audit 1 R8, R10). heatpump_defrost's event (heat output below 0 W while the compressor draws more than 50 W) is defined partly on electrical power, which is also an input; the F3 thresholds were computed from 10 head seeds on the kept datasets and the new datasets' gate rows from 3. Both are stated on the card. 7. Other disclosures. UAV-SEAD keeps battery voltage, which falls through a flight (its elapsed-time score is 0.585, inside the gate) (audit 1 R13). The SNS gate compared combo-10M with the board's random-init LeSaturn row (3.7M parameters), the twin used for "combo minus twin" since 0.9.6 (audit 1 R16). A at w = 512 (item 7) is not computed in this run unless time remains (audit 1 R11). The leave-one-out boards hold every non-admitted dataset out so that a campaign sibling cannot enter (fixed in sens097.py at 22:06Z, before the final board).

Note 2026-11-06 10:02Z (ipshocks_fluxes and standing check 3; written after the 0.9.7 board was built, before any ipshocks_fluxes score was read). The board leaves ipshocks_fluxes out under the standing admission check 3 (section "Admission", item 3: "FE: at least 1,000 events"), which Amendment 13 item 7 keeps ("everything else is 0.9.6"): it has 914 events. Amendment 13 item 5 sets the gates for new FE datasets (at least 300 events, at least 30 independent units or days, F3, best free forecast at most 0.85), and the note of 21:59Z admitted it under those gates as the 16th FE dataset; neither text says whether the 300-event gate replaces check 3 for new FE datasets. The conflict was not seen before: the provisional runs of 22:00Z to 22:30Z had no cells on the new FE units and left them out for that reason. The board code (board097.py, unchanged since before any 0.9.7 score) applies check 3, and the run's escalation rule takes the default "leave the unit out" when a decision needs Jonas Petersen. 0.9.7 therefore has 15 FE datasets; ipshocks_fluxes is listed as admitted by item 5 and left out by check 3. Its cells exist (Clariden job 3868625) and are kept. Only Jonas Petersen can reverse this, in a dated note; the board would then be rebuilt with 16 FE datasets and every result file regenerated. The 0.9.7 test of item 9, the tiers and every reported number are those of the 15-dataset board. No ipshocks_fluxes score has been read.

Note 2026-11-06 10:10Z (addendum to the note of 10:02Z, after audit 3 finding F-A1; no rule changed). The 10:02Z note says no ipshocks_fluxes score had been read. The writer had not looked at one, but that cannot be shown: the cells of every row on ipshocks_fluxes were in ~/.hepa09/r097/cells097/F.json from the build of about 09:25Z and in the card's data/F.json. Audit 3 then built the board with ipshocks_fluxes (check 3 lowered to 900 events) as a check. That board is now a reported sensitivity (results/board097_ips.json, test097_ips.json; release097.sh), shown on the card beside the test. Any reversal of the 10:02Z default would therefore be made with those scores known, and the card would say so. The published board stays the 15-FE-dataset board.

Note 2026-11-06 11:05Z (decision on ipshocks_fluxes by Jonas Petersen; closes the notes of 10:02Z and 10:10Z). Jonas Petersen decided: ipshocks_fluxes stays out of 0.9.7. Standing check 3 (FE at least 1,000 events) holds for every FE dataset, new ones included; Amendment 13 item 5's 300-event gate is an additional gate for new FE datasets and does not replace check 3. The decision was made with the board with ipshocks_fluxes known (sensitivity results/board097_ips.json: 58 pairs resolved against 61, pj_better_3p7m and pjgood-warp-xview-combo-10M swap ranks 2 and 3; the item-9 test fails either way), and is the default the run took at 10:02Z. 0.9.7 has 15 FE datasets.

Amendment 15 (CAFE 0.9.7.2, written 2026-11-06 ~23:50Z (first headed 2026-11-07 ~00:10Z, a clock error; commit 199d87582 at 23:51:45Z), before any 0.9.7.2 build, measurement or cell on a dataset not yet measured, and before any model cell on a 0.9.7.1 or 0.9.7.2 candidate; the released 0.9.7 board, card and tags are not changed by it). Requested by Jonas Petersen on 2026-11-06 at ~23:45Z: a 0.9.7.2 with more audits, more robustness checks and more separability on A, adding further A datasets that pass every rule (the TSB-AD collection named as a source), aiming at an A signal-to-noise above 9; published with the earlier versions kept on the leaderboard, and checked so that the 1.0 conversion can run on his go. Disclosure: the released 0.9.7 board and its per-dataset A separability (signal-to-noise per dataset from 1.0 on UKWIND to 6.1 on KUKA; the task value 4.04 is limited mainly by the noise of the datasets with few Eval series) were known. So were the 0.9.7.1 measurements of 2026-11-05 22:37Z to 22:49Z (UTAQ, RCFPOWER, PVLAZ, NOBOOM_M202210; no model was run on them). Conflict of interest: as in Amendment 13 item 1. 1. Versions. 0.9.7.1 was never built past the measurements or published. Amendment 14's candidates are carried into 0.9.7.2, and the name 0.9.7.1 is not used for a release. The tabular reference update planned as card revision v0.9.7-r1 is also folded into 0.9.7.2. 2. Rules. Amendment 13 items 2 and 4 and Amendment 14 items 1 and 2 are unchanged and apply to every candidate, measured with measure097.py before any cell. Checks 1 to 4 of Amendment 8 apply as written (physical system, physical label, test material, one dataset per campaign). No existing A dataset is dropped or changed, except by a pooling that Amendment 14 already names (NOBOOM_M202210 into NOBOOM; the later UKWIND years into UKWIND). 3. TSB-AD. Each source collection in TSB-AD-U and TSB-AD-M is judged on its own against item 2, from the publisher's original files where TSB-AD has cut, resampled or normalized them. Sources already decided keep their decision (SMAP and MSL are in NASA; SWaT needs an access form; SKAB was removed by Amendment 13). Sources whose anomalies were injected into recorded data by software, simulated plants, IT or web metrics, and recordings of people or animals fail check 1 or check 2 and are listed with that reason. 4. The 0.9.7.2 board is the 0.9.7 board plus the admitted A datasets and the tabular reference rows. C and FE do not change. Item 9 of Amendment 13 applies unchanged: at least 56 of the 105 pairs resolved with the datasets resampled, and an A signal-to-noise of at least 6 (goal 9). 0.9.7 is shown beside 0.9.7.2. If 0.9.7.2 passes, it is the 1.0 candidate; the conversion still runs only on Jonas's explicit go. 5. Order of work and cut-off. As Amendment 13 item 8: build and measure, then a dated admission note, then cells. A candidate whose units are not on Clariden by 2026-11-07 03:30Z is not in 0.9.7.2. Its builder and measurement are handed over in a ranked list. A candidate on which any ranked row has no cell after two attempts leaves 0.9.7.2 as a whole, with the reason on the card. No admission, pooling or cut-off decision reads a model score on a candidate. 6. Exit. At least three fresh audits, the last two in a row with no blocker, before the upload (Amendment 5 rule 13 for 1.0 is unchanged).

Note 2026-11-06 23:54Z (first headed 2026-11-07 00:35Z, a clock error; commit dcf9de80a at 23:54:39Z) (0.9.7.2 admission, wave 1; Amendment 15 item 5; before any model cell on the dataset). In: UTAQ, the Utah aquatic sensor networks (Logan River Observatory with the iUTAH GAMUT Red Butte and Provo networks; HydroShare, CC BY 4.0, open). Measured on 2026-11-05 at 22:49Z with measure097.py, no model run: 85 Eval and 15 Tuning series (site-year P2 blocks, cap 100), 404 anomaly segments in Eval, median scored anomaly share 0.027, all 85 Eval series with a prefix of at least 512, elapsed-time best 0.560 (centre; gate 0.65). Rules 2(a), 2(b), 2(c) and the elapsed-time gate hold. Amendment 14's condition holds as built: the label is the union of the technicians' qualifier codes for sensor malfunction, power failure, maintenance, sediment, low water and calibration, fixed in the builder before any series was built (~/stage097/logs/build_utaq_labelrule_frozen_2240Z.py, 22:40Z), and the inputs are the raw logger files, not the quality-controlled files. Checks 1 to 4 of Amendment 8: in-stream sondes are a physical system; the qualifier codes mark instrument and site events entered by technicians, not thresholds; Eval and Tuning are disjoint site-year blocks; the three networks are one dataset (same authors, item 2 pooling rule). Not a flight set. The A gate threshold is recomputed over the final 0.9.7.2 A list in the note that closes the list, before any 0.9.7.2 A cell is read.

Note 2026-11-07 00:02Z (first headed 00:15Z, a clock error; commit a1f71c5f6) (Amendment 15 item 3 result; two further candidates named before any download or build of them). TSB-AD: no source collection passes (work/r097/tsbad_screen0972.md). GHL, CATSv2 and Exathlon are simulated or carry injected events; SMD, PSM, WSD, IOPS, NEK, YAHOO, Stock and CreditCard are IT, web or finance metrics; MITDB, SVDB, LTDB, Daphnet and OPPORTUNITY record people; GECCO's events were imputed by the organisers; TAO has no documented label and no series with a 512-sample prefix; Power labels holidays; Genesis needs a login; NAB has two physical non-human series; the UCR families are synthetic or hand-edited anomalies with no 15 independent series from one campaign. SMAP and MSL stay in NASA; SWaT needs an access form. Nothing was built. Named as candidates under the unchanged rules of item 2 (each out unless it passes every rule and checks 1 to 4 when built): LEAD1.0 (building electricity meters with human-annotated anomalies; admitted only if its labels are human annotations of recorded events, the licence is usable, and it is not the same campaign as the C unit nicola15/bdg2_meter6, which is drawn from BDG2; if it is, it is out under check 4); USGS NWIS gauges with equipment or ice qualifier codes (admitted only if the qualified stretches keep the recorded signal in the published record). No score of any kind exists on either.

Note 2026-11-07 00:10Z (committed 00:07:31Z as 4683f1fec; the heading time was written ahead of the clock) (the two candidates of the 00:02Z note; no download of model data, no build, no score). Out: LEAD1.0, check 4: its meters are the BDG2 electricity meters of the ASHRAE GEPIII subset (LEAD paper, arXiv 2203.17256; BDG2 README), the campaign the C unit nicola15/bdg2_meter6 is built from; it would also fail the licence rule (no licence stated; the full data sit behind a Kaggle competition login). Its labels would have passed (people annotated every point with a web tool). USGS NWIS: on ice-affected stretches the published discharge is an estimate written in place of the record and the recorded gage height carries no qualifier, so the label marks an estimate or a gap, not a recorded event (check 2); no equipment-coded stretch was found in the probes.

Note 2026-11-07 01:45Z (committed 01:42:32Z as bced91633; the heading time was written ahead of the clock) (0.9.7.2 admission, wave 2, the final 0.9.7.2 A list and the A gate threshold; Amendment 15 item 5; before any model cell on the wave-2 datasets and before any 0.9.7.2 A cell is read). Measured with measure097.py, no model run (builders in data097/src/, record data097/admit0972.json from admit0972.py). In: RCFPOWER (Zenodo, CC BY 4.0): 84 Eval series, 103 segments, median share 0.010, elapsed best 0.513, all with a prefix of 512. Amendment 14's condition holds: only the physically induced categories are labelled (load, voltage, frequency, power quality, outage; sensor, context and compound categories excluded), 136 such events (109 without the gradual-drift and flicker types, whose induction protocol the datasheet does not state). UKWIND pooled with the later years of the same publisher (Amendment 14): Kelmarsh and Penmanshiel 2016 to 2022, 85 Eval series under the cap of 100 in hash order, 128 segments, median share 0.027, elapsed best 0.541; the 2016 rebuild reproduces all 21 stored series byte for byte. Kelmarsh 2023 and 2024 are left out because each turbine file holds 81 overlapping exports (2.17 million rows for 52,560 timestamps), which the unchanged 0.8 builder cannot read without a new de-duplication rule; this was decided from the file structure before any UKWIND measurement. Of the 17 Eval series of 0.9.7, 12 stay in Eval and 5 fall outside the cap. The inherited flag of 0.8 stays: availability and lost-production channels computed by the SCADA are inputs. REASSEMBLE (TU Wien, CC BY 4.0): 18 Eval series, 44 segments, median share 0.115, elapsed best 0.556. Its condition holds: the files hold only measured robot state (no command or haptic-master stream) and the failed actions are physical outcomes (196 of 1,957 action segments). Disclosed: the fetch was stopped at 00:57Z because the VM was overloaded, so REASSEMBLE is built from the recordings at zip positions 0 to 78 (one failed transfer), a label-blind subset; 74 recordings after the README known-issue drops give 18 Eval series. Fetching the rest (about 35 GB) would roughly double them; it is handed over. Out: PVLAZ (rule 2(a), 5 Eval series; elapsed-time gate 0.683). NOBOOM_M202210: 85 of its 89 runs are NOBOOM runs with the same inputs and a second label column (21 of its kept series are byte-identical to NOBOOM series, and the pooled list would put identical inputs in Eval twice and in Eval and Tuning), so it adds no new test material (checks 3 and 4); NOBOOM stays as in 0.9.7. Final 0.9.7.2 A list (13 datasets, 992 Eval series): ESA 95, NASA 37, UKWIND 85, HOT 50, PREDIST 41, NOBOOM 20, HAI 58, UAV-SEAD 101, UR5e 63, KUKA 255, UTAQ 85, RCFPOWER 84, REASSEMBLE 18. One flight set (UAV-SEAD) of 13. A gate threshold (Amendment 5 rule 5(b), recomputed over the final list): 0.5502 = mean over the 13 datasets of the highest data-free null (centre): ESA 0.6109, NASA 0.5162, UKWIND 0.5408, HOT 0.5293, PREDIST 0.5124, NOBOOM 0.5605, HAI 0.5328, UAV-SEAD 0.5851, UR5e 0.6156, KUKA 0.5194, UTAQ 0.5604, RCFPOWER 0.5130, REASSEMBLE 0.5560. No 0.9.7.2 A cell has been read (UTAQ's cells exist on Clariden and were only checked for completeness).

Comparison across versions since 0.9.4

0.9.4 (0.9.4.1)0.9.50.9.60.9.70.9.7.212345678910111213141516pj_better_10m ◊: 0.9.4 (0.9.4.1) rank 6, 0.9.5 rank 1, 0.9.6 rank 1, 0.9.7 rank 1, 0.9.7.2 rank 1pj_better_10m ◊Mantis: 0.9.4 (0.9.4.1) rank 4, 0.9.5 rank 3, 0.9.6 rank 3, 0.9.7 rank 4, 0.9.7.2 rank 2Mantispj_better_3p7m ◊: 0.9.4 (0.9.4.1) rank 7, 0.9.5 rank 2, 0.9.6 rank 2, 0.9.7 rank 2, 0.9.7.2 rank 3pj_better_3p7m ◊pjgood-warp-xview-combo-10M ◊: 0.9.4 (0.9.4.1) rank 5, 0.9.5 rank 4, 0.9.6 rank 4, 0.9.7 rank 3, 0.9.7.2 rank 4pjgood-warp-xview-combo-10M ◊Chronos-2: 0.9.4 (0.9.4.1) rank 12, 0.9.5 rank 5, 0.9.6 rank 5, 0.9.7 rank 5, 0.9.7.2 rank 5Chronos-2LeSaturn-warppool-factorized-7.4M ◊: 0.9.4 (0.9.4.1) rank 10, 0.9.5 rank 6, 0.9.6 rank 6, 0.9.7 rank 6, 0.9.7.2 rank 6LeSaturn-warppool-factorized-7.4M ◊pj_better_1m ◊: 0.9.4 (0.9.4.1) rank 9, 0.9.5 rank 7, 0.9.6 rank 7, 0.9.7 rank 7, 0.9.7.2 rank 7pj_better_1m ◊MantisV2: 0.9.4 (0.9.4.1) rank 1, 0.9.5 rank 8, 0.9.6 rank 8, 0.9.7 rank 8, 0.9.7.2 rank 8MantisV2NuTime: 0.9.4 (0.9.4.1) rank 14, 0.9.5 rank 9, 0.9.6 rank 9, 0.9.7 rank 9, 0.9.7.2 rank 9NuTimeTimestral-209M: 0.9.4 (0.9.4.1) rank 3, 0.9.5 rank 10, 0.9.6 rank 10, 0.9.7 rank 10, 0.9.7.2 rank 10Timestral-209MTimestral-123M: 0.9.4 (0.9.4.1) rank 2, 0.9.5 rank 11, 0.9.6 rank 11, 0.9.7 rank 11, 0.9.7.2 rank 11Timestral-123MTimestral-5M: 0.9.4 (0.9.4.1) rank 13, 0.9.5 rank 12, 0.9.6 rank 14, 0.9.7 rank 14, 0.9.7.2 rank 12Timestral-5MTimestral-28M: 0.9.4 (0.9.4.1) rank 11, 0.9.5 rank 13, 0.9.6 rank 13, 0.9.7 rank 13, 0.9.7.2 rank 13Timestral-28MToto-Open-Base-1.0: 0.9.4 (0.9.4.1) rank 15, 0.9.5 rank 14, 0.9.6 rank 12, 0.9.7 rank 12, 0.9.7.2 rank 14Toto-Open-Base-1.0TimesFM-2.5-200M: 0.9.4 (0.9.4.1) rank 16, 0.9.5 rank 15, 0.9.6 rank 15, 0.9.7 rank 15, 0.9.7.2 rank 15TimesFM-2.5-200MTiRex-2: 0.9.4 (0.9.4.1) rank 8TiRex-2 (not ranked later)rank on the CAFE score
Rank of each ranked model on the CAFE score in each version. A diamond is a ◊ row. Ranks inside tier 1 are no claim; the tables give the tiers.

The versions differ in datasets, heads and, for 0.9.4, in the ranked rows (TiRex-2 is ranked in 0.9.4 only), so scores of different versions are means over different material; ranks, tiers and resolved pairs are what can be compared. 0.9.4 is the released 0.9.4 board, unchanged in 0.9.4.1 (board094_frozen.json).

Datasets and pairs resolved. Main board: datasets fixed, test material resampled. Datasets resampled: the measure of the 1.0 test.

versionC datasetsA datasetsFE datasetsranked modelstasks in CAFECAFE pairs resolved, mainofCAFE pairs resolved, datasets resampledof
0.9.4 (0.9.4.1)1671616C, A, FE39 (1 within seed noise)12016120
0.9.517121615C, A, FE73 (1 within seed noise)10549105
0.9.625141715C, A, FE73 (1 within seed noise)10551105
0.9.724101515C, A, FE85 (10 within seed noise)10561105
0.9.7.224131515C, A, FE90 (15 within seed noise)10562105

Per task. Pairs resolved on the task board, main / datasets resampled, and signal-to-noise (mean absolute gap between two ranked rows over the median bootstrap SD of that gap; separability096.py). A dash: no such file for that version.

versionC pairsA pairsFE pairsC signal-to-noiseA signal-to-noiseFE signal-to-noiseCAFE signal-to-noise
0.9.4 (0.9.4.1)73 / –24 / –69 / –––––
0.9.570 / 1518 / 487 / 678.392.4815.107.32
0.9.674 / 2137 / 687 / 668.983.0014.937.89
0.9.780 / 2560 / 987 / 6710.064.0416.2610.88
0.9.7.280 / 2563 / 1887 / 6710.064.8116.2612.20

No separability file exists for 0.9.4.

First place. The first row of each board; inside tier 1 the order is no claim.

versionCAFECAFEmodels in CAFE tier 1
0.9.4 (0.9.4.1)MantisV2Timestral-209MTimestral-123Mpj_better_1m ◊13
0.9.5pj_better_10m ◊Chronos-2pj_better_3p7m ◊Mantis6
0.9.6pj_better_10m ◊Chronos-2pj_better_3p7m ◊Mantis5
0.9.7pj_better_10m ◊Chronos-2pj_better_10m ◊Mantis1
0.9.7.2pj_better_10m ◊Chronos-2MantisMantis2

Rank and tier of each ranked model on the CAFE score, rank (tier):

model0.9.4 (0.9.4.1)0.9.50.9.60.9.70.9.7.2
pj_better_10m ◊6 (1)1 (1)1 (1)1 (1)1 (1)
Mantis4 (1)3 (1)3 (1)4 (2)2 (1)
pj_better_3p7m ◊7 (1)2 (1)2 (1)2 (2)3 (2)
pjgood-warp-xview-combo-10M ◊5 (1)4 (1)4 (1)3 (2)4 (2)
Chronos-212 (1)5 (1)5 (1)5 (2)5 (2)
LeSaturn-warppool-factorized-7.4M ◊10 (1)6 (1)6 (2)6 (3)6 (3)
pj_better_1m ◊9 (1)7 (2)7 (2)7 (3)7 (4)
MantisV21 (1)8 (2)8 (2)8 (4)8 (4)
NuTime14 (2)9 (3)9 (3)9 (5)9 (5)
Timestral-209M3 (1)10 (3)10 (3)10 (5)10 (6)
Timestral-123M2 (1)11 (3)11 (3)11 (6)11 (6)
Timestral-5M13 (1)12 (3)14 (3)14 (6)12 (7)
Timestral-28M11 (1)13 (3)13 (3)13 (6)13 (7)
Toto-Open-Base-1.015 (3)14 (3)12 (3)12 (6)14 (7)
TimesFM-2.5-200M16 (3)15 (4)15 (4)15 (7)15 (8)
TiRex-28 (1)––––

What changed, by the amendments of each version and the datasets added (+) or removed (-) against the version before:

versionrulesdatasets
0.9.4 (0.9.4.1)Amendment 5first version compared
0.9.5Amendments 6, 7, 8, 9 and 10C: +2 (aursad095, voraus095) -1 (voraus); A: +5 (ESA_M1, CARE_A, MSL, W3, SKAB)
0.9.6Amendments 11 and 12C: +8 (chillan5, haptic_texture8, openrainer_m, poseit_ft, px4_airframe3, soredd_abc, swiss_load10, ved_powertrain3); A: +2 (PREDIST, NOBOOM); FE: +1 (ngafid_hastall)
0.9.7Amendment 13C: +2 (bdg2_meter6, robo6_rb6) -3 (3w, bosch, soredd_abc); A: +5 (ESA, NASA, UAVSEAD, UR5E, KUKA) -9 (ESA_M1, ESA_M2, CARE_A, SMAP, MSL, CARE_C, RIGS, W3, SKAB); FE: +1 (heatpump_defrost) -3 (epa_aqs_pm25, substorm_omni, ngafid_hastall)
0.9.7.2Amendment 15A: +3 (UTAQ, RCFPOWER, REASSEMBLE)

Each version's own changes are in the change log.

0.9.6 and 0.9.7.2 side by side

Scores of the two versions are means over different datasets, so a change in score is not a change in the model; ranks and tiers are what can be compared.

Ranked rows of both versions, in the 0.9.7.2 order.

modelrank 0.9.6rank 0.9.7.2movetier 0.9.6tier 0.9.7.2score 0.9.6score 0.9.7.2change
pj_better_10m ◊11=110.73630.7405+0.0042
Mantis32↑1110.73290.7385+0.0056
pj_better_3p7m ◊23↓1120.73470.7367+0.0020
pjgood-warp-xview-combo-10M ◊44=120.73260.7363+0.0036
Chronos-255=120.73010.7351+0.0050
LeSaturn-warppool-factorized-7.4M ◊66=230.72700.7313+0.0043
pj_better_1m ◊77=240.72240.7264+0.0040
MantisV288=240.71920.7230+0.0038
NuTime99=350.70670.7098+0.0030
Timestral-209M1010=360.70040.7033+0.0029
Timestral-123M1111=360.69900.7013+0.0023
Timestral-5M1412↑2370.69650.6996+0.0032
Timestral-28M1313=370.69650.6987+0.0022
Toto-Open-Base-1.01214↓2370.69720.6948−0.0024
TimesFM-2.5-200M1515=480.66570.6703+0.0046

Separability by task, dataset and model

Computed by separability096.py on the 0.9.7.2 board, over the 15 ranked rows only. Spread SD: SD of the rows' scores. Pair noise: median bootstrap SD of the paired difference of two rows. Signal-to-noise: mean absolute gap between two rows over the pair noise. Resolved alone: pairs resolved on a board of that dataset alone. Tau with the task: Kendall tau between a dataset's order and the mean of the task's other datasets. Board effect: CAFE pairs resolved without the dataset minus with it (negative: the dataset helps).

By task.

taskdatasetsspread SDrangepair noisesignal-to-noisein 0.9.6pairs resolveddatasets resampledtau with the other tasksshare of CAFE varianceCAFE pairs without itsame, resampled
C240.01840.07160.001810.068.9880250.1810.156-2-1
A130.02080.07160.00464.813.0063180.6190.274+1-2
FE150.04380.12290.003016.2614.9387670.3900.569-27-34
CAFE520.02110.07020.002012.207.899062––––

By dataset (● new in 0.9.7 or 0.9.7.2; ◐ regrown in 0.9.7.2: UKWIND; ○ pooled from 0.9.6 datasets whose scores were published), highest signal-to-noise first within each task.

All 52 datasets
taskdatasetspread SDrangepair noisesignal-to-noisepairs resolved alonetau with the tasktask pairs without itCAFE pairs without it, resampled
Cphys/lendb0.11520.39180.004624.45970.429-7+1
Cnicola15/bdg2_meter6 ●0.02790.09520.00378.53890.219+0+1
Cnicola15/robo6_rb6 ●0.02390.07410.00387.40810.295+1+0
Cmv06/stead0.01690.06420.00277.03800.371-4+0
Cmv06/hydraulic0.09590.36500.01256.43620.314-5+2
Cnicola15/ved_powertrain30.01910.06250.00336.16730.524-3+1
Cnicola15/haptic_texture80.01610.06770.00315.96750.162+0+1
Cucr/ElectricDevices0.01260.04060.00314.74630.486-1+0
Cucr/SonyAIBORobotSurface20.05920.24050.01484.20470.019-3+0
Cnicola15/openrainer_m0.01480.04720.00424.12610.105+1+1
Cucr/FreezerRegularTrain0.01170.04560.00304.07460.452+0-1
Cmv06/voraus0950.03440.10550.00953.97540.200-2+0
Cnicola15/chillan50.00800.03180.00253.60540.600+0+0
Cmv06/aursad0950.02010.06240.00663.59570.200-1+0
Cnicola15/px4_airframe30.02110.06330.00683.50600.295-5+0
Cucr/MoteStrain0.02810.10790.00883.41380.410-2+1
Cucr/Wafer0.01080.03830.00333.33330.421-1-1
Cuea/AsphaltPavementTypeCoordinates0.02330.08970.00843.16470.143+0-1
Cucr/PLAID0.03830.15720.01412.94350.238-1-3
Cnicola15/swiss_load100.01340.04730.00522.91510.077-1+1
Cucr/FordA0.02070.07470.00822.89310.524-5-2
Cnicola15/poseit_ft0.01600.05730.00652.84500.562-3+1
Cmv06/gas2510.01550.05480.00612.73390.257+1-1
Cucr/ACSF10.03940.12220.02371.94150.010+1-1
AKUKA ●0.01990.06400.00386.08810.410-4+0
AUKWIND ◐0.04400.19680.00983.69340.067+2+1
AHAI0.03680.12070.01253.46580.600-9-1
AUR5E ●0.02930.11960.00873.44390.048+3+0
AESA ○0.02870.07440.01013.22420.486-4+0
ARCFPOWER ●0.02950.10340.01192.87390.486-3-1
AHOT0.03440.15070.01272.85380.524-3+2
AUTAQ ●0.02500.08890.00972.40260.676+0+1
AUAVSEAD ●0.01700.05890.00912.09150.086-3+0
APREDIST0.03230.11980.01762.04190.524-3+1
ANOBOOM0.04010.12930.02631.76250.562-5+0
ANASA ○0.03280.12550.02451.48110.162+5+1
AREASSEMBLE ●0.02890.10680.02361.35100.124+8+2
FEmegathrust_sync_v20.06460.19250.005912.37790.486+1-2
FErotary_utrecht0.03710.10580.004110.18750.733-1-1
FEminati_circuits0.05770.17220.00798.37790.657+0+0
FEmemristor_jerk0.06460.25320.00788.30720.733+0-1
FEasos_lows_b0.06110.15310.00897.77660.733+1-3
FEstickslip_poincloux0.04150.14430.00716.88810.638+0+1
FEndbc_buoy_v2_b0.04800.12850.00985.72710.810-1-2
FEgeyser_strokkur0.03720.10790.00785.66730.676+0-1
FEcdip_waves_c0.06000.15920.01325.35740.848-2-1
FEnoaa_coops_surge_b0.06480.16640.01554.83560.829+0-3
FEheatpump_defrost ●0.03610.10180.01183.60540.638+1+0
FEnasa_battery_rw0.02540.08660.00833.41440.410+1-1
FEusgs_reach_v2_b0.04190.10520.01543.10400.600+1-1
FEwind_genbearing_hot0.02760.07690.01053.08350.676-1+0
FEtowie_genbearing_hot0.04720.12690.01952.82390.600+3-2

By model. Pairs resolved: pairs on the CAFE main board in which the model is resolved above or below the other. Datasets first: datasets where it has the highest score among the ranked rows.

modelranktierpairs resolvedtier, datasets resampleddatasets firstmedian rank over datasets
pj_better_10m ◊1113 of 14144
Mantis2110 of 14166
pj_better_3p7m ◊3211 of 14135
pjgood-warp-xview-combo-10M ◊4210 of 14155
Chronos-25211 of 14157
LeSaturn-warppool-factorized-7.4M ◊6313 of 14118
pj_better_1m ◊7413 of 14228
MantisV28413 of 14248
NuTime9514 of 14259
Timestral-209M10613 of 14329
Timestral-123M11611 of 143310
Timestral-5M12711 of 143011
Timestral-28M13711 of 143310
Toto-Open-Base-1.014712 of 143914
TimesFM-2.5-200M15814 of 144014

Separability by dataset

Every dataset's effect on the board

Each row is the 0.9.7.2 board rebuilt without one dataset, every other rule unchanged (52 of 52 datasets). Tau: Kendall tau of its order with the main order. Main board: pj_better_10m ◊ first, 2 models in tier 1, 90 pairs resolved (15 within seed noise), 62 with the datasets resampled.

taskdatasetfirstits CAFEtier 1tier 1 changepairs resolveddatasets resampledtau
Cucr/MoteStrainpj_better_10m ◊0.73912same87631.000
Cucr/SonyAIBORobotSurface2pj_better_10m ◊0.74022same90620.981
Cucr/Waferpj_better_10m ◊0.73832same90611.000
Cucr/ACSF1pj_better_10m ◊0.74112same90611.000
Cucr/PLAIDpj_better_10m ◊0.73982same89591.000
Cucr/FreezerRegularTrainpj_better_10m ◊0.73842same88611.000
Cucr/FordApj_better_10m ◊0.73942same90601.000
Cucr/ElectricDevicespj_better_10m ◊0.74122same89621.000
Cphys/lendbpj_better_10m ◊0.73972same88630.981
Cmv06/hydraulicpj_better_10m ◊0.73882same85640.981
Cmv06/gas251pj_better_10m ◊0.73992same89611.000
Cmv06/steadpj_better_10m ◊0.73892same91621.000
Cuea/AsphaltPavementTypeCoordinatespj_better_10m ◊0.73902same89611.000
Cmv06/aursad095pj_better_10m ◊0.74262same90621.000
Cmv06/voraus095pj_better_10m ◊0.74192same86621.000
Cnicola15/chillan5pj_better_10m ◊0.73982same90621.000
Cnicola15/haptic_texture8pj_better_10m ◊0.74432same88631.000
Cnicola15/openrainer_mpj_better_10m ◊0.73992same92631.000
Cnicola15/poseit_ftpj_better_10m ◊0.74372same90631.000
Cnicola15/px4_airframe3pj_better_10m ◊0.74142same90621.000
Cnicola15/swiss_load10pj_better_10m ◊0.74352same91631.000
Cnicola15/ved_powertrain3pj_better_10m ◊0.73942same89631.000
Cnicola15/bdg2_meter6 ●pj_better_10m ◊0.74132same85631.000
Cnicola15/robo6_rb6 ●pj_better_10m ◊0.74142same86621.000
AHAIpj_better_10m ◊0.74162same84611.000
AUKWIND ◐pj_better_10m ◊0.73692same82631.000
AHOTpj_better_10m ◊0.74162same84641.000
APREDISTpj_better_10m ◊0.73962same84631.000
ANOBOOMpj_better_10m ◊0.73662same89621.000
AESA ○pj_better_10m ◊0.73982same89620.981
ANASA ○Mantis0.74025+Chronos-2, +pj_better_3p7m ◊ and +pjgood-warp-xview-combo-10M ◊89630.981
AUAVSEAD ●pj_better_10m ◊0.74202same89621.000
AUR5E ●pj_better_10m ◊0.74232same86621.000
AKUKA ●pj_better_10m ◊0.74292same90621.000
AUTAQ ●pj_better_10m ◊0.74102same87631.000
ARCFPOWER ●pj_better_10m ◊0.73992same85610.962
AREASSEMBLE ●pj_better_10m ◊0.74262same89641.000
FEstickslip_poinclouxpj_better_10m ◊0.74122same88631.000
FEminati_circuitspj_better_10m ◊0.73892same89621.000
FEgeyser_strokkurpj_better_10m ◊0.74112same86611.000
FEndbc_buoy_v2_bpj_better_10m ◊0.74222same90601.000
FEnasa_battery_rwpj_better_10m ◊0.73722same84611.000
FEwind_genbearing_hotpj_better_10m ◊0.74262same86621.000
FEtowie_genbearing_hotpj_better_10m ◊0.74082same87601.000
FEmemristor_jerkpj_better_10m ◊0.73622same88611.000
FEnoaa_coops_surge_bpj_better_10m ◊0.74062same90591.000
FEasos_lows_bpj_better_10m ◊0.74142same88591.000
FEusgs_reach_v2_bpj_better_10m ◊0.74112same86611.000
FErotary_utrechtpj_better_10m ◊0.74222same87611.000
FEmegathrust_sync_v2pj_better_10m ◊0.74032same87601.000
FEcdip_waves_cpj_better_10m ◊0.74132same89611.000
FEheatpump_defrost ●pj_better_10m ◊0.74102same84620.981
Rank of every ranked model on the board without each dataset (main rank in the header)
withoutpj_better_10m ◊ (1)Mantis (2)pj_better_3p7m ◊ (3)pjgood-warp-xview-combo-10M ◊ (4)Chronos-2 (5)LeSaturn-warppool-factorized-7.4M ◊ (6)pj_better_1m ◊ (7)MantisV2 (8)NuTime (9)Timestral-209M (10)Timestral-123M (11)Timestral-5M (12)Timestral-28M (13)Toto-Open-Base-1.0 (14)TimesFM-2.5-200M (15)
C:MoteStrain123456789101112131415
C:SonyAIBORobotSurface2123456789101113121415
C:Wafer123456789101112131415
C:ACSF1123456789101112131415
C:PLAID123456789101112131415
C:FreezerRegularTrain123456789101112131415
C:FordA123456789101112131415
C:ElectricDevices123456789101112131415
C:lendb123456789101112141315
C:hydraulic123456789101112141315
C:gas251123456789101112131415
C:stead123456789101112131415
C:AsphaltPavementTypeCoordinates123456789101112131415
C:aursad095123456789101112131415
C:voraus095123456789101112131415
C:chillan5123456789101112131415
C:haptic_texture8123456789101112131415
C:openrainer_m123456789101112131415
C:poseit_ft123456789101112131415
C:px4_airframe3123456789101112131415
C:swiss_load10123456789101112131415
C:ved_powertrain3123456789101112131415
C:bdg2_meter6123456789101112131415
C:robo6_rb6123456789101112131415
A:HAI123456789101112131415
A:UKWIND123456789101112131415
A:HOT123456789101112131415
A:PREDIST123456789101112131415
A:NOBOOM123456789101112131415
A:ESA123546789101112131415
A:NASA213456789101112131415
A:UAVSEAD123456789101112131415
A:UR5E123456789101112131415
A:KUKA123456789101112131415
A:UTAQ123456789101112131415
A:RCFPOWER124356789101113121415
A:REASSEMBLE123456789101112131415
FE:stickslip_poincloux123456789101112131415
FE:minati_circuits123456789101112131415
FE:geyser_strokkur123456789101112131415
FE:ndbc_buoy_v2_b123456789101112131415
FE:nasa_battery_rw123456789101112131415
FE:wind_genbearing_hot123456789101112131415
FE:towie_genbearing_hot123456789101112131415
FE:memristor_jerk123456789101112131415
FE:noaa_coops_surge_b123456789101112131415
FE:asos_lows_b123456789101112131415
FE:usgs_reach_v2_b123456789101112131415
FE:rotary_utrecht123456789101112131415
FE:megathrust_sync_v2123456789101112131415
FE:cdip_waves_c123456789101112131415
FE:heatpump_defrost123546789101112131415

Admission of 0.9.7.2

A. Every candidate was measured without a model before any cell on it (Amendment 13 item 8): Eval series, anomaly segments in them, median share of the scored part that is labelled, and the better of the rising and falling elapsed-time scores. Rules: (a) at least 15 Eval series, (b) at least 20 segments, (c) median share at most 0.3; new datasets also need an elapsed-time best of at most 0.65.

candidatestatuscapEval seriessegmentsmedian shareelapsed-time bestrulesdecided (UTC)reason
ESAinexisting10095970.0320.611a:yes b:yes c:yes gate:yes–items 2(a)-(c) hold: 95 Eval, 97 segments, median share 0.032; elapsed-time best 0.611 (gate applies to new datasets only)
HAIinexisting100581030.0300.533a:yes b:yes c:yes gate:yes2026-11-05T19:21Zitems 2(a)-(c) hold: 58 Eval, 103 segments, median share 0.030; elapsed-time best 0.533 (gate applies to new datasets only)
HOTinexisting100501070.0140.529a:yes b:yes c:yes gate:yes–items 2(a)-(c) hold: 50 Eval, 107 segments, median share 0.014; elapsed-time best 0.529 (gate applies to new datasets only)
KUKAinnew3002555340.1080.519a:yes b:yes c:yes gate:yes2026-11-05T19:23Zitems 2(a)-(c) hold: 255 Eval, 534 segments, median share 0.108; elapsed-time best 0.519 (gate 0.65 holds)
NASAinexisting10037470.0400.516a:yes b:yes c:yes gate:yes–items 2(a)-(c) hold: 37 Eval, 47 segments, median share 0.040; elapsed-time best 0.516 (gate applies to new datasets only)
NOBOOMinexisting30020370.1910.561a:yes b:yes c:yes gate:yes–items 2(a)-(c) hold: 20 Eval, 37 segments, median share 0.191; elapsed-time best 0.561 (gate applies to new datasets only)
PREDISTinexisting10041410.0510.512a:yes b:yes c:yes gate:yes–items 2(a)-(c) hold: 41 Eval, 41 segments, median share 0.051; elapsed-time best 0.512 (gate applies to new datasets only)
RCFPOWERinnew100841030.0100.513a:yes b:yes c:yes gate:yesPREREG09 note 2026-11-07 01:45Z (wave 2)items 2(a)-(c) hold: 84 Eval, 103 segments, median share 0.010; elapsed-time best 0.513 (gate 0.65)
REASSEMBLEinnew30018440.1150.556a:yes b:yes c:yes gate:yesPREREG09 note 2026-11-07 01:45Z (wave 2)items 2(a)-(c) hold: 18 Eval, 44 segments, median share 0.115; elapsed-time best 0.556 (gate 0.65)
UAVSEADinnew3001011470.0830.585a:yes b:yes c:yes gate:yes2026-11-05T18:56Zitems 2(a)-(c) hold: 101 Eval, 147 segments, median share 0.083; elapsed-time best 0.585 (gate 0.65 holds)
UKWINDinexisting100851280.0270.541a:yes b:yes c:yes gate:yesPREREG09 note 2026-11-07 01:45Z (wave 2)items 2(a)-(c) hold: 85 Eval, 128 segments, median share 0.027; elapsed-time best 0.541 (gate applies to new datasets only)
UR5Einnew30063740.1910.616a:yes b:yes c:yes gate:yes2026-11-05T18:54Zitems 2(a)-(c) hold: 63 Eval, 74 segments, median share 0.191; elapsed-time best 0.616 (gate 0.65 holds)
UTAQinnew100854040.0270.560a:yes b:yes c:yes gate:yesPREREG09 note 2026-11-06 23:54Z (wave 1)items 2(a)-(c) hold: 85 Eval, 404 segments, median share 0.027; elapsed-time best 0.560 (gate 0.65)
BASICoutnew300––––a:– b:– c:– gate:–2026-11-05T19:12Zcheck 1 / item 4 condition: the flights are software-in-the-loop, not real (Ahmad and Akram, Data in Brief 2024, doi:10.1016/j.dib.2024.110069: 'The BASiC dataset is exclusively derived from simulations conducted in a SITL environment using the ArduPilot platform'); not built
CAFUC2outnew300962860.0210.508a:yes b:yes c:yes gate:yes2026-11-05T18:56Zcheck 2 (physical label; PREREG09 note of 2026-10-31 ~20:50Z: injected or staged faults pass when the onset is observed in the system): CAFUC2's faults are synthetic signals written into five copies of each recorded flight after the flight; no aircraft experienced them, so the label is not a state of or an event in the physical system. Decided before any cell; only Jonas Petersen can reverse this, in a dated note
CRANFIELDoutnew–––––a:– b:– c:– gate:–2026-11-05T19:23Zthe official download (MATLAB File Exchange #50938) needs a MathWorks sign-in; the other copies (Kaggle, IEEE DataPort) need a login; item 4 / Amendment 13: sets that need a registration are not added
EAWAGoutnew–––––a:– b:– c:– gate:–2026-11-05T18:55Zonly 4 ponds (2D, 3D, 5A, 5B), phase 1, carry the expert 'Anomaly' column, with 5 segments in total: rules 2(a) and 2(b) cannot hold; the licence field is empty (portal: public domain for data outside Swiss copyright)
GREEKWINDoutnew100780.0120.501a:no b:no c:yes gate:yes2026-11-05T19:23Z2(a): 7 Eval series < 15; 2(b): 8 anomaly segments < 20
LEAD1.0outnew–––––a:– b:– c:– gate:–PREREG09 note 2026-11-07 00:10Zcheck 4: its meters are the BDG2 electricity meters of the ASHRAE GEPIII subset, the campaign of the C unit nicola15/bdg2_meter6; no licence stated (data behind a Kaggle login)
MASToutnew–––––a:– b:– c:– gate:–2026-11-05T18:55Zno precursor-only cut without an elapsed-time leak: a scored block that ends before the current quench carries the precursor label at its end (a rising elapsed-time score reaches 0.933 on synthetic series of that shape); TokaMark ships no disruption labels or quench times
NOBOOM_M202210outnew–––––a:– b:– c:– gate:–PREREG09 note 2026-11-07 01:45Z (wave 2)check 3/4: 85 of its 89 runs are NOBOOM runs with the same X and a second label column (21 kept series byte-identical to NOBOOM series; pooled, 2 Eval pairs and 3 Eval-Tuning pairs share X); NOBOOM stays as in 0.9.7
ONBOARDEUoutnew10085940.1730.708a:yes b:yes c:yes gate:no2026-11-05T19:19Zitem 4 elapsed-time gate: 0.708 > 0.65
PMUoutnew300––––a:– b:– c:– gate:–2026-11-05T19:22Zthe continuous 1 s TransnetBW record (Zenodo 15784548) is one channel (CE frequency) and ends 2024-04-25, covering 23 of 88 cases; P4 needs >= 2 channels and 23 cases give at most 19 Eval series and 19 segments (rule 2(b)); not built
PVLAZoutnew–––––a:– b:– c:– gate:–PREREG09 note 2026-11-07 01:45Z (wave 2)rule 2(a): 5 Eval series < 15; item 4 elapsed-time gate 0.683 > 0.65
RFLYMADoutnew3002554050.2890.731a:yes b:yes c:yes gate:no2026-11-05T19:12Zitem 4 elapsed-time gate: 0.731 > 0.65
TCABRoutnew3001381380.1991.000a:yes b:yes c:yes gate:no2026-11-05T19:22Zitem 4 elapsed-time gate: 1.000 > 0.65
TSB-ADoutnew–––––a:– b:– c:– gate:–PREREG09 note 2026-11-07 00:02Zno source collection passes item 2 and checks 1-4 (simulated or injected events, IT/web/finance metrics, recordings of people, imputed or undocumented labels; work/r097/tsbad_screen0972.md); SMAP and MSL stay in NASA, SWaT needs an access form
USGS NWISoutnew–––––a:– b:– c:– gate:–PREREG09 note 2026-11-07 00:10Zcheck 2: on ice-affected stretches the published discharge is an estimate written in place of the record, so the label marks an estimate or a gap, not a recorded event; no equipment-coded stretch found
Admission waves, as written before any 0.9.7 or 0.9.7.2 cell on their units
wavewritten (UTC)inoutpending
02026-11-05T18:46ZHOTnonenone
12026-11-05T18:56ZESA, NASA, UKWIND, HOT, PREDIST, NOBOOM, UAVSEAD and UR5EEAWAG, CAFUC2 and MASTHAI, RFLYMAD, PMU, ONBOARDEU, TCABR, BASIC, KUKA, GREEKWIND and CRANFIELD
C12026-11-05T19:10Znicola15/bdg2_meter6 and nicola15/robo6_rb6nonenicola15/sns_hvcm_b_end
22026-11-05T19:12ZnoneRFLYMAD and BASICnone
C22026-11-05T19:13Znonenicola15/sns_hvcm_b_endnone
32026-11-05T19:21ZHAIONBOARDEUnone
42026-11-05T19:23ZKUKATCABR, PMU, GREEKWIND and CRANFIELDnone
FE12026-11-05T19:44Zheatpump_defrostnonenone
FE22026-11-05T21:59Zipshocks_fluxespyrnetnone
0.9.7.2 wave 12026-11-06 23:54Z (commit dcf9de80a)UTAQnoneRCFPOWER, UKWIND, REASSEMBLE, PVLAZ and NOBOOM_M202210
0.9.7.2 screen2026-11-07 00:02Z and 00:10Z (commits a1f71c5f6, 4683f1fec)noneTSB-AD (every source collection), LEAD1.0 and USGS NWISnone
0.9.7.2 wave 22026-11-07 01:45Z (commit bced91633)RCFPOWER, UKWIND (regrown) and REASSEMBLEPVLAZ and NOBOOM_M202210none

FE. Model-free gates (Amendment 13 item 5; thresholds computed 2026-11-05T18:53:31Z on the 14 kept FE datasets, before any new FE download): at least 300 events, at least 30 independent units or days, time-since-last-event row at most 0.7956, time-of-day row at most 0.5851, best free forecast at most 0.85.

The reference rows on the kept FE datasets (the source of the thresholds)
datasettime since last eventtime of dayposition onlyevent ratebest free forecast
stickslip_poincloux0.685no calendar0.4980.595early-warning statistics 0.701
minati_circuits0.569no calendar0.5070.501curvature 0.757
geyser_strokkur0.7340.5040.4900.611early-warning statistics 0.730
ndbc_buoy_v2_b0.5490.4760.5060.463current value 0.688
nasa_battery_rw0.7960.5120.5170.676current value 0.873
wind_genbearing_hot0.6510.5290.5340.594current value 0.704
towie_genbearing_hot0.5380.5850.5660.504level statistics 0.752
memristor_jerk0.635no calendar0.5100.505curvature 0.882
noaa_coops_surge_b0.6540.5030.4950.533current value 0.797
asos_lows_b0.5080.4920.4910.436trend 0.743
usgs_reach_v2_b0.5050.4960.4810.486early-warning statistics 0.652
rotary_utrecht0.582no calendar0.5030.520level statistics 0.647
megathrust_sync_v20.759no calendar0.4960.562level statistics 0.674
cdip_waves_c0.5660.5200.5150.494current value 0.738
candidatestatuseventsunits or daystime since last eventtime of daybest free forecastreason
heatpump_defrostin53,9781130.6590.5660.729item 5 gates hold: 53,978 events >= 300; 113 systems >= 30; time since last event 0.659 <= 0.796; time of day 0.566 <= 0.585; best free forecast 0.729 <= 0.85
ngafid_hastallout–––––Amendment 13 item 5
substorm_omniout–––––Amendment 13 item 5
epa_aqs_pm25out–––––Amendment 13 item 5
ipshocks_fluxesadmitted by item 5; off the board by check 3 (FE at least 1,000 events)9146290.4900.5130.695item 5 gates hold: 914 events; 629 distinct shock days (both spacecraft at L1, counted by day); time since last event 0.490; time of day 0.513; best free forecast 0.695
pyrnetout–––––not built and gated by the 21:30Z cutoff (the 4.8 GB download finished 21:58Z); builder written, never run

ipshocks_fluxes passed the item 5 gates (at least 300 events) but has fewer than the 1,000 events of the standing admission check 3, which Amendment 13 item 7 keeps; the board leaves it out, so 0.9.7 has 15 FE datasets. Neither text said whether the 300-event gate replaces check 3 for new FE datasets; Jonas Petersen decided that ipshocks_fluxes stays out of 0.9.7, with the board that includes it known (PREREG09 note of 2026-11-06 11:05Z; see the sensitivities of the test above). Its cells exist and are kept.

C. New C datasets were rebuilt from their builders and accepted only if the sha256 of X equals the logged one (Amendment 13 item 6), after the exp-nicola-15 screening that scored a ◊ row on them (see What changed).

datasetstatuskindX sha256 equalclasseswindowsgroupslicencereason
nicola15/bdg2_meter6innew (item 6)yes675,4091,615CC BY-SA 4.0 (github.com/buds-lab/building-data-genome-project-2 LICENSE)rebuilt from the exp-nicola-15 builder; X sha256 equals LOG.md (item 6)
nicola15/robo6_rb6innew (item 6)yes66,8695,352MIT (Hugging Face dataset card kyselica/RoBo6)rebuilt from the exp-nicola-15 builder; X sha256 equals LOG.md (item 6)
nicola15/sns_hvcm_b_endoutnew (item 6)yes23,32315CC BY 4.0 (Mendeley Data kbbrw99vh8 v5)item 6 gate: combo-10M minus random twin positive in 3 of 5 repeats (needs 4) and mean +0.0169 (needs +0.02); results/cgate097.json
mv06/3woutdropped (item 6)–––––Amendment 13 item 6
mv06/boschoutdropped (item 6)–––––Amendment 13 item 6
nicola15/soredd_abcoutdropped (item 6)–––––Amendment 13 item 6

Combo minus twin on every C dataset (Amendment 13 item 6): pjgood-warp-xview-combo-10M minus the random-init LeSaturn row (3.7 M parameters, another architecture; not a matched twin of the 10.6 M combo), per repeat, under the C head; the criterion is positive in at least 4 of 5 repeats and a mean of at least +0.02. It decided only the new SNS HVCM unit; a kept dataset that fails it is reported here and not dropped.

C datasetpositive repeatsofmean differenceper repeatcriterion
nicola15/sns_hvcm_b_end (candidate)35+0.0169-0.019 +0.016 +0.047 -0.015 +0.055fails
ucr/MoteStrain11+0.0279+0.028n/a: one fixed split
ucr/SonyAIBORobotSurface211+0.2475+0.247n/a: one fixed split
ucr/Wafer11+0.0209+0.021n/a: one fixed split
ucr/ACSF101-0.0000-0.000n/a: one fixed split
ucr/PLAID11+0.1064+0.106n/a: one fixed split
ucr/FreezerRegularTrain11+0.1232+0.123n/a: one fixed split
ucr/FordA11+0.0229+0.023n/a: one fixed split
ucr/ElectricDevices11+0.0263+0.026n/a: one fixed split
phys/lendb55+0.1144+0.103 +0.131 +0.122 +0.102 +0.114passes
mv06/hydraulic55+0.0706+0.077 +0.074 +0.096 +0.039 +0.066passes
mv06/gas25155+0.0979+0.107 +0.152 +0.099 +0.069 +0.062passes
mv06/stead55+0.0614+0.077 +0.072 +0.056 +0.053 +0.049passes
uea/AsphaltPavementTypeCoordinates11+0.0797+0.080n/a: one fixed split
mv06/aursad09555+0.0850+0.078 +0.098 +0.093 +0.070 +0.086passes
mv06/voraus09555+0.0839+0.069 +0.080 +0.110 +0.072 +0.089passes
nicola15/chillan555+0.0605+0.056 +0.051 +0.076 +0.051 +0.069passes
nicola15/haptic_texture855+0.0545+0.054 +0.044 +0.069 +0.048 +0.057passes
nicola15/openrainer_m55+0.0424+0.058 +0.031 +0.025 +0.039 +0.059passes
nicola15/poseit_ft55+0.0869+0.164 +0.081 +0.044 +0.051 +0.094passes
nicola15/px4_airframe355+0.0761+0.087 +0.067 +0.087 +0.050 +0.090passes
nicola15/swiss_load1055+0.0476+0.069 +0.047 +0.045 +0.043 +0.034passes
nicola15/ved_powertrain355+0.0736+0.077 +0.072 +0.074 +0.055 +0.089passes
nicola15/bdg2_meter655+0.0329+0.050 +0.018 +0.040 +0.029 +0.027passes
nicola15/robo6_rb655+0.0850+0.079 +0.084 +0.085 +0.083 +0.094passes

What took how long

From the run's event log (timing097.json, UTC): the first event is at 2026-11-05T18:33:41Z and the last at 2026-11-07T01:40:36Z, 31.12 hours apart. Builds and measurements ran in parallel, so the spans below overlap and do not add up to the run.

Phases: 4, 835.3 minutes in total, the longest phase2 (615.0 min).

whatstartendminutesnote
phase011-05 18:33:4111-05 18:428.3setup, probes, dashboard, runner; ended when builders were launched (~18:42Z); logged late
phase111-05 18:4211-05 21:59197.0final FE list (PREREG09 note 21:59Z); logged after the outage
phase211-05 18:4911-06 05:04615.0last cell job 3868626 ended (unattended, VM offline); tabular references queued separately 2026-11-06
phase311-06 09:1711-06 09:3215.0
Downloads: 28, 1979.7 minutes in total, the longest REASSEMBLE (1571.7 min).
whatstartendminutesnote
UAVSEAD11-05 18:38:5811-05 18:40:311.6
UR5E11-05 18:39:0011-05 18:46:247.4
BASIC11-05 18:39:0811-05 18:40:030.9
TCABR11-05 18:39:3311-05 18:55:3015.9
HAI11-05 18:40:1311-05 18:42:482.6
KUKA11-05 18:40:1311-05 18:59:1919.1
RFLYMAD11-05 18:40:2511-05 18:57:4217.3
EAWAG11-05 18:40:2611-05 18:49:389.2
ONBOARDEU11-05 18:41:0411-05 18:54:2513.3
PMU11-05 18:41:1411-05 18:45:053.9
CAFUC211-05 18:41:2911-05 18:42:120.7
CRANFIELD11-05 18:41:4811-05 18:46:104.4
GREEKWIND11-05 18:41:4811-05 18:46:104.4
PREDIST11-05 18:42:3111-05 18:44:261.9
C_robo611-05 18:42:3211-05 19:04:3222.0fetch_inside_build_robo6.py_end_is_card_mtime
C_bdg211-05 18:43:1411-05 18:43:450.5
C_sns-hvcm11-05 18:43:1411-05 18:44:561.7
NOBOOM11-05 18:44:4711-05 18:46:281.7
MAST11-05 18:51:5611-05 18:54:512.9out_no_build
fe_ipshocks11-05 18:54:5111-05 19:45:3450.7
fe_heatpump11-05 18:55:0111-05 19:10:4615.8
fe_pyrnet11-05 18:55:4211-05 21:58:12182.5
UTAQ (0.9.7.1 work under Amendment 14, carried into 0.9.7.2 by Amendment 15)11-05 22:36:2011-05 22:40:444.4
PVLAZ (0.9.7.1 work under Amendment 14, carried into 0.9.7.2 by Amendment 15)11-05 22:36:4211-05 22:37:370.9
UKWIND (0.9.7.1 work under Amendment 14, carried into 0.9.7.2 by Amendment 15)11-05 22:36:5511-05 22:54:3617.7
RCFPOWER (0.9.7.1 work under Amendment 14, carried into 0.9.7.2 by Amendment 15)11-05 22:38:1111-05 22:40:392.5
NOBOOM_M202210 (0.9.7.1 work under Amendment 14, carried into 0.9.7.2 by Amendment 15)11-05 22:42:1811-05 22:44:272.1
REASSEMBLE (0.9.7.1 work under Amendment 14, carried into 0.9.7.2 by Amendment 15)11-05 22:46:0711-07 00:57:501571.7stopped_early_VM_overload
Builds: 25, 1713.9 minutes in total, the longest UKWIND (1611.2 min).
whatstartendminutesnote
HOT_reselect11-05 18:40:1811-05 18:40:420.4
C_robo611-05 18:42:3211-05 19:04:3222.0end_is_card_mtime
C_bdg211-05 18:43:4511-05 18:45:091.4end_logged_late_actual_end_is_card_mtime
UAVSEAD11-05 18:43:4711-05 18:53:119.4
HAI11-05 18:43:5411-05 18:56:2512.5
PREDIST11-05 18:44:3111-05 18:48:374.1
C_sns-hvcm11-05 18:44:5611-05 18:45:140.3end_logged_late_actual_end_is_card_mtime
CAFUC211-05 18:45:3111-05 18:46:381.1
NOBOOM11-05 18:46:3211-05 18:46:370.1
UR5E11-05 18:50:2411-05 18:52:111.8
select_UR5E11-05 18:52:2311-05 18:53:080.8
ONBOARDEU11-05 18:54:3011-05 19:02:468.3
GREEKWIND11-05 18:59:2311-05 18:59:400.3
KUKA11-05 18:59:4011-05 19:09:079.4
RFLYMAD11-05 19:00:4711-05 19:08:217.6
fe_heatpump11-05 19:10:4611-05 19:12:321.8
TCABR11-05 19:11:5011-05 19:20:098.3
fe_ipshocks11-05 19:45:3411-05 19:47:151.7
PVLAZ (0.9.7.1 work under Amendment 14, carried into 0.9.7.2 by Amendment 15)11-05 22:37:3711-05 22:37:450.1
UKWIND (0.9.7.1 work under Amendment 14, carried into 0.9.7.2 by Amendment 15)11-05 22:38:0711-07 01:29:211611.2
RCFPOWER (0.9.7.1 work under Amendment 14, carried into 0.9.7.2 by Amendment 15)11-05 22:40:3911-05 22:40:470.1
UTAQ (0.9.7.1 work under Amendment 14, carried into 0.9.7.2 by Amendment 15)11-05 22:40:5311-05 22:46:485.9
NOBOOM_M202210 (0.9.7.1 work under Amendment 14, carried into 0.9.7.2 by Amendment 15)11-05 22:44:2711-05 22:44:350.1
NOBOOM_pooled11-06 23:54:3911-06 23:54:440.1
REASSEMBLE (0.9.7.1 work under Amendment 14, carried into 0.9.7.2 by Amendment 15)11-07 00:58:0911-07 01:03:145.1
Model-free measurements: 24, 53.5 minutes in total, the longest ESA (10.8 min).
whatstartendminutesnote
ESA11-05 18:41:1011-05 18:51:5510.8ESA_NASA_UKWIND
fe_F311-05 18:45:5711-05 18:54:328.6
CAFUC211-05 18:46:4511-05 18:49:262.7
PREDIST11-05 18:50:3411-05 18:51:010.5
NOBOOM11-05 18:51:0111-05 18:51:190.3
UR5E11-05 18:53:2011-05 18:53:510.5
UAVSEAD11-05 18:54:5711-05 18:55:430.8
HAI11-05 18:57:4111-05 18:58:190.6
GREEKWIND11-05 18:59:4911-05 19:00:080.3
KUKA11-05 19:09:5711-05 19:19:459.8
RFLYMAD11-05 19:10:0311-05 19:10:590.9
fe_heatpump11-05 19:12:3211-05 19:13:020.5
fe_heatpump_r211-05 19:13:1511-05 19:20:056.8
ONBOARDEU11-05 19:16:2411-05 19:17:070.7
HAI11-05 19:20:0511-05 19:20:320.5
TCABR11-05 19:20:2211-05 19:21:311.1
HAI_withEnd11-05 19:20:3211-05 19:21:140.7
PVLAZ (0.9.7.1 work under Amendment 14, carried into 0.9.7.2 by Amendment 15)11-05 22:37:5011-05 22:37:560.1
RCFPOWER (0.9.7.1 work under Amendment 14, carried into 0.9.7.2 by Amendment 15)11-05 22:40:5411-05 22:41:280.6
NOBOOM_M202210 (0.9.7.1 work under Amendment 14, carried into 0.9.7.2 by Amendment 15)11-05 22:45:0911-05 22:45:510.7
UTAQ (0.9.7.1 work under Amendment 14, carried into 0.9.7.2 by Amendment 15)11-05 22:46:5211-05 22:49:372.8
NOBOOM_pooled11-06 23:55:3911-06 23:56:120.6
REASSEMBLE (0.9.7.1 work under Amendment 14, carried into 0.9.7.2 by Amendment 15)11-07 01:05:5111-07 01:06:330.7
UKWIND (0.9.7.1 work under Amendment 14, carried into 0.9.7.2 by Amendment 15)11-07 01:38:3411-07 01:40:362.0

Started and not logged as ended when this card was built: crefs_C_new (since 11-05 19:11:13), build_UKWIND (0.9.7.1 work under Amendment 14, carried into 0.9.7.2 by Amendment 15) (since 11-06 23:53:00), download_REASSEMBLE (0.9.7.1 work under Amendment 14, carried into 0.9.7.2 by Amendment 15) (since 11-06 23:57:25) and phase4 (since 11-06 10:13:14).

Cluster jobs of the run (37, from sacct; node-hours = elapsed x nodes):

jobnamepartitionstatestart (UTC)end (UTC)task listnode-hours
3867625s097debugCOMPLETED11-05 18:49Z11-05 18:53Zjobs097_smoke.txt0.07
3867627p097preemptableCOMPLETED11-05 18:50Z11-05 18:55Zjobs097_w0.txt0.08
3867656p097preemptablePREEMPTED11-05 18:58Z11-05 19:17Zjobs097_w1.txt0.31
3867709g097preemptableCOMPLETED11-05 19:08Z11-05 19:08Z–0.01
3867710c097preemptableCANCELLED–11-05 19:08Z–0.00
3867712c097preemptableCOMPLETED11-05 19:09Z11-05 19:10Z–0.02
3867715g097preemptableCOMPLETED11-05 19:09Z11-05 19:10Z–0.01
3867716c097preemptableCOMPLETED11-05 19:10Z11-05 19:11Z–0.01
3867721e097preemptableCOMPLETED11-05 19:10Z11-05 19:11Z–0.02
3867722c097preemptableCOMPLETED11-05 19:15Z11-05 19:16Z–0.01
3867750gate097preemptableCOMPLETED11-05 19:15Z11-05 19:16Z–0.02
3867751c097preemptableCOMPLETED11-05 19:17Z11-05 19:17Z–0.01
3867791p097preemptableCOMPLETED11-05 19:21Z11-05 19:29Z–0.12
3867812p097preemptablePREEMPTED11-05 19:26Z11-05 19:34Z–0.15
3867837p097preemptableCOMPLETED11-05 19:26Z11-05 19:41Z–0.26
3867848c097preemptableCOMPLETED11-05 19:26Z11-05 19:26Z–0.01
3867960p097preemptableCOMPLETED11-06 04:04Z11-06 04:04Z–0.01
3867977f097preemptableCANCELLED–11-05 22:35Z–0.00
3867978fe097preemptableCANCELLED–11-05 22:35Z–0.00
3868446r097preemptableCANCELLED–11-05 22:35Z–0.00
3868447fe097preemptableCANCELLED–11-05 22:35Z–0.00
3868625a097preemptableCOMPLETED11-06 04:06Z11-06 05:01Z–0.90
3868626a097preemptableCOMPLETED11-06 04:06Z11-06 05:04Z–0.97
3868685a097preemptableCOMPLETED11-06 00:38Z11-06 01:16Z–0.63
3871836t097tabdebugCOMPLETED11-06 09:36Z11-06 09:48Zjobs097_tabsmoke.txt0.21
3872079t097tabpreemptableCANCELLED–11-06 10:00Z–0.00
3872371t097tabpreemptablePREEMPTED11-06 18:00Z11-06 18:01Zjobs097_tab.txt0.02
3872372t097tabpreemptablePREEMPTED11-06 18:00Z11-06 18:01Zjobs097_tab.txt0.02
3880281t097tabnormalCANCELLED–11-07 03:01Z–0.00
3880282t097tabnormalCANCELLED–11-07 03:01Z–0.00
3880285t097tabpreemptableCOMPLETED11-06 23:49Z11-07 02:08Z–2.32
3880314q097normalCANCELLED–11-06 23:58Z–0.00
3880315q097preemptableCOMPLETED11-06 23:55Z11-06 23:58Z–0.04
3880783q097preemptableCOMPLETED11-07 02:00Z11-07 02:47Z–0.79
3880784q097normalCANCELLED–11-07 03:01Z–0.00
3880786t097tabpreemptableCOMPLETED11-07 02:00Z11-07 03:01Z–1.01
3880787t097tabnormalCANCELLED–11-07 03:01Z–0.00
Other logged events
timeeventnote
11-05 18:33:41run_startsession launched ~18:33Z
11-05 18:38:13measure097_validatedmeasure097 reproduces nulls095 HOT 29/29 max diff 1.1e-16
11-05 18:45:00download_C_sns-hvcmbytes_correction_previous_end_entry
11-05 18:47:09build_UAVSEAD_restartadded_asset_field
11-05 18:50:41gate_A_reproA6g through CAFE-097 tree on Clariden: ZERO NOBOOM 20/20, pj_better_3p7m PREDIST 29/29 equal to 0.9.6 cells, max diff 0; smoke job 3867625
11-05 19:29:20phase1_A_C_donefinal A list and C admission written by 19:23Z; FE pending
11-06 10:13:14outageVM froze 2026-11-05 ~22:56Z (memory), network lost 23:11Z, reset 2026-11-06 09:01Z; agents stopped 00:16Z-02:55Z (DECISIONS097.md)
11-07 01:11:09build_UKWINDrestart_years_2016-2022_K2023_24_excluded_dup_exports

Inference and benchmarking time

FE feature-extraction time on the new FE datasets is not measured for 28 model-dataset tasks: their tasks saved the features and then ended with an error in a scoring step the board does not use (DECISIONS097.md, 2026-11-06 09:35Z), so they have no clean timing record. Their FE head time is measured (the FT tasks).

Per model, from the task-queue records of the cell jobs (timing097_infer.json, timing097.py). task wall = end - start of one queue task (one row on one dataset: embedding + head + scoring, with up to NW tasks sharing one GH200, so it is an upper bound of the GPU time of that task); A throughput = Eval series per second and scored windows per second of task wall; GPU-hours = sum of task walls / 3600 (an upper bound: tasks share GPUs); node-hours = sacct elapsed x nodes. Seconds are the sum over the model's tasks of each kind. Time is reported, never scored.

modelparametersA cells, sA gate, sA reference, sA7, sB7, sC embedding (Toto m1), sC embedding (menu row), sC embedding (stored row), sFE cells, sFE cells (head on stored features), sFE embedding, sFE gate, sFE reference, sA windows / sA series / sGPU-hours, upper bound
Chronos-2119.5 M2,60081–2,726––63––34–––850.2181.529
TimesFM-2.5-200M231.3 M1,793––3,076–––20–39–––1230.3171.369
Timestral-209M209.3 M1,876––2,760––32––31–––1170.3031.305
Timestral-123M122.6 M1,436––1,646––32––28–––1530.3960.873
Toto-Open-Base-1.0151.3 M1,545––1,425–28–––35–––1420.3680.842
MantisV2–1,123––1,319––26––37–––1960.5060.696
rawknn–––1,188–1,091––––––––––0.633
Mantis8.0 M889––806––23––38–––2470.6390.488
Timestral-28M27.8 M795––832––26––26–––2770.7150.466
pj_better_10m ◊10.6 M571––1,006––19––263––3850.9950.451
pjgood-warp-xview-combo-10M ◊10.6 M595––837––33––263––3700.9540.415
pj_better_3p7m ◊3.8 M32213–674––25––27327–6831.7630.303
LeSaturn-warppool-factorized-7.4M ◊7.4 M376––536––22––303––5851.5100.268
pj_better_1m ◊1.0 M177––697––13––233––1,2443.2120.254
Random-init LeSaturn (control)3.7 M274––568–––19628–––8032.0730.249
NuTime2.4 M341––348–––17–29–––6451.6670.204
Timestral-5M5.2 M340––321––24––33–––6461.6690.200
ZERO018112–126–––9–32–35–1,2153.1380.110
FEREF–––––––––––––87––0.024
levelonly–––46–38––––––––––0.023
maxz–––21–12––––––––––0.009
maxdz–––17–12––––––––––0.008

The cluster jobs of the run used 8.02 node-hours (32.09 GPU-hours at 4 GH200 per node), from sacct.

A material per dataset (the denominators of the A throughput)
A datasetEval seriesscored windowsrows
ESA9537,9291,187,429
NASA3714,049234,147
UKWIND8533,650905,922
HOT5019,818524,844
PREDIST4115,980220,652
NOBOOM207,750130,077
HAI5822,887688,644
UAVSEAD10137,668404,044
UR5E6323,617327,889
KUKA25599,9522,933,620
UTAQ8533,530873,265
RCFPOWER8433,2641,041,058
REASSEMBLE186,52966,477

Earlier cost table, for comparison (costs09.json, CAFE 0.9.1 datasets and settings, One A100 40 GB, 6 cores. 0.9.1: A measured on a shared GPU; part of the board estimated (see the text). The hardware and the datasets differ from 0.9.7, so compare orders of magnitude only):

modelC, minA, minFE, mintotal, minFE measured
TimesFM-2.5-200M18.7112.474.8206yes
TiRex-24.5116.263.5184scaled
Chronos-26.199.466.1172scaled
Timestral-209M9.742.669.2121scaled
Timestral-123M7.143.159.8110scaled
Toto-Open-Base-1.012.416.437.867yes
Timestral-28M2.712.645.961scaled
Mantis2.235.718.456scaled
LeSaturn-scm-10M-s436.412.025.244scaled
pjgood-warp-xview-combo-10M ◊3.523.514.441scaled
LeSaturn-scm-3.8M-s433.19.628.041scaled
LeSaturn-scm-10M-s425.011.323.440scaled
LeSaturn-scm-3.8M-s442.89.526.739scaled
pj_better_10m ◊3.220.613.337yes
LeSaturn-scm-10M-s445.38.722.937scaled
Timestral-5M4.94.920.831scaled
LeSaturn-scm-3.8M6.810.411.629scaled
LeSaturn-warppool-factorized-7.4M ◊3.511.912.628scaled
pj_better_3p7m ◊2.613.010.526scaled
NuTime0.012.911.124yes
Twin-LeSaturn-scm-3.8M3.44.614.923scaled
LeSaturn-warppool-factorized-3.8M3.35.412.121scaled
LeSaturn-warppool-factorized-3.8M-s433.25.211.720scaled
ZERO4.92.79.117yes

Whole-board hours, runs one after another (C / A / FE): 0.8.6: 1.5 / 15.9 / 38.4 = 55.8; 0.8.6 protocol, same 24 runs: 2.63 / 25.91 / 54.25 = 82.79; 0.9: 2.09 / 3.52 / 4.15 = 9.76.

The 0.9.1 speed note (speed09.json)

Scoring one model on CAFE 0.9.1 (15 C, 10 A and 16 FE datasets) takes about 37 min for pj_better_10m and 17 min for ZERO on one A100, against 2.4 h and 2.0 h in 0.8.6. The slowest model, TimesFM-2.5-200M, takes about 3.4 h. The whole board, 24 runs (17 models, 5 extra pretraining seeds, 2 baselines) one after another, costs about 25 hours, against 83 h for the same runs under the 0.8.6 protocol (NuTime, not on the 0.8.6 board, is missing from that total, and its C time from the 0.9.1 total).

Where the time goes now. A and FE now cost about the same (10.7 h and 11.7 h for the board). On A the kNN head is cheap, but every window of every A series is embedded, and large encoders are slow at that (Chronos-2 1.7 h, TimesFM 1.9 h on A). The 9 FE datasets that 0.9.1 restores cost 520 s for pj_better_10m and 391 s for ZERO: on the fast path they are cheap, because their 0.8.6 cost was the head (10 seeds on the CPU), not the encoder.

The fast path changes no number. With one exception, every fresh cell equals the stored cell bit for bit: five models rerun from scratch on the 0.9.0 datasets, and pj_better_10m and ZERO on the 9 restored FE datasets (18 of 18 cells equal seeds 0–2 of their 0.8.6 cells). The new models' cells were each repeated on one dataset with identical results. The exception is one Mantis FE value, 8.9e-7 away from its 0.7-era cell, because of that run's forward pass.

How it was measured: one A100 40 GB with 6 physical cores. C and the 7 FE datasets of 0.9.0: 5 models timed one at a time on an otherwise idle machine, 19 estimated from their 0.8.6 times; the C times are for the 13 C datasets of that study, which lack hydraulic, 3w and FreezerRegularTrain of the 15 in 0.9.1 (and include FreezerSmallTrain). Restored FE: measured for pj_better_10m, ZERO, NuTime, Toto and TimesFM; the other rows scaled by the measured ratio. A: measured for every model except TiRex-2 (its cells were copied), on 12 A sources (the board uses 10) and on a GPU shared with other jobs, so the A times lean high. Basis per run in cafe09/results/costs09.json.

model0.8.6 total0.9.1 C0.9.1 A0.9.1 FE (16 datasets)0.9.1 total
pj_better_3p7m1.1 h3 min13 min11 min26 min
ZERO2.0 h5 min3 min9 min17 min
pj_better_10m2.4 h3 min21 min13 min37 min
Mantis2.9 h2 min36 min18 min56 min
Chronos-24.6 h6 min99 min66 min172 min

C and the 7 FE datasets of 0.9.0 measured for all five; the 9 restored FE datasets measured for pj_better_10m and ZERO, scaled for the other three; A measured on a shared GPU. 0.8.6 totals of Mantis and Chronos-2 are modelled (0.8.6 study).

Cost per model

Minutes of GPU time per model (sum of its task wall times, an upper bound because up to several tasks share one GH200), on this version's cells only; reused cells of earlier versions cost nothing here. So the basis differs by row: rows differ in which of their tasks ran and left a clean record in this run (pj_better_3p7m and Chronos-2 also ran gate tasks; FE feature tasks on the new FE datasets count only where they ended cleanly, see above), and the front below compares this run's cost, not the cost of scoring a model on the whole of CAFE (see the cost table of the earlier versions for that). Cost is reported, never scored. The whole board of ranked rows took 9.66 GPU-hours by this measure; the cluster jobs used 8.02 node-hours.

modelseedsCAFEGPU minuteson the front
Timestral-5M10.699612.0yes
NuTime10.709812.2yes
pj_better_1m ◊10.726415.2yes
LeSaturn-warppool-factorized-7.4M ◊10.731316.1yes
pj_better_3p7m ◊10.736718.2yes
pjgood-warp-xview-combo-10M ◊10.736324.9
pj_better_10m ◊10.740527.1yes
Timestral-28M10.698728.0
Mantis10.738529.3
MantisV210.723041.8
Toto-Open-Base-1.010.694850.5
Timestral-123M10.701352.4
Timestral-209M10.703378.3
TimesFM-2.5-200M10.670382.1
Chronos-210.735191.7

On the front: no other ranked row has a higher CAFE score with the same or less GPU time.

Jobs and cells that did not finish

Logged as started and not as ended in the event log: 2 items (crefs_C_new and phase4). Rows with a missing cell on a task board (unranked on that task): 17. The list is in data/unfinished097.json.

10 min20 min50 min100 min0.670.680.690.700.710.720.730.74GPU minutes on this version's cells (upper bound), log scalepj_better_10m: CAFE 0.7405, 27.1 GPU min, seeds 1, on the frontpj_better_10m ◊Mantis: CAFE 0.7385, 29.3 GPU min, seeds 1pj_better_3p7m: CAFE 0.7367, 18.2 GPU min, seeds 1, on the frontpj_better_3p7m ◊pjgood-warp-xview-combo-10M: CAFE 0.7363, 24.9 GPU min, seeds 1Chronos-2: CAFE 0.7351, 91.7 GPU min, seeds 1LeSaturn-warppool-factorized-7.4M: CAFE 0.7313, 16.1 GPU min, seeds 1, on the frontLeSaturn-warppool-factorized-7.4M ◊pj_better_1m: CAFE 0.7264, 15.2 GPU min, seeds 1, on the frontpj_better_1m ◊MantisV2: CAFE 0.7230, 41.8 GPU min, seeds 1NuTime: CAFE 0.7098, 12.2 GPU min, seeds 1, on the frontNuTimeTimestral-209M: CAFE 0.7033, 78.3 GPU min, seeds 1Timestral-123M: CAFE 0.7013, 52.4 GPU min, seeds 1Timestral-5M: CAFE 0.6996, 12.0 GPU min, seeds 1, on the frontTimestral-5MTimestral-28M: CAFE 0.6987, 28.0 GPU min, seeds 1Toto-Open-Base-1.0: CAFE 0.6948, 50.5 GPU min, seeds 1TimesFM-2.5-200M: CAFE 0.6703, 82.1 GPU min, seeds 1
CAFE score against GPU minutes, one point per ranked model; a diamond is a ◊ row; the orange line joins the front, whose rows are labelled. Hover a point for its numbers. Cost is reported, never scored.

Change log

One section per version since 0.9.4, generated from the boards' dataset lists and the amendments; earlier releases follow in CHANGELOG.md.

v0.9.7.2 (2026-11-07)

24 C, 13 A and 15 FE datasets; 15 ranked models; CAFE = mean of C, A, FE; tier 1: pj_better_10m ◊ and Mantis.

v0.9.7 (2026-11-06)

24 C, 10 A and 15 FE datasets; 15 ranked models; CAFE = mean of C, A, FE; tier 1: pj_better_10m ◊.

v0.9.6

25 C, 14 A and 17 FE datasets; 15 ranked models; CAFE = mean of C, A, FE; tier 1: pj_better_10m ◊, pj_better_3p7m ◊, Mantis, pjgood-warp-xview-combo-10M ◊ and Chronos-2.

v0.9.5

17 C, 12 A and 16 FE datasets; 15 ranked models; CAFE = mean of C, A, FE; tier 1: pj_better_10m ◊, pj_better_3p7m ◊, Mantis, pjgood-warp-xview-combo-10M ◊, Chronos-2 and LeSaturn-warppool-factorized-7.4M ◊.

v0.9.4

16 C, 7 A and 16 FE datasets; 16 ranked models; CAFE = mean of C, A, FE; tier 1: MantisV2, Timestral-123M, Timestral-209M, Mantis, pjgood-warp-xview-combo-10M ◊, pj_better_10m ◊, pj_better_3p7m ◊, TiRex-2, pj_better_1m ◊, LeSaturn-warppool-factorized-7.4M ◊, Timestral-28M, Chronos-2 and Timestral-5M.

Provenance of the rules

Every amendment since 0.9.4 and every dated note since Amendment 13 (0.9.7 and 0.9.7.2), with the commit that first added it to PREREG09 (git log), the version it belongs to, the last released board known when it was written, what of 0.9.7 existed at that commit (first 0.9.7 cell: 2026-11-05 18:49Z; 0.9.7 board first committed: 2026-11-06 09:38Z; a provisional board, without the new FE datasets, was run from about 22:10Z on 2026-11-05 as a code check), and who requested it where the text says so. The headings in PREREG09 give the stated writing times.

ruleversionboard known0.9.7 at commitfirst committed (UTC)commitrequested by
Amendment 50.9.40.9.3.1–2026-11-02 21:35Z3be8b9d49Jonas Petersen
Amendment 60.9.50.9.4.1–2026-11-03 11:14Z799bbc68bJonas Petersen
Amendment 70.9.50.9.4.1–2026-11-03 13:54Zee6e213a0–
Amendment 80.9.50.9.4.1–2026-11-03 14:32Z4e04a33d8–
Amendment 90.9.50.9.4.1–2026-11-03 16:40Z06a56d1c1–
Amendment 100.9.50.9.4.1–2026-11-03 17:09Z4449cea4c–
Amendment 110.9.60.9.5–2026-11-04 21:47Z6024a5a7dJonas Petersen
Amendment 120.9.60.9.5–2026-11-04 22:01Z20333d8f5Jonas Petersen
Amendment 130.9.70.9.6no 0.9.7 cell2026-11-05 18:32Zadbffba1dJonas Petersen
Note 2026-11-05 18:46Z0.9.70.9.6no 0.9.7 cell2026-11-05 18:46Z2baf2b359–
Note 2026-11-05 18:57Z0.9.70.9.60.9.7 cells existed2026-11-05 18:57Z9c5d1019b–
Note 2026-11-05 19:02Z0.9.70.9.60.9.7 cells existed2026-11-05 19:02Z24d87c8ff–
Note 2026-11-05 19:05Z0.9.70.9.60.9.7 cells existed2026-11-05 19:05Z007d876a8–
Note 2026-11-05 19:10Z0.9.70.9.60.9.7 cells existed2026-11-05 19:10Z597b2c8e1–
Note 2026-11-05 19:12Z0.9.70.9.60.9.7 cells existed2026-11-05 19:12Z110f9d1aa–
Note 2026-11-05 19:13Z0.9.70.9.60.9.7 cells existed2026-11-05 19:13Z5fc6706a9–
Note 2026-11-05 19:15Z0.9.70.9.60.9.7 cells existed2026-11-05 19:15Zaba266e99–
Note 2026-11-05 19:21Z0.9.70.9.60.9.7 cells existed2026-11-05 19:21Z0fe475fab–
Note 2026-11-05 19:23Z0.9.70.9.60.9.7 cells existed2026-11-05 19:23Zb5a9676b8–
Note 2026-11-05 19:44Z0.9.70.9.60.9.7 cells existed2026-11-05 19:44Zc9d83054c–
Note 2026-11-05 21:59Z0.9.70.9.60.9.7 cells existed2026-11-05 21:59Zcf7f5facb–
Note 2026-11-05 22:32Z0.9.70.9.60.9.7 provisional board run (code check)2026-11-05 22:32Z4755a995b–
Note 2026-11-06 10:02Z0.9.70.9.60.9.7 board built2026-11-06 10:03Z84f5cc9ab–
Note 2026-11-06 10:10Z0.9.70.9.60.9.7 board built2026-11-06 10:22Zba8370e94–
Note 2026-11-06 11:05Z0.9.70.9.60.9.7 board built2026-11-06 11:05Zeb73278bbJonas Petersen
Amendment 140.9.7.10.9.70.9.7 provisional board run (code check)2026-11-05 22:35Zb70ca1a90Jonas Petersen
Amendment 150.9.7.20.9.70.9.7 board built2026-11-06 23:51Z199d87582Jonas Petersen
Note 2026-11-06 23:54Z0.9.7.20.9.70.9.7 board built2026-11-06 23:54Zdcf9de80a–
Note 2026-11-07 00:02Z0.9.7.20.9.70.9.7 board built2026-11-07 00:01Za1f71c5f6–
Note 2026-11-07 00:10Z0.9.7.20.9.70.9.7 board built2026-11-07 00:07Z4683f1fec–
Note 2026-11-07 01:45Z0.9.7.20.9.70.9.7 board built2026-11-07 01:42Zbced91633–

The headings of 2026-11-07 00:02Z (first commit 00:01Z), 2026-11-07 00:10Z (first commit 00:07Z) and 2026-11-07 01:45Z (first commit 01:42Z) give a time a few minutes after their first commit (the clock error corrected in the other two 0.9.7.2 headings); the commit time is the one to rely on.

Baselines

Per dataset, the best baseline (rows with no learned encoder: ZERO, and current value on FE) against the best and the median ranked model. A free forecast beating the best model was part of the reason to drop epa_aqs_pm25 and substorm_omni, while kept FE datasets where a free forecast also wins stay in (see the FE table below); the 0.9.7 admission is not free of model scores: the SNS HVCM gate read model cells and six of the drops were informed by scores: C 3W, bosch and soredd_abc, FE ngafid_hastall, substorm_omni and epa_aqs_pm25 (Amendment 13 items 1 and 6; item 1 calls the 3W drop structural, but item 6 and the screening log drop it on combo minus twin; see the conflict-of-interest section).

All 52 datasets
taskdatasetbest baselineits scorebest modelits scoremedian modelmodels below the baselineflag
Cucr/MoteStrainZERO0.859NuTime0.9490.9271 of 15
Cucr/SonyAIBORobotSurface2ZERO0.798LeSaturn-warppool-factorized-7.4M ◊0.9110.8363 of 15
Cucr/WaferZERO0.761Timestral-28M0.9980.9900 of 15
Cucr/ACSF1ZERO0.622Chronos-20.9060.8500 of 15
Cucr/PLAIDZERO0.598Mantis0.9420.8860 of 15
Cucr/FreezerRegularTrainZERO0.960pj_better_3p7m ◊0.9940.9801 of 15
Cucr/FordAZERO0.489Timestral-209M0.9480.9190 of 15
Cucr/ElectricDevicesZERO0.628Timestral-123M0.8050.7880 of 15
Cphys/lendbZERO0.574Timestral-209M0.9390.8981 of 15
Cmv06/hydraulicZERO0.987Timestral-123M0.9830.95815 of 15baseline above every model
Cmv06/gas251ZERO0.866MantisV20.9020.8822 of 15
Cmv06/steadZERO0.599Timestral-123M0.9770.9570 of 15
Cuea/AsphaltPavementTypeCoordinatesZERO0.586pjgood-warp-xview-combo-10M ◊0.9540.9190 of 15
Cmv06/aursad095ZERO0.653Timestral-28M0.7260.6920 of 15
Cmv06/voraus095ZERO0.639Chronos-20.7570.7310 of 15
Cnicola15/chillan5ZERO0.565Chronos-20.8940.8850 of 15
Cnicola15/haptic_texture8ZERO0.524MantisV20.6060.5710 of 15
Cnicola15/openrainer_mZERO0.805Chronos-20.8900.8740 of 15
Cnicola15/poseit_ftZERO0.595pjgood-warp-xview-combo-10M ◊0.6340.6143 of 15
Cnicola15/px4_airframe3ZERO0.607Timestral-28M0.8270.7760 of 15
Cnicola15/swiss_load10ZERO0.614pj_better_1m ◊0.6470.6313 of 15
Cnicola15/ved_powertrain3ZERO0.823pjgood-warp-xview-combo-10M ◊0.9210.9050 of 15
Cnicola15/bdg2_meter6ZERO0.580NuTime0.7920.7640 of 15
Cnicola15/robo6_rb6ZERO0.594pjgood-warp-xview-combo-10M ◊0.7760.7410 of 15
AHAIZERO0.660Mantis0.6640.59914 of 15baseline at or above the median model
AUKWINDZERO0.792NuTime0.8290.78710 of 15baseline at or above the median model
AHOTZERO0.645NuTime0.6900.61713 of 15baseline at or above the median model
APREDISTZERO0.701NuTime0.6930.65015 of 15baseline above every model
ANOBOOMZERO0.789pj_better_1m ◊0.8130.77110 of 15baseline at or above the median model
AESAZERO0.654pjgood-warp-xview-combo-10M ◊0.6850.62410 of 15baseline at or above the median model
ANASAZERO0.585MantisV20.6710.6381 of 15
AUAVSEADZERO0.616Toto-Open-Base-1.00.6290.59214 of 15baseline at or above the median model
AUR5EZERO0.491Chronos-20.6120.5870 of 15
AKUKAZERO0.571pj_better_10m ◊0.5710.53614 of 15baseline at or above the median model
AUTAQZERO0.618Mantis0.6440.6282 of 15
ARCFPOWERZERO0.682MantisV20.6980.65712 of 15baseline at or above the median model
AREASSEMBLEZERO0.536Mantis0.6350.6101 of 15
FEstickslip_poinclouxZERO0.617Toto-Open-Base-1.00.7650.6820 of 15
FEminati_circuitscurrent value0.647Toto-Open-Base-1.00.8210.7750 of 15
FEgeyser_strokkurcurrent value0.623Mantis0.7400.6870 of 15
FEndbc_buoy_v2_bcurrent value0.689Toto-Open-Base-1.00.6830.63215 of 15baseline above every model
FEnasa_battery_rwcurrent value0.872Mantis0.8790.86014 of 15baseline at or above the median model
FEwind_genbearing_hotcurrent value0.703Toto-Open-Base-1.00.6520.62315 of 15baseline above every model
FEtowie_genbearing_hotcurrent value0.739pj_better_3p7m ◊0.7160.67415 of 15baseline above every model
FEmemristor_jerkZERO0.844Toto-Open-Base-1.00.9330.9003 of 15
FEnoaa_coops_surge_bcurrent value0.795Toto-Open-Base-1.00.7370.69015 of 15baseline above every model
FEasos_lows_bcurrent value0.730Toto-Open-Base-1.00.7270.68615 of 15baseline above every model
FEusgs_reach_v2_bZERO0.580pj_better_3p7m ◊0.7080.6860 of 15
FErotary_utrechtZERO0.612pj_better_10m ◊0.6600.6376 of 15
FEmegathrust_sync_v2ZERO0.542pj_better_10m ◊0.7400.7120 of 15
FEcdip_waves_ccurrent value0.738Toto-Open-Base-1.00.7010.63515 of 15baseline above every model
FEheatpump_defrostcurrent value0.701pj_better_10m ◊0.7080.66413 of 15baseline at or above the median model

On FE, the best forecast with no encoder (current value, trend, curvature, early-warning statistics, spectral shift, level statistics) per dataset against the best ranked model. On 7 of 15 FE datasets a no-encoder forecast scores above every ranked model (ndbc_buoy_v2_b, wind_genbearing_hot, towie_genbearing_hot, noaa_coops_surge_b, asos_lows_b, cdip_waves_c and heatpump_defrost). For the new FE datasets the last column is pjgood-warp-xview-combo-10M ◊ minus the random-init LeSaturn row (3.7 M; not a matched twin), the second number Amendment 13 item 5 asks the card to show.

All 15 FE datasets
FE datasetbest no-encoder forecastits scorebest ranked modelits scoreranked rows below itflagcombo-10M minus random-init LeSaturn (new datasets)
stickslip_poinclouxearly-warning statistics0.702Toto-Open-Base-1.00.76510 of 15
minati_circuitscurvature0.761Toto-Open-Base-1.00.8216 of 15
geyser_strokkurearly-warning statistics0.731Mantis0.74012 of 15
ndbc_buoy_v2_bcurrent value0.689Toto-Open-Base-1.00.68315 of 15free forecast above every ranked model
nasa_battery_rwcurrent value0.872Mantis0.87914 of 15
wind_genbearing_hotcurrent value0.703Toto-Open-Base-1.00.65215 of 15free forecast above every ranked model
towie_genbearing_hotlevel statistics0.754pj_better_3p7m ◊0.71615 of 15free forecast above every ranked model
memristor_jerkcurvature0.883Toto-Open-Base-1.00.9336 of 15
noaa_coops_surge_bcurrent value0.795Toto-Open-Base-1.00.73715 of 15free forecast above every ranked model
asos_lows_btrend0.744Toto-Open-Base-1.00.72715 of 15free forecast above every ranked model
usgs_reach_v2_bearly-warning statistics0.653pj_better_3p7m ◊0.7086 of 15
rotary_utrechtlevel statistics0.647pj_better_10m ◊0.6609 of 15
megathrust_sync_v2level statistics0.674pj_better_10m ◊0.7406 of 15
cdip_waves_ccurrent value0.738Toto-Open-Base-1.00.70115 of 15free forecast above every ranked model
heatpump_defrost ●curvature0.729pj_better_10m ◊0.70815 of 15free forecast above every ranked model+0.027

Per task, the three strongest unranked rows (controls, no-encoder references and the specialist row) and how many ranked rows score below them on the task score; a row with no ranked row below it is not listed. The per-dataset baseline above is ZERO or current value only; on FE several no-encoder forecasts score higher than current value.

taskrowkindtask scoreranked rows below it
CMultiROCKET-Hydraspecialist0.86415 of 15
AZEROcontrol0.6428 of 15
ARandom-init LeSaturn (control)control0.6112 of 15
Amax-|z|anchor0.5921 of 15
FElevel statisticsadmission0.7199 of 15
FEcurvatureadmission0.7077 of 15
FERandom-init LeSaturn (control)control0.7037 of 15

On A, the best of the three data-free rows (no data read: a score that rises or falls with elapsed time, and a random score per window) against the ranked models. The elapsed-time gate of Amendment 13 item 4 (at most 0.65) admits a dataset; it does not promise that every model beats these rows.

A datasetbest data-free rowits scorebest modelits scoremedian modelmodels below itflag
HAInull: elapsed time rising0.533Mantis0.6640.5990 of 15
UKWINDnull: elapsed time rising0.541NuTime0.8290.7870 of 15
HOTnull: elapsed time rising0.529NuTime0.6900.6170 of 15
PREDISTnull: elapsed time rising0.512NuTime0.6930.6500 of 15
NOBOOMnull: elapsed time rising0.561pj_better_1m ◊0.8130.7710 of 15
ESAnull: elapsed time rising0.611pjgood-warp-xview-combo-10M ◊0.6850.6241 of 15
NASAnull: elapsed time rising0.516MantisV20.6710.6380 of 15
UAVSEADnull: elapsed time rising0.585Toto-Open-Base-1.00.6290.5926 of 15
UR5Enull: elapsed time rising0.616Chronos-20.6120.58715 of 15data-free row above every ranked model
KUKAnull: elapsed time falling0.519pj_better_10m ◊0.5710.5362 of 15
UTAQnull: elapsed time rising0.560Mantis0.6440.6281 of 15
RCFPOWERnull: elapsed time falling0.513MantisV20.6980.6570 of 15
REASSEMBLEnull: elapsed time rising0.556Mantis0.6350.6101 of 15

Flagged: UR5E (null: elapsed time rising 0.616 against the best model Chronos-2 0.612). On such a dataset the ranking orders models that do no better than knowing the position in the series.

Per model, the datasets where it scores more than 0.01 (the practical-equality margin) below the best baseline:

modeldatasetswhich
pj_better_10m ◊11C:hydraulic, A:HAI, A:HOT, A:PREDIST, A:UAVSEAD, FE:ndbc_buoy_v2_b, FE:wind_genbearing_hot, FE:towie_genbearing_hot, FE:noaa_coops_surge_b, FE:asos_lows_b, FE:cdip_waves_c
Mantis10C:hydraulic, A:PREDIST, A:ESA, A:UAVSEAD, A:KUKA, FE:ndbc_buoy_v2_b, FE:wind_genbearing_hot, FE:towie_genbearing_hot, FE:noaa_coops_surge_b, FE:cdip_waves_c
pj_better_3p7m ◊11C:hydraulic, A:HAI, A:HOT, A:PREDIST, A:UAVSEAD, FE:ndbc_buoy_v2_b, FE:wind_genbearing_hot, FE:towie_genbearing_hot, FE:noaa_coops_surge_b, FE:asos_lows_b, FE:cdip_waves_c
pjgood-warp-xview-combo-10M ◊12C:hydraulic, A:HAI, A:HOT, A:PREDIST, A:UAVSEAD, A:RCFPOWER, FE:ndbc_buoy_v2_b, FE:wind_genbearing_hot, FE:towie_genbearing_hot, FE:noaa_coops_surge_b, FE:asos_lows_b, FE:cdip_waves_c
Chronos-216C:hydraulic, A:HAI, A:HOT, A:PREDIST, A:NOBOOM, A:ESA, A:UAVSEAD, A:KUKA, A:RCFPOWER, FE:ndbc_buoy_v2_b, FE:nasa_battery_rw, FE:wind_genbearing_hot, FE:towie_genbearing_hot, FE:noaa_coops_surge_b, FE:cdip_waves_c, FE:heatpump_defrost
LeSaturn-warppool-factorized-7.4M ◊16C:hydraulic, A:HAI, A:UKWIND, A:HOT, A:PREDIST, A:UAVSEAD, A:KUKA, A:RCFPOWER, FE:ndbc_buoy_v2_b, FE:nasa_battery_rw, FE:wind_genbearing_hot, FE:towie_genbearing_hot, FE:noaa_coops_surge_b, FE:asos_lows_b, FE:cdip_waves_c, FE:heatpump_defrost
pj_better_1m ◊14C:SonyAIBORobotSurface2, C:hydraulic, A:HAI, A:UKWIND, A:HOT, A:PREDIST, A:RCFPOWER, FE:ndbc_buoy_v2_b, FE:wind_genbearing_hot, FE:towie_genbearing_hot, FE:noaa_coops_surge_b, FE:asos_lows_b, FE:cdip_waves_c, FE:heatpump_defrost
MantisV215C:hydraulic, A:HAI, A:UKWIND, A:PREDIST, A:NOBOOM, A:ESA, A:UAVSEAD, A:KUKA, FE:ndbc_buoy_v2_b, FE:wind_genbearing_hot, FE:towie_genbearing_hot, FE:noaa_coops_surge_b, FE:asos_lows_b, FE:cdip_waves_c, FE:heatpump_defrost
NuTime15C:hydraulic, A:HAI, A:NOBOOM, A:ESA, A:UAVSEAD, A:KUKA, A:RCFPOWER, FE:ndbc_buoy_v2_b, FE:wind_genbearing_hot, FE:towie_genbearing_hot, FE:noaa_coops_surge_b, FE:asos_lows_b, FE:rotary_utrecht, FE:cdip_waves_c, FE:heatpump_defrost
Timestral-209M18C:hydraulic, A:HAI, A:HOT, A:PREDIST, A:NOBOOM, A:ESA, A:UAVSEAD, A:KUKA, A:RCFPOWER, FE:ndbc_buoy_v2_b, FE:nasa_battery_rw, FE:wind_genbearing_hot, FE:towie_genbearing_hot, FE:noaa_coops_surge_b, FE:asos_lows_b, FE:rotary_utrecht, FE:cdip_waves_c, FE:heatpump_defrost
Timestral-123M17A:HAI, A:HOT, A:PREDIST, A:NOBOOM, A:ESA, A:UAVSEAD, A:KUKA, A:RCFPOWER, FE:ndbc_buoy_v2_b, FE:nasa_battery_rw, FE:wind_genbearing_hot, FE:towie_genbearing_hot, FE:noaa_coops_surge_b, FE:asos_lows_b, FE:rotary_utrecht, FE:cdip_waves_c, FE:heatpump_defrost
Timestral-5M17A:HAI, A:HOT, A:PREDIST, A:NOBOOM, A:ESA, A:UAVSEAD, A:KUKA, FE:ndbc_buoy_v2_b, FE:nasa_battery_rw, FE:wind_genbearing_hot, FE:towie_genbearing_hot, FE:memristor_jerk, FE:noaa_coops_surge_b, FE:asos_lows_b, FE:rotary_utrecht, FE:cdip_waves_c, FE:heatpump_defrost
Timestral-28M19C:hydraulic, A:HAI, A:HOT, A:PREDIST, A:NOBOOM, A:ESA, A:UAVSEAD, A:KUKA, A:RCFPOWER, FE:ndbc_buoy_v2_b, FE:nasa_battery_rw, FE:wind_genbearing_hot, FE:towie_genbearing_hot, FE:memristor_jerk, FE:noaa_coops_surge_b, FE:asos_lows_b, FE:rotary_utrecht, FE:cdip_waves_c, FE:heatpump_defrost
Toto-Open-Base-1.021C:SonyAIBORobotSurface2, C:FreezerRegularTrain, C:lendb, C:hydraulic, C:gas251, C:poseit_ft, A:HAI, A:UKWIND, A:HOT, A:PREDIST, A:NOBOOM, A:ESA, A:KUKA, A:UTAQ, A:RCFPOWER, FE:nasa_battery_rw, FE:wind_genbearing_hot, FE:towie_genbearing_hot, FE:noaa_coops_surge_b, FE:cdip_waves_c, FE:heatpump_defrost
TimesFM-2.5-200M25C:MoteStrain, C:hydraulic, C:gas251, C:swiss_load10, A:HAI, A:UKWIND, A:HOT, A:PREDIST, A:NOBOOM, A:ESA, A:NASA, A:UAVSEAD, A:KUKA, A:UTAQ, A:RCFPOWER, FE:ndbc_buoy_v2_b, FE:nasa_battery_rw, FE:wind_genbearing_hot, FE:towie_genbearing_hot, FE:memristor_jerk, FE:noaa_coops_surge_b, FE:asos_lows_b, FE:rotary_utrecht, FE:cdip_waves_c, FE:heatpump_defrost

Best baseline, median and best model on every dataset

Noise budget

Signal against test-material noise, over the ranked rows: the SD of their scores (signal) and the bootstrap SD of the paired difference of two rows (test material, median over pairs). A single-seed pair closer than 0.009 is marked within seed noise; that margin comes from the 0.9.2 noise budget, which also counted pretraining seed, head training set and readout choice and was not rerun.

boardsignal SDtest material, SD of a pairspread over pair noise (not the test's signal-to-noise)
C0.01840.001810.15
A0.02080.00464.50
FE0.04380.003014.39
CAFE0.02110.002010.68

The datasets themselves are a further source: with them resampled, 62 of 105 pairs stay resolved, against 90 on the main board (15 of them within seed noise).

Elo

Elo from paired per-dataset wins of the ranked rows (ZERO = 1000; 2,000 draws; elo095.py's method). It is a figure (inner intervals that do not overlap are not a resolved pair) and gives no verdict: the CAFE score, its Holm verdicts and tiers are the board. Inner interval: test material resampled; outer: datasets resampled too. Guests are rated against the frozen ratings of the ranked rows; a guest without a cell on a dataset takes ZERO's value there, and the share is in the last column. Third-party pool: the Elo of a fit without the ◊ rows in the pool.

‡: some of the row's dataset cells are missing and were filled with ZERO's value for the Elo matches (column imputed). The tabular rows are among them where their cells are missing (see 0.9.7.2 beside 0.9.7).

rowkindElotest materialand datasetsElo CElo AElo FEthird-party poolimputed
pj_better_10m ◊ranked, one readout12801263 to 12971238 to 13291506107114831269
pjgood-warp-xview-combo-10M ◊ranked, one readout12501234 to 12681209 to 12931544107313151254
pj_better_3p7m ◊ranked, one readout12491233 to 12661202 to 12981478106714081245
Mantisranked, one readout12491231 to 12671201 to 13011416106914981246
Chronos-2ranked, one readout12181201 to 12361177 to 1262154094213741223
LeSaturn-warppool-factorized-7.4M ◊ranked, one readout11771161 to 11931133 to 1222139594014181181
MantisV2ranked, one readout11681149 to 11871111 to 12271455100412071172
pj_better_1m ◊ranked, one readout11671153 to 11831124 to 12181377101013041179
NuTimeranked, one readout10941076 to 11121036 to 115313969879891107
Timestral-209Mranked, one readout10651050 to 10801024 to 110315138628561066
LightGBMtabular, defaults10561036 to 1079994 to 1125––––
Timestral-123Mranked, one readout10511036 to 10661011 to 109214938488471055
Timestral-28Mranked, one readout10301014 to 1046986 to 107214578507961027
Toto-Open-Base-1.0ranked, one readout10301015 to 1044972 to 1085114367114531013
Timestral-5Mranked, one readout1012996 to 1028961 to 105913678508791017
ZEROno encoder1000977 to 1023924 to 1067––––
Random-init LeSaturn (control)control978956 to 999911 to 1043––––
Raw referenceno encoder933914 to 950820 to 1022––––
TimesFM-2.5-200Mranked, one readout838814 to 861776 to 8971242651631831
CatBoost ‡tabular, defaults11531132 to 11751075 to 1230––––23%
XGBoost ‡tabular, defaults11011080 to 11221031 to 1170––––10%
RealMLP ‡tabular, defaults992968 to 1019912 to 1069––––33%

Open dataset questions

Questions about datasets that this version leaves open; none changes a score.

Heat maps

One row per model, one column per dataset. Lighter is lower and darker is higher, so the scale also reads in grey and for colour-blind readers. A thick edge marks the best ranked model on a dataset. The rows follow the switches above the board.

Hover or tap a cell for its value.

Breakdowns

Per dataset

Best and median are over ranked models. A flag says a baseline beats every model on that dataset, or reaches at least the median model. Those datasets stay in on purpose.

Dataset groups

Domain groups are for browsing and carry no weight in any score. Numbers are the main board's. To see the board without a group, use the switches at the top.

Pair verdicts

Row against column. Hover or tap a cell for the difference, its interval and the Holm p. Unresolved does not mean equal.

Hover or tap a cell.

Tiers

CAFE score

Tier 1 pj_better_10m ◊, Mantis

Tier 2 pj_better_3p7m ◊, pjgood-warp-xview-combo-10M ◊, Chronos-2

Tier 3 LeSaturn-warppool-factorized-7.4M ◊

Tier 4 pj_better_1m ◊, MantisV2

Tier 5 NuTime

Tier 6 Timestral-209M, Timestral-123M

Tier 7 Timestral-5M, Timestral-28M, Toto-Open-Base-1.0

Tier 8 TimesFM-2.5-200M

C, classification

Tier 1 Chronos-2, pjgood-warp-xview-combo-10M ◊, Timestral-209M

Tier 2 Timestral-123M, pj_better_10m ◊, pj_better_3p7m ◊

Tier 3 Timestral-28M

Tier 4 Mantis, LeSaturn-warppool-factorized-7.4M ◊, NuTime, Timestral-5M, MantisV2

Tier 5 pj_better_1m ◊

Tier 6 TimesFM-2.5-200M

Tier 7 Toto-Open-Base-1.0

A, anomaly detection

Tier 1 Mantis, pj_better_10m ◊, pj_better_3p7m ◊, pjgood-warp-xview-combo-10M ◊, pj_better_1m ◊, MantisV2, NuTime, LeSaturn-warppool-factorized-7.4M ◊, Chronos-2

Tier 2 Timestral-5M, Timestral-209M, Timestral-28M, Timestral-123M

Tier 3 TimesFM-2.5-200M

Tier 4 Toto-Open-Base-1.0

FE, forecasting events

Tier 1 Mantis, pj_better_10m ◊, Toto-Open-Base-1.0, LeSaturn-warppool-factorized-7.4M ◊

Tier 2 pj_better_3p7m ◊, Chronos-2

Tier 3 pjgood-warp-xview-combo-10M ◊, pj_better_1m ◊

Tier 4 MantisV2

Tier 5 NuTime

Tier 6 Timestral-5M, Timestral-209M, Timestral-123M

Tier 7 Timestral-28M

Tier 8 TimesFM-2.5-200M

Sensitivities

With and without A

Computed from the per-dataset values on this page. Ranks count the ranked rows only.

Other rule sets and dataset choices

Each row is a whole board under one changed rule, from sens097.json.

Datasets resampled

The datasets are drawn again with replacement within each task. Computed by sens097.json (sens095.py), 10,000 draws, each also resampling the test material. This asks how much the order depends on which datasets are in the benchmark, which the board's own interval does not cover.

Noise

Spread over pair noise, this version (the 1.0 test's A signal-to-noise is in the 1.0 test section)

Recomputed from this version's draws: the spread of the ranked rows' scores divided by the test-material noise of a pair. The other sources below are not in it.

A null floors per dataset

Each null is normalised as the dataset's score is, so 0.5 is chance. They are shown as unranked rows; the elapsed-time nulls also gate new A datasets (Amendment 13 item 4). In 0.9.7.2 an A dataset is admitted only with at least 15 Eval series, at least 20 anomaly segments and a median anomaly share of at most 0.3, and a new one also needs the better elapsed-time null at most 0.65 (Amendment 13 items 2 and 4; the card's Admission section lists every candidate). These scores read no model.

13 A datasets are admitted; the rule asks for at least 5. The mean of their best data-free scores is 0.550, and 15 of 15 ranked rows score above it on those datasets; the rule asks for at least half. So A counts in the CAFE score.

Source: the data-free rows of board097.json (centre assignment, Eval series).

Rules in plain words

Inputs

Every encoder gets real samples with no padding and no mask. What an encoder does inside, such as padding to its own context or normalising its input, counts as part of the model under test. The models table lists the input rules each row used.

On C, a recording of 512 samples or fewer is stretched to 512 by linear interpolation for every model, which changes the time scale in the same way for every encoder. A longer recording goes to the model whole: a model that accepts variable length gets it up to its maximum context, most recent samples kept, and a model with a fixed input length of 512 gets it block-averaged to 512. On A, the window is 256 real samples ending at the scored moment, never stretched; its score goes to the window's centre step, and a series needs a prefix of at least 512 samples or it leaves the list. On FE, every model gets the last 512 samples before the forecast time. Rows that embed no window, the tabular rows and hand-made references, keep their registered inputs.

Heads

The head is the only thing fitted, and it is the same for every model. C uses L2 logistic regression on one block per channel: a model's native width when it is at most 256 per channel, and wider embeddings projected to 256 per channel by a fixed random projection (at most 8,192 in total), with its penalty chosen by cross-validation inside the training split. A uses no labels: it scores a window by the mean of its 5 smallest Mahalanobis distances to the windows of the series' own prefix. FE fits one ridge regression per lead time on the embedding at the end of the context, with 3 head seeds.

Scores

Each dataset gives one number on the same scale, 0.5 for chance and 1 for perfect. C uses balanced accuracy rescaled for the number of classes, A uses VUS-PR after the prefix rescaled against random scores, and FE uses stratified AUROC averaged over lead times and head seeds. A task score is the plain mean over its datasets. The CAFE score is the mean of C, A and FE.

Readouts

Each model has one readout, the same on C, A and FE. The candidates are m0 (the released embedding), m1, m2, m3 and m6, as far as the model's code exposes them; the pick has the highest validation CAFE score, the mean over the tasks of cross-validation on the training split for C, the Tuning series for A and the inner cross-validation of the ridge head for FE. A candidate within 0.005 of the best loses to an earlier one. No test value is read. The picks were made in 0.9.5 and not rerun; the A part of the validation score used the Tuning series of the 0.9.5 A datasets, which were the Forgis rows' development set (Amendment 9 item 2).

Intervals

An interval is a paired 95% percentile bootstrap of the test material inside each dataset: test samples or splits on C, Eval series on A with near-duplicates resampled together, time blocks on FE. There are 10,000 draws, the same for every model. The datasets are held fixed, so an interval says nothing about how a model would do on other datasets; the resampled table under Sensitivities speaks to that.

Verdicts and tiers

Every pair of ranked models gets one of three verdicts. It is better if the Holm-corrected two-sided bootstrap p is below 0.05. It is practically equal if not resolved and the 95% interval of the difference lies inside plus or minus 0.01. Otherwise it is unresolved, which means the data do not say. A pair of single-seed models less than 0.009 apart is also marked within seed noise. Tier 1 holds the models above the first one resolved below the top model, and the next tier starts there, so a model in a lower tier can still be unresolved against the top one. Rank inside a tier is not a claim.

Admission checks

A dataset must come from an inanimate physical system, its label must be a state, type or identity of that system or an event observed in the record, it needs at least 100 test samples on C or 1,000 events on FE, and only the largest dataset of a recording campaign stays. In 0.9.7.2 an A dataset needs at least 15 Eval series, at least 20 anomaly segments and a median anomaly share of at most 0.3, with at most 300 series per source of independent runs and 100 per source with several series per asset; a new A dataset also needs the better of the rising and falling elapsed-time nulls at most 0.65, a data-free check that removed RflyMAD, OnboardEU and TCABR; a new FE dataset needs at least 300 events, 30 independent units or days and data-free reference rows below fixed limits (Amendment 13 items 2, 4 and 5). A series needs the prefix above. Not every check is free of model scores: the C candidate SNS HVCM had to pass a combo-10M minus random-init LeSaturn gate on its own cells, and six of the 0.9.7.2 drops (C 3W, bosch, soredd_abc; FE ngafid_hastall, substorm_omni, epa_aqs_pm25) were informed by scores (Amendment 13 items 1 and 6). A free forecast beating the best model was part of the reason to drop epa_aqs_pm25 and substorm_omni, while kept FE datasets where a free forecast also wins stay in; the boards show where that happens.

Conflict of interest

Forgis maintains CAFE and developed 5 of the 15 ranked models, all with CAFE Eval scores visible. Jonas Petersen of Forgis requested Amendment 13 with the 0.9.6 board known, and Amendment 15 (0.9.7.2) with the released 0.9.7 board and its per-dataset A signal-to-noise known (from 1.0 on UKWIND to 6.1 on KUKA), after UTAQ, RCFPOWER, PVLAZ and NOBOOM_M202210 had been measured without a model (2026-11-05 22:37Z to 22:49Z). Forgis staff chose the drops, adds, caps and thresholds of 0.9.7 with the 0.9.6 board, its separability table and a per-dataset comparison of the Forgis rows known; some drops were informed by scores (Amendment 13 item 1), the two C additions were accepted after a screening that scored pjgood-warp-xview-combo-10M ◊ against its random-init twin on them, and the A thresholds were placed in gaps of the 0.9.6 data. Applied to the 0.9.6 scores, the drops alone raise the mean ◊ lead (see What changed in 0.9.7.2). The pair count of the test that decides between 0.9.7.2 and 0.9.5 as the CAFE 1.0 candidate was fixed before any 0.9.7 cell existed (Amendment 13 item 9; Amendment 15 item 4 applies it to 0.9.7.2); its reading of the A signal-to-noise was fixed at 19:02Z on 2026-11-05, after the first 0.9.7 A cells existed and before any was read. The boards under older rules stay published at their tags. FE is built with PrecursorBench, a Forgis benchmark (built by a Forgis team member; Forgis/PrecursorBench on the Hub): 14 of the 15 FE datasets come from it, and the ◊ variant counts above include variants scored on PrecursorBench Eval data. The mean ◊ lead (mean ◊ row minus mean other ranked row) is +0.009 on C, +0.021 on A and +0.050 on FE; the boards without each task are under Sensitivities. The C admission gate of Amendment 13 item 6 measures the pretraining gain of one ◊ model (pjgood-warp-xview-combo-10M against the random-init LeSaturn row); no third-party model's gain is part of it, so it is score-informed in favour of the ◊ rows. It stays as written for this version and is to be replaced in the next version by a gate that reads no ranked row.

What the board does not show

The board does not show performance after fine-tuning, because nothing is fine-tuned. It does not cover datasets outside the list, and its intervals leave out pretraining seed, head training set and readout choice. It says nothing about value forecasting, because FE scores events only. Run time is reported and never scored. CAFE publishes scores and no data.

Known limitations, as written on the dataset card
  • Each model keeps the readout chosen for it in 0.9.5 (Amendment 13 item 7), chosen without the 0.9.7.2 datasets.
  • The drops of 0.9.7 were chosen with the 0.9.6 board known; some of them were informed by scores (Amendment 13 item 1), and the A thresholds sit in gaps seen in the 0.9.6 data, so they are post-hoc cut-offs.
  • The SNS HVCM gate cells were computed after the A cells of the first admission waves, not first as Amendment 13 item 6 said (PREREG09 note of 2026-11-05 19:05Z); no score had been read.
  • UR5e is under a non-commercial licence (CC BY-NC 4.0).
  • On HAI (7 recordings), UKWIND (5 recordings), HOT (9 recordings), ESA (2 recordings), KUKA (33 recordings), UTAQ (7 recordings) and RCFPOWER (3 recordings), some A Tuning series are other blocks of the same recordings as Eval series, though no series is in both. The readouts were chosen in 0.9.5 on the Tuning series of CARE_A, CARE_C, ESA_M1, ESA_M2, HAI, HOT, MSL, SKAB, SMAP, UKWIND and W3, so this matters where those datasets overlap the 0.9.7.2 ones; for datasets new in 0.9.7.2 (KUKA among them) no readout was chosen on their Tuning series. The recording-clustered sensitivity of the test is under the 1.0 test. The A Tuning series were also the ◊ rows' development set, so on these datasets ◊ development touched recordings whose other blocks are in Eval.
  • On 7 of 15 FE datasets (ndbc_buoy_v2_b, wind_genbearing_hot, towie_genbearing_hot, noaa_coops_surge_b, asos_lows_b, cdip_waves_c and heatpump_defrost) a forecast with no encoder scores above every ranked model; FE is also where the ◊ lead is largest. A free forecast beating the best model was part of the reason to drop epa_aqs_pm25 and substorm_omni, and the same holds on these kept or added datasets (see Baselines).
  • On UR5E a data-free elapsed-time row (0.616) scores above every ranked model (best 0.612); see Baselines. Each UR5e series puts the last rows of a separate normal run, recorded earlier, before the anomalous run (build_ur5e.py), so prefix and scored part come from different runs.
  • PreDist: the 7 days appended after each fault are the one build step that reads an event time; on PreDist and NoBOOM a score that follows elapsed time does well, and the data-free rows show by how much.
  • Recording length carries label information in two C datasets, in the source data itself: a classifier that sees only the length scores uea/AsphaltPavementTypeCoordinates 0.720 and ucr/PLAID 0.596 (see Datasets admitted and removed); every other C dataset has one length per recording. The variable-length models see the length of recordings above 512 samples and the block-averaged models do not.
  • FactoryNet's README warns that recording context partly predicts fault labels, so real fault evaluation should stay within one recording session. mv06/voraus and the other pooled C datasets mix sessions, and a voraus split tests settings not seen in training.
  • voraus-AD comes from FactoryNet's copy, not the publisher's files, under CC BY-NC-SA 4.0. Forgis is a company, so CAFE publishes scores only, and a legal reading is still open.
  • The weights of the Forgis rows are not public. Each carries a sha256 under Models, so a later release can be checked against it, and until then outsiders cannot rerun those rows.
  • At least 66 distinct Saturn-family checkpoints, including the five ranked Forgis rows and two HEPA-lineage precursors, have CAFE or PrecursorBench 1.4 Eval scores on datasets of the 0.9.3.1 board, with records dated from 2026-08-04 to 2026-10-28. Before its submission, each ranked row had at least this many other variants scored: LeSaturn-warppool-factorized-7.4M 43, pjgood-warp-xview-combo-10M 59, pj_better_3p7m and pj_better_10m 64, and pj_better_1m 65. These are lower bounds counted from repository records, because some run trees and lane outputs are unavailable, and they exclude readout variants, controls, evaluation seeds, PrecursorBench v1 runs and development-harness runs.
  • MantisV2 is read through the sweep's adapter, and its m0 is the sweep's released-readout rule, not independently confirmed.
  • Hill of Towie labels are inferred stops, with rules made without any score. cdip_waves_c was rebuilt because series.npy differs from the pinned file; every ranked row was recomputed on the rebuilt file.
  • The near-duplicate rule clusters SMAP series with long constant stretches although their correlation is near zero, and four UKWIND turbines had identical series that still need checking at the source (found on the series of 0.9.7 and earlier; not rechecked on the 85 UKWIND Eval series of 0.9.7.2).
  • Mantis and NuTime were pretrained on data that includes UCR and UEA training splits. The pretraining corpora of the ◊ rows are not stated on this card: the TheGem report (Forgis) says a generator of synthetic physical systems supplies all of Saturn's pretraining data, and the ◊ rows are Saturn-family models, but this card has not checked it, and whether any CAFE recording, its source system, or FactoryNet data was in their pretraining is not established for any ◊ row. This is an open item.
  • FE reads the last 512 samples and A scores windows of 256 for every model; the ◊ rows were trained on 512-sample windows (the input caps under Models), and models with longer native contexts (Chronos-2, TimesFM-2.5, Toto) are cut to 512 on FE. The rule is the same for every model; whether a fixed 512-sample input favours models trained on that length was not measured.
  • REASSEMBLE is built from a label-blind partial fetch (zip positions 0 to 78, one failed transfer; 18 Eval series); a full fetch would roughly double it, and it is the noisiest A dataset (see 0.9.7.2 beside 0.9.7).
  • 23 tabular tasks ran over the 1-hour per-dataset cap of Amendment 3 and were not retried, so those tabular rows have no CAFE score; the missing cells are compute-capped, not failures of the method.
  • Re-selection under the cap moved UKWIND__K06_b0004 (UKWIND) from Tuning into Eval: a Tuning series of 0.9.5 on which the readouts were chosen is now scored in Eval (the rule as written: hash order, Tuning = the first ceil(0.15 m)). It is the only such series of the 992.
  • The 0.9.7.2 A series files were checked against the files staged on Clariden by hand (audit 16: 321 of 321 sha256 equal); the units_sha gate of gates097.json covers the 0.9.7 units only.
  • The dataset builders ran in the environment recorded in data097/src/ENV_builders0972.txt; rebuilding UTAQ under pandas 3.1 changes 9 series by at most 4.5e-13 (audit 16), far below any score digit shown.
  • The run's event log (timing097.json) has no submit events for the 0.9.7.2 cluster jobs; for 0.9.7.2 the job record is sacct (the job table under What took how long).
  • On the Space, the tabs v0.9.6/ and v0.9.5/ were added by earlier releases and are never re-sent, so their banners still name 0.9.7 as the current release; the main page is always the current one.
  • 0.9.7.1 was never released; its candidates were folded into 0.9.7.2 (Amendment 15 item 1). The FE candidate PyrNet named for 0.9.7.1 is not in 0.9.7.2: FE is unchanged by Amendment 15 item 4.

How to submit a model

CAFE has no upload form. To propose a model, open a discussion on the dataset page with a public checkpoint pinned to a commit, its licence, code that turns an array of shape (samples, channels, T) into an embedding, with the largest T it accepts, the list of readouts it exposes, and the number of variants you scored on CAFE datasets before submitting, which the protocol publishes with the row. Forgis then runs its cells under the rules above, and the row appears with its marks in a dated version.

Datasets and models

Datasets and licences

CAFE redistributes no data.

Models

Every model is frozen.

Version history

Protocol · Changelog · leaderboard.json

Figures and tables for reports

Every figure is a file in this Space. Use the links under each one. Tables are below in a form you can copy, and each also exists as a CSV file. Version 0.9.7.2.

baselines0972
baselines0972SVG
elo0972
elo0972PNG
separability0972
separability0972PNG
The main board with intervals, one panel, drawn for this page.
The main board with intervals, one panel, drawn for this page.SVG
Heat map of C on the main board, drawn for this page. Colour runs from 0.5 or lower to the highest score on the task.
Heat map of C on the main board, drawn for this page. Colour runs from 0.5 or lower to the highest score on the task.SVG
Heat map of A on the main board, drawn for this page. Colour runs from 0.5 or lower to the highest score on the task.
Heat map of A on the main board, drawn for this page. Colour runs from 0.5 or lower to the highest score on the task.SVG
Heat map of FE on the main board, drawn for this page. Colour runs from 0.5 or lower to the highest score on the task.
Heat map of FE on the main board, drawn for this page. Colour runs from 0.5 or lower to the highest score on the task.SVG

Not offered here (figures of the released 0.9.7 page, or older than this board): baselines097.svg, elo097.png, baselines.png, cost.png, elo_cost.png, gaps.png, leaderboard.png, method_board.png, money_plot.pdf, money_plot.png, variance_breakdown.png, variance_pie.png.

Tables

Glossary

A
Anomaly detection: find the abnormal stretches in a series. Scored by VUS-PR after the prefix.
anomaly segments
Labelled runs of consecutive anomalous steps in the Eval series of an A dataset.
AUROC
The chance that a randomly picked positive case gets a higher score than a randomly picked negative one. FE uses a stratified version, compared within fold, unit and month.
balanced accuracy
Accuracy averaged over classes, so a large class does not dominate. C rescales it so that chance is 0.5.
baseline
A row with no learned encoder: ZERO on every task, and current value on FE. Shown on every board, never ranked. The random-init control and the other references are listed beside them.
best baseline
On one dataset, the highest-scoring of the baselines the board compares with.
board effect
Pairs resolved on the CAFE score without a dataset minus with it; negative means the dataset helps.
bootstrap
Drawing the test material again with replacement to see how far a score moves. CAFE uses 10,000 draws.
C
Classification: say what a recording shows. Scored by balanced accuracy.
CAFE score
The mean of the task scores that count on this release: C, A and FE.
cap
The most series a source may give an A dataset: 300 for independent runs, 100 for several series per asset (Amendment 13 item 2).
cell
One model's score on one dataset, stored with its bootstrap draws.
chance-normalised
Rescaled so that random output gives 0.5 and a perfect result gives 1.
check 2
The admission check that the label is a state of, or an event in, the physical system.
combo minus twin
C score of pjgood-warp-xview-combo-10M minus that of the random-init LeSaturn row (3.7 M parameters, another architecture; not a matched twin of the 10.6 M combo), per repeat: a model-specific check that pretraining helps on a dataset.
conflict of interest
Forgis runs CAFE and also builds some of the models it ranks. Those rows carry the mark ◊.
datasets resampled
The board recomputed 10,000 times with the datasets of each task drawn with replacement as well as the test material; the measure of the 1.0 test.
drops only
The 0.9.6 board with the datasets removed in 0.9.7 taken out and nothing added.
elapsed-time best
The better of the two data-free scores that rise or fall with time since the prefix.
elapsed-time gate
A new A dataset is admitted only if the better of the rising and falling elapsed-time scores is at most 0.65.
embedding
The vector of numbers a frozen model returns for a window. It is all a head ever sees.
Eval series
The series of an A dataset that are scored. Tuning series are kept apart.
F3 thresholds
The highest time-since-last-event and time-of-day scores on the kept FE datasets; a new FE dataset may not exceed them.
FE
Forecasting events: say whether an event will happen within the next lead time. Scored by AUROC.
foundation model
A network pretrained on many time series and then used for jobs it did not see.
free forecast
An FE reference row that reads the series without a model, such as current value or trend.
frozen
The weights are not changed for CAFE. No fine-tuning on any dataset.
GPU-hours, upper bound
Sum of task wall times over 3600; tasks share GPUs, so the GPU time used is at most this.
H0, H1, H2
The C head candidates of Amendment 12: the 0.9.5 head, the same with an 11-value penalty grid, and that after a 256-component projection fitted on the training split.
head
The small model fitted on the embedding for one task. It is the same for every model.
Holm
A step-down correction that keeps the error rate over a whole family of pair tests at the stated level.
input rules
What each encoder gets: on C the whole recording (block-averaged to 512 for fixed-length models; stretched to 512 when it is 512 samples or shorter), on A a window of 256 real samples, on FE the last 512 samples. No padding, no mask.
Kendall tau
Agreement between two orderings: 1 means the same order, 0 means no relation.
lead time
How far ahead an FE head has to warn.
Mahalanobis distance
Distance that takes account of the spread and correlation of the normal windows.
median share
Median over Eval series of the labelled share of the scored part; at most 0.3 to be admitted.
near-duplicate cluster
Eval series joined by a rule on the data alone and resampled as one block, so that copies do not narrow the interval.
node-hours
Elapsed time of a cluster job times its nodes, from sacct.
non-commercial
A licence that allows use for non-commercial purposes only, such as CC BY-NC 4.0 (UR5e).
not admitted
A candidate measured without a model and left out by a rule, with the reason recorded before any cell.
null
A score that reads no model: elapsed time rising, elapsed time falling, or a random number per window.
null floor
A dataset's highest null. A model at or below it has shown nothing the clock or chance could not.
One readout
The one readout a model uses on C, A and FE, picked by the highest validation CAFE score without reading any test value. This is the main board's rule.
pair noise
Median over all pairs of ranked models of the bootstrap SD of their paired score difference.
paired
Every model is scored on the same random draws, so a difference between two models does not depend on which material a draw picked.
pooled
Datasets of one release or paper counted as one dataset, its score the mean over all their Eval series (Amendment 13 item 2).
practically equal
A verdict: not resolved, and the 95% interval of the difference lies inside plus or minus 0.01.
prefix
The normal stretch at the start of an A series, used to fit the head.
readout
Which internal tensor of a model is taken as its embedding.
reference
A hand-made row that embeds nothing, such as max-|z| or current value. Shown, never ranked.
resolved alone
Pairs of ranked models resolved on a board made of one dataset alone.
signal-to-noise (separability)
On one dataset or task, the mean absolute score gap between two ranked models divided by the median bootstrap SD of that gap.
spread SD
SD of the ranked models' scores on one dataset or task: how far apart the models are.
tabular row
A gradient-boosted or MLP learner on hand-written statistics of the window. Shown, never ranked.
tau with the task
Kendall tau between the models' order on one dataset and their mean over the task's other datasets.
tier
Models in score order. Tier 1 holds the models above the first one resolved below the top model; the next tier starts there and is built the same way, so a model in a lower tier can still be unresolved against the top one.
Tuning series
Series of an A dataset kept out of scoring. Only readout selection reads them.
unresolved
A verdict: the data do not separate the two models, and the interval is too wide to call them equal.
VUS-PR
Volume under the precision-recall surface, the metric for A.
windows per second
Scored A windows of a model's A cells divided by their task wall time.
within seed noise
A pair of single-seed models closer than 0.009 on the CAFE score or on a task board, 1.96 times the total SD of a single-seed difference in the 0.9.2 noise budget (0.0044, of which pretraining seed is 0.0018), a normal approximation.
ZERO
A fixed random projection of the raw window with no learning. The no-encoder reference.
†
A mean over fewer datasets than the view has, because some cells do not exist in this readout mode.
◊
A model developed by Forgis with access to CAFE Eval scores.
◊ lead
Mean CAFE score of the ◊ rows minus that of the other ranked rows; reported, part of no rule.

Scores with 95% intervals

Each bar runs from chance (0.5) to the model's score on the main board. The whisker is its 95% interval, and the colour is its tier: models in the same tier are not resolved from one another. The CAFE score is the mean of C, A and FE. Baselines, references and tabular rows are unranked and drawn in grey. The row switches above the board apply here too.

CAFE score
C · classification
A · anomaly detection
FE · forecasting events

Datasets: spread and uncertainty

How far apart the ranked models are on each dataset, and how much each dataset adds to the uncertainty of the CAFE score. The CAFE score's 95% interval comes from resampling each dataset's test material. Datasets are independent, so the interval's variance splits exactly into one share per dataset. A dataset gets a large share when its test material is small or noisy, and when its task has few datasets (a dataset weighs 1/(3 × the number of datasets in its task) of the CAFE score). NEW marks a dataset added in this version.

Task
Model scores on each dataset

Each row is one dataset. The grey line spans the lowest to the highest ranked model, grey dots are the other ranked models, and the coloured dots are the tier-1 models. Sorted by the tier-1 models' range: at the top they disagree most.

Each dataset's share of the uncertainty

Share of the CAFE interval: the dataset's share of the variance behind the CAFE score's 95% interval, averaged over the ranked models. Share of tier-1 pair noise: its share of the noise in the score differences between tier-1 models. That noise decides whether they can be told apart. Click a column to sort.