Detailed results

Sensor strips, end to end, all 755 held-out strips

The 32-beat window refuses strips below about 66 beats per minute (fewer than 33 beats in 30 s); the 16-beat window removes that. Remaining refusals are strips whose engine beats all exceed 200 per minute (artefacts) and strips in the probability band.

Clean strips only (expert-flagged artefacts removed, 424 strips), 32-beat model, per strip: Se 94.5 %, Sp 99.2 %, PPV 98.7 %. On the same 424 strips the previous model’s delivered output gives Se 99.4 %, Sp 96.1 %, PPV 94.3 %, but all 47 patients were in its training set.

Where the inconclusive strips come from. Of the 240 refusals, 199 have no plausible window: 139 are expert-flagged artefacts, 24 strips are only about 10 s long (this set mixes 10-s and 30-s recordings; a 16-beat window cannot exist in 10 s below ~100 bpm), and the rest are unflagged artefacts (saturated signal, spike trains at 240 “beats” per minute). Restricted to recordings of at least 25 s that the expert did not flag, the product rule gives:

On readable 30-s strips the inconclusive rate is 14 %, inside the proposed 15 % limit.

Confidence intervals, held-out sensor strips (product configuration)

Model 9, 16-beat windows, AF if the highest window probability is at least 0.8, not AF if at most 0.5. All 755 held-out strips of 51 patients; 12 patients contribute AF strips. Wilson and Clopper-Pearson intervals treat strips as independent; the cluster bootstrap resamples patients (2,000 draws) and is the honest interval when strips of one patient are correlated.

The strip-level lower bounds meet the proposed acceptance limits (sensitivity >= 90 %, specificity >= 95 %). The patient-level bounds are wide because only 12 held-out patients have AF; the pivotal study needs on the order of 100 subjects with AF and 100 without.

MIT-BIH AF Database, EC57, RR features only

Reference: 293 episodes in 23 records. False episodes have a median length of 33 s; 63 of 410 exceed one minute, two exceed five minutes. Positive predictivity restricted to detected episodes of at least 60 s is 83.9 %. The false windows are statistically indistinguishable from AF on RR intervals (irregularity index 0.65 vs 0.68, RR CV 0.205 vs 0.208) and sit in eight records with frequent ectopy; the pending engine-native test adds P-wave, QRS and quality features to this benchmark.

Cross-check of the EC57 scorer against the reference WFDB tools

Our episode and duration statistics are computed by a Python implementation of the EC57 definitions. To verify it, the model-8 AFDB episodes were written as WFDB rhythm annotations and scored with epicmp, the reference program of the WFDB software package (built from source on the analysis machine), gross statistics by sumstats.

Agreement within about one point on every statistic; the episode-count differences come from how episodes touching each other or the record boundaries are merged. Submission numbers will be taken from epicmp.

Sensitivity by heart-rate band (window level)

Beat detection, unchanged engine, MIT-BIH Arrhythmia Database

bxb rule, 150 ms window, all reference beats counted: Se 98.1 %, +P 98.7 % (109 176 reference beats).

Previous model, same scoring

Per-beat Random Forest as delivered, on MIT-BIH which was in its training set: EC57 episode 91.6 / 58.6, duration 92.9 / 86.5.

External comparison: ArNet2 (Technion AIM lab, pretrained, CC BY-NC, comparison only)

ArNet2 is a deep network on 60-beat RR windows trained on about 51,000 hours of private Holter data. Run with its own runner on the AFDB beat annotations and scored with the same EC57 procedure (consecutive positive windows merged):

At matched duration sensitivity ArNet2 holds 3-4 points more duration precision and produces far fewer, longer episodes, at the cost of missing most episodes under 60 seconds. Its window specificity on AFDB, 98.6 %, is the level the morphology features are expected to bring ours to. Whether AFDB was part of its development data is not stated. The raw-ECG model of the same toolkit could not be run: its exported graph fails on a public record under the repository’s own runner.

Work in progress, night of 15 September (preliminary, will be replaced)

PhysioNet/CinC 2017, 300-record validation sample, frozen rule (max window probability, 0.8 / 0.5). Single-lead 9-61 s strips from a consumer device, labels normal / AF / other / noisy. Sample used for development; the 8,528-record set is being processed as the test set.

Overall on the three interpretable classes: Se 87.2 % (95 % CI 73-94), Sp 97.9 % (95-99), inconclusive 15.6 %. Sensitivity is below the 95 % target on this sample, so every error was reviewed on the waveform:

• Two of the five misses are questionable labels: one is a bigeminal rhythm (every second beat early, identical pattern throughout) labelled AF; one is a regular rhythm at 70 bpm without clear P waves. One miss is a poor-quality strip with 37 % of the signal marked out-of-signal by the engine; that strip should be inconclusive, not “not AF” (a quality gate is being added). Two misses are genuine: irregular rhythms where the model’s probability stayed at 0.5.
• Of the four false alarms, two are “regularly irregular” rhythms (bigeminy-like alternation of short and long RR intervals), one is sinus arrhythmia with a very noisy baseline, and one is a mostly regular strip whose last two windows were irregular.

Figure. A CinC 2017 strip labelled AF; the RR intervals alternate 970 / 600 ms with two beat morphologies, a bigeminal pattern. The model correctly did not call AF.

Figure. A genuine miss: irregular RR intervals without P waves; the window probabilities reached only 0.48.

Aggregating windows by median or mean instead of maximum removed all false alarms but cut sensitivity to about 70 %, so the maximum rule stays. Actions taken tonight: (1) two new RR features that detect regularly-irregular patterns (median RR difference at lag 2 and lag 3 relative to lag 1: about 0 for bigeminy or trigeminy, about 1 for AF; on this sample the AF median is 0.92 against 1.50 for normal and 1.35 for other rhythms); (2) a quality gate that routes strips with out-of-signal windows to inconclusive; (3) retraining with these features (experiment 11) and an experiment that adds the previous model’s per-beat AF output as a feature (experiment 10), both scored on the held-out sensor strips, the CinC sample, the full CinC set and AFDB with engine beats.

Previous model’s own rhythm output on AFDB (engine run on 10,000 of 14,041 minutes so far). EC57 episode Se 93.7 %, +P 74.3 %; duration Se 43.8 %, +P 84.3 %. Inside long AF the engine’s state machine relabels most of the time as “sinus arrhythmia” or “irregular”; in the two all-AF records only 11-13 % of the AF time is labelled AF. Final numbers follow when the run completes.