Table 5 Inter-rater agreement for single video data, subgroup analyses
Effect of BMI on LVEF estimation. BMI ≥ 30 or BMI < 30 | ||||
|---|---|---|---|---|
Observation | Rater 1 of LVEF | Rater 2 of LVEF | ICC (95% CI) BMI ≥ 30 (n = 80) | ICC (95% CI) BMI < 30 (n = 260) |
1 | ML model | Level III expert visual LVEF on randomized single videos | 0.813 (0.247, 1.000) | 0.749 (0.740, 1.000) |
2 | ML model | Level III expert segmentation LVEF on randomized single videos | 0.829 (0.165, 1.000) | 0.709 (0.098, 1.000) |
3 | ML model | Level III expert visual LVEF, accounting for all videos for a participant | 0.822 (0.129, 0.999) | 0.771 (0.551, 1.000) |
4 | ML model | Level III expert segmentation LVEF, accounting for all videos for a participant | 0.909 (0.481, 1.000) | 0.802 (0.243, 1.000) |
5 | ML model | Derived LVEF from echo reports | 0.870 (0.610, 1.000) | 0.741 (0.071, 1.000) |
Effect of sex on LVEF estimation: male or female | ||||
|---|---|---|---|---|
Observation | Rater 1 of LVEF | Rater 2 of LVEF | ICC (95% CI) male (n = 293) | ICC (95% CI) female (n = 49) |
1 | ML model | Level III expert visual LVEF on randomized single videos | 0.693 (0.089, 1.000) | 0.901 (0.520, 1.000) |
2 | ML model | Level III expert segmentation LVEF on randomized single videos | 0.705 (0.073, 1.000) | 0.869 (0.293, 1.000) |
3 | ML model | Level III expert visual LVEF, accounting for all videos for a participant | 0.740 (0.067, 0.999) | 0.877 (0.503, 1.000) |
4 | ML model | Level III expert segmentation LVEF, accounting for all videos for a participant | 0.796 (0.176, 1.000) | 0.901 (0.477, 1.000) |
5 | ML model | Derived LVEF from echo reports | 0.758 (0.131, 1.000) | 0.859 (0.279, 1.000) |
Effect of atrial fibrillation (AF) or atrial flutter (AFL) on LVEF estimation | ||||
|---|---|---|---|---|
Observation | Rater 1 of LVEF | Rater 2 of LVEF | ICC (95% CI) AF or AFL (n = 108) | ICC (95% CI) Non-AF or non-AFL (n = 234) |
1 | ML Model | Level III expert visual LVEF on randomized single videos | 0.684 (-0.143, 1.000) | 0.809 (0.346, 1.000) |
2 | ML Model | Level III expert segmentation LVEF on randomized single videos | 0.596 (-0.067, 0.999) | 0.829 (0.135, 0.999) |
3 | ML Model | Level III expert visual LVEF, accounting for all videos for a participant | 0.708 (0.182, 1.000) | 0.831 (0.210, 1.000) |
4 | ML model | Level III expert segmentation LVEF, accounting for all videos for a participant | 0.823 (0.350, 1.000) | 0.860 (0.428, 1.000) |
5 | ML Model | Derived LVEF from echo reports | 0.673 (-0.043, 1.000) | 0.841 (0.271, 1.000) |
Effect of scanner on LVEF estimation: physician (MD) vs Nurse Practitioner (N) | ||||
|---|---|---|---|---|
Observation | Rater 1 of LVEF | Rater 2 of LVEF | ICC (95% CI) MD (n = 95) | ICC (95% CI) N (247) |
1 | ML model | Level III expert visual LVEF on randomized single videos | 0.582 (−0.015, 0.997) | 0.800 (0.154, 1.000) |
2 | ML model | Level III expert segmentation LVEF on randomized single videos | 0.649 (−0.028, 0.999) | 0.810 (0.373, 1.000) |
3 | ML model | Level III expert visual LVEF, accounting for all videos for a participant | 0.550 (−0.020, 0.997) | 0.845 (0.784, 1.000) |
4 | ML model | Level III expert segmentation LVEF, accounting for all videos for a participant | 0.569 (−0.015, 0.995) | 0.884 (0.490, 1.000) |
5 | ML model | Derived LVEF from echo reports | 0.657 (−0.003, 0.998) | 0.840 (0.715, 1.000) |