This article separates probability estimation from the scientific decision rule. gp3ml does not treat 0.5 as a universally justified decision threshold. Thresholds must be predeclared or selected using analysis/inner-resampling data, never an outer assessment or independent external-validation set.
truth <- factor(rep(c("pass", "review"), 20), levels = c("pass", "review"))
probability <- seq(0.05, 0.95, length.out = 40)
evaluation <- evaluate_gazepoint_thresholds(
truth = truth,
probability = probability,
positive = "review",
thresholds = seq(0.2, 0.8, by = 0.05)
)
rule <- select_gazepoint_threshold(
evaluation,
metric = "balanced_accuracy",
direction = "maximize",
generalization_target = "new_participants",
scientific_justification =
"Balance sensitivity and specificity for predefined recording-quality review status."
)
validate_gazepoint_decision_rule(rule, require_threshold = TRUE)
#> $status
#> [1] "pass"
#>
#> $checks
#> check status detail
#> 1 class pass
#> 2 metric pass
#> 3 direction pass
#> 4 threshold pass
#> 5 threshold_origin pass
#> 6 training_partition pass
#> 7 generalization_target pass
#> 8 scientific_justification pass
#> 9 abstention pass
#>
#> attr(,"class")
#> [1] "gp3ml_decision_rule_validation"
plot(evaluation)
An abstention interval can be declared when the scientific protocol permits withholding a forced classification. Abstentions must be reported explicitly, including coverage and error among non-abstained predictions.