RunMat
GitHub

perfcurve — Compute classifier performance curves and AUC values.

perfcurve(labels,scores,posclass) treats posclass as the positive class, all other nonmissing classes as negative classes, and returns ROC false-positive-rate values. [X,Y,T,AUC,OPTROCPT,SUBY,SUBYNAMES] = perfcurve(...) returns the Y criterion, thresholds, trapezoidal AUC, optimal ROC point, per-negative-subclass Y values, and subclass labels as a cell array.

Syntax

X = perfcurve(labels, scores, posclass)
X = perfcurve(labels, scores, posclass, Name, Value)
[X,Y,T,AUC,OPTROCPT,SUBY,SUBYNAMES] = perfcurve(___)

Inputs

NameTypeRequiredDefaultDescription
labelsAnyYes—True class labels.
scoresNumericArrayYes—Classifier scores for the positive class.
posclassAnyYes—Positive class label.
nameValuePairsAnyVariadic—Name-value options such as XCrit, YCrit, NegClass, Weights, Prior, Cost, TVals, XVals, UseNearest, and ProcessNaN.

Returns

NameTypeDescription
XNumericArrayX-coordinate criterion values for the performance curve.
YNumericArrayY-coordinate criterion values for the performance curve.
TNumericArrayScore thresholds used to compute the curve.
AUCNumericScalarArea under the X/Y performance curve.
OPTROCPTNumericArrayOptimal operating point on the ROC curve.
SUBYNumericArrayY-criterion values computed for each negative subclass.
SUBYNAMESAnyNegative subclass labels corresponding to columns of SUBY.

Returned values from perfcurve depend on how many outputs the caller requests.

Errors

IdentifierWhenMessage
RunMat:perfcurve:InvalidArgumentLabels, scores, class selections, criteria, thresholds, weights, priors, costs, or name-value options are malformed.perfcurve: invalid argument
RunMat:perfcurve:UnsupportedA documented advanced mode requires confidence bounds, bootstrap/cross-validation outputs, or score matrix class-column inference that is not implemented yet.perfcurve: unsupported mode
RunMat:perfcurve:InternalRunMat cannot allocate or construct perfcurve outputs.perfcurve: internal error

How perfcurve works

  • Labels may be numeric, logical, string, character rows, categorical values, or cell strings. Text label families can be mixed between string, char, cell-string, and categorical inputs.
  • scores must be a numeric vector with one score per label. Score matrices for class-column inference are rejected explicitly until the classifier object/class-order surface is available.
  • Rows with missing labels, zero weights, or NaN scores are omitted by default. "ProcessNaN","addtofalse" keeps NaN scores: positive-class NaN scores are counted as false negatives and negative-class NaN scores as false positives at every threshold.
  • Default thresholds include an initial reject-all point whose threshold duplicates the highest score, followed by each unique finite score in descending order. "TVals",t uses explicit thresholds and cannot be combined with "XVals".
  • "XCrit" and "YCrit" support common criteria: "fpr", "tpr"/"sens"/"reca", "tnr"/"spec", "fnr"/"miss", "ppv"/"prec", "npv", "accu", "tp", "fn", "fp", "tn", "tp+fp", "rpp", "rnp", and "ecost".
  • "NegClass",labels restricts the negative class set. SUBY contains one Y-criterion column per selected negative subclass and SUBYNAMES returns the corresponding labels as an n-by-1 cell array.
  • "Weights",w applies nonnegative observation weights. "Prior" accepts "empirical", "uniform", or a two-element numeric positive/negative prior vector and rescales class weights before the curve is computed.
  • "Cost",C accepts a nonnegative 2-by-2 cost matrix and is used for "ecost" and when choosing OPTROCPT for standard ROC curves. OPTROCPT is [NaN NaN] for non-ROC criteria.
  • "XVals",x returns curve values at requested X positions and cannot be combined with "TVals". "UseNearest" defaults to true; nearest mode returns the nearest actual X values in requested order, while "UseNearest",false sorts requested X values and linearly interpolates between finite X/Y points. AUC is computed over the full curve interval covered by XVals.
  • Confidence-bound and bootstrap modes such as "NBoot" are not implemented yet and fail with a dedicated unsupported-mode error rather than returning placeholder intervals.

Examples

Compute a basic ROC curve and AUC

labels = [1; 0; 1; 0];
scores = [0.9; 0.8; 0.4; 0.1];
[X,Y,T,AUC] = perfcurve(labels, scores, 1)

Expected output:

AUC is 0.75 for this ranking.

Compute precision-recall values at explicit thresholds

[R,P] = perfcurve(labels, scores, 1, "XCrit", "reca", "YCrit", "prec", "TVals", [0.8 0.4])

Restrict negative classes

[X,Y,T,AUC,opt,suby,names] = perfcurve(labels, scores, "hit", "NegClass", "miss")

Using perfcurve with coding agents

Open a RunMat example with live inputs, then ask the agent to explain how perfcurve changes the result.

Run a small perfcurve example, explain the result, then change one input and compare the output.

FAQ

Does perfcurve train a classifier?⌄

No. It evaluates existing classifier scores against known labels. Train or evaluate a model separately and pass the score vector for the positive class.

Why are confidence intervals not returned?⌄

RunMat currently implements deterministic curve, threshold, AUC, optimal-point, weighting, prior, cost, and subclass outputs. Bootstrap confidence-bound modes require a separate statistical resampling implementation and are rejected explicitly.

Options

statget · statset

Open-source implementation

Unlike proprietary runtimes, every RunMat function is open-source. Read exactly how perfcurve is executed, line by line, in Rust.

About RunMat

RunMat is an open-source runtime that executes MATLAB-syntax code blazing on any GPU. It is licensed under the Apache 2.0 license.

  • RunMat automatically optimizes your math for GPU execution on Apple, Nvidia, and AMD hardware. No code changes needed. Simulations that took hours now take minutes.
  • Start running code in seconds. RunMat runs in the browser, on the desktop, or from the CLI. No license server, no IT ticket.

Getting started · Benchmarks · Pricing

Download RunMat

Download RunMat for full performance, or use RunMat in your browser for zero setup.

Download RunMatOpen Sandbox