Skip to content
National Cancer Institute National Cancer Institute U.S. National Institutes of Health
Division of Cancer Prevention logo
Home Site Map Contact DCP
Programs & Resources

Biometry Research Group

Statistical Software

Evaluating a new marker for risk prediction: A review of medical decision-making approaches involving risk thresholds

Stuart G. Baker [wrote software]
Ben Van Calster,
Ewout W. Steyerberg

The program requires Mathematica 8.0 Exit Disclaimer

Key Function


Format for data

(1) for individual data with binary outcomes,
data={mat ,Model 1","Model 2","data set name","indiv"};
ind is a list of indicators of disease or not,
r1 is a list of risks under Model 1,
r2 is a list of risks under Model 2,

(2) for individual data with survival outcomes
data={mat,"Model 1", "Model 2"," data set name","surv"};
times is a list of survival times,
cens =1 if (right) censored and 0 if not censored
targettime is time evaluate survival
r1 is a list of risks under Model 1
r2 is a list of risks under Model 2,


Prev::usage="Automatic or probability of developing disease"
MaxBoot::usage="maximum number of bootstrap iterations"
RiskScore::usage="either basic or recal for binary; only basic for survival"
TwoStage::usage="True use intermediate category; False do not use"
TwoStageLowerBound::usage="low risk for Model 1 intermediate category"
TwoStageUpperBound::usage="high risk for Model 1 intermeidiate category"
NumberOfIntervals::usage="number of intervals for observed estimate"

(*risk thresholds*)
RiskThreshold::usage="Automatic or list of risk thresholds"
RelevantRegion::usage="Right or Left if RiskThreshold is Automatic"

PlotVerticalRange::usage="fixed or varied"
PlotType::usage="RU or DC for relative utility or decision curve"
PlotVerticalLines::usage="False or True for vertical lines for risk thresholds"
ExportName::usage="export name added to plot for the analysis"

ShowTables::usage="True or False"
ShowDetails::usage="True or False"

To run provided example:

(1)To input files
copy all files into some folder  "FOLDER"
start a new Mathematica session
evaluate SetDirectory["FOLDER"]
- evaluate << rucall.m

(2)To reproduce bootstrap calculations in manuscript type

(3) To reproduce figures in manuscript type


Download All (zip 34kb)

File name / sizeDescription
rucall.m (M File, 8KB)initial package for reproducing results; calls in ruc.m
ruc.m (M File, 8KB)basic package; calls other packages.
ruccore.m (M File, 12KB)predicted, observed, hybrid estimation
rucplot.m (M File, 6KB)plots for ROC curve, RU curve, decision curve
rucdru.m (M File, 2KB)difference in relative utility
ruccalibrate.m (M File, 4KB)calibration plots, refit for recalibration
rucperson.m (M File, 21KB)computations for individual data (binary or survival outcomes)
rucdata.m (M File, 9KB)generate data sets
ructhreshold.m (M File, 3KB)generate automatic risk thresholds (if desired)
rucdataset1.m (M File, 23KB)raw data for data set 1 for manuscript (import text file into rucdata.m)
rucmanufig.m (M File, 13KB)figures based on hypothetical data for illustration in paper

Important Note

This code is provided "as is", without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose and noninfringement. In no event shall the NCI or the individual developers be liable for any claim, damages or other liability of any kind. Use of this code by recipient is at recipient's own risk. NCI makes no representations that the use of the code will not infringe any patent or proprietary rights of third parties.

Data Sources

The data were obtained from the following studies.

Data set 1 is from Vergouwe Y, Steyerberg EW, Foster RS, Sleijfer DT, Fosså SD, Gerl A, de Wit R, Roberts JT, Habbema JD. Predicting retroperitoneal histology in postchemotherapy testicular germ cell cancer: a model update and multicentre validation with more than 1000 patients. Eur Urol. 2007 Feb;51(2):424-32

Data set 2 is simulated individual survival outcomes

Last updated: November 29, 2011

Back to top