This command is available with the Predictive Analytics Module. Click here for more information about how to activate the module.
Then, the importance score for the predictor equals the sum of the model improvement scores across all the trees.
where
Term | Description |
---|---|
N | sample size of the full or training data set |
w_{i} | weight for the i^{th} observation in the full or training data set |
y_{i} | i^{th} response value that is 1 for the event and 0 otherwise for the full or training data set |
predicted probability of the event for the i^{th} row in the full or training data set | |
fitted value from the model |
where
Term | Description |
---|---|
N | sample size of the full or training data |
n_{k} | sample size of fold k |
w_{i, k} | weight for the i^{th} observation in fold k |
y_{i, k} | binary response value of case i in fold k. y_{i, k} = 1 for event class, and 0 otherwise. |
predicted probability for case i in fold k. The predicted probability is from the model that does not use the data in fold k. | |
fitted value for case i in fold k. The fitted value is from the model that does not use the data in fold k. |
where
Term | Description |
---|---|
n_{Test} | sample size of the test data set |
w_{i}_{, Test} | weight for the i^{th} observation in the test data set |
y_{i}_{, Test} | binary response value of case i in fold k in the test data set. y_{i, k} = 1 for event class, and 0 otherwise. |
predicted probability for case i in the test data set | |
fitted value for case i in the test data set |
where
Term | Description |
---|---|
sample size of the full or training data set | |
w_{i} | weight for the i^{th} observation in the full or training data set |
y_{i, q} | i^{th} response value that is 1 when and 0 otherwise |
predicted probability of the q^{th} level of the response for the i^{th} row in the full or training data set | |
fitted value from the q^{th} sequence of trees for the i^{th} row, which is used to calculate the predicted probability of the q^{th} level of the response |
where
Term | Description |
---|---|
N | sample size of the training data |
n_{k} | sample size of fold k |
w_{i, k} | weight for the i^{th} observation in fold k |
y_{i, k, q} | i^{th} response value of case i in fold k that is 1 when and 0 otherwise. |
The predicted probability of the q^{th} level of the response for the i^{th} row in fold k. The predicted probability is from the model that does not use the data in fold k. | |
The fitted value from the q^{th} sequence of trees for the i^{th} row in fold k, which is used to calculate the predicted probability of the q^{th} level of the response. The fitted value is from the model that does not use the data in fold k. |
where
Term | Description |
---|---|
n_{Test} | sample size of the test data |
w_{i, }_{Test} | weight for the i^{th} observation in the test data |
y_{i, }_{Test, }_{q} | i^{th} response value of case i in the test data set that is 1 when and 0 otherwise. |
The predicted probability of the q^{th} level of the response for the i^{th} row in the test data. The predicted probability is from the model that does not use the test data. | |
The fitted value for the q^{th} sequence of trees for the i^{th} row in the test data, which is used to calculate the predicted probability of the q^{th} level of the response. The predicted probability is from the model that does not use the test data. |
where k is the number of distinct event probabilities and (x_{0}, y_{0}) is the point (0, 0).
To compute the area for a curve from a test data set or from cross-validated data, use the points from the corresponding curve.
Term | Description |
---|---|
TPR | true positive rate |
FPR | false positive rate |
TP | true positive, events that were correctly assessed |
FN | false negative, events that were incorrectly assessed |
P | number of actual positive events |
FP | false positive, nonevents that were incorrectly assessed |
N | number of actual negative events |
FNR | false negative rate |
TNR | true negative rate |
x (false positive rate) | y (true positive rate) |
---|---|
0.0923 | 0.3051 |
0.4154 | 0.7288 |
0.7538 | 0.9322 |
1 | 1 |
The following interval gives the upper and lower bounds for the confidence interval:
The computation of the standard error of the area under the ROC curve () comes from Salford Predictive Modeler^{®}. For general information about estimation of the variance of the area under the ROC curve, see the following references:
Engelmann, B. (2011). Measures of a ratings discriminative power: Applications and limitations. In B. Engelmann & R. Rauhmeier (Eds.), The Basel II Risk Parameters: Estimation, Validation, Stress Testing - With Applications to Loan Risk Management (2nd ed.) Heidelberg; New York: Springer. doi:10.1007/978-3-642-16114-8
Cortes, C. and Mohri, M. (2005). Confidence intervals for the area under the ROC curve. Advances in neural information processing systems, 305-312.
Feng, D., Cortese, G., & Baumgartner, R. (2017). A comparison of confidence/credible interval methods for the area under the ROC curve for continuous diagnostic tests with small sample size. Statistical Methods in Medical Research, 26(6), 2603-2621. doi:10.1177/0962280215602040
Term | Description |
---|---|
A | area under the ROC curve |
0.975 percentile of the standard normal distribution |
To see general calculations for cumulative lift, go to Methods and formulas for the lift chart for Fit Model and Discover Key Predictors with TreeNet® Classification.
In the weighted case, use weighted counts in place of counts.
For k-fold cross-validation, the misclassed count is the sum of the misclassifications from when each fold is the test data set.
For validation with a test data set, the misclassed count is the sum of misclassifications in the test data set and the total count is for the test data set.