auc

Reason of having high AUC and low accuracy in a balanced dataset

由 余生颓废 提交于 2020-01-24 07:27:06
问题 Given a balanced dataset (size of both classes are the same), fitting it into an SVM model I yield a high AUC value (~0.9) but a low accuracy (~0.5). I have totally no idea why would this happen, can anyone explain this case for me? 回答1: I recently stumbled upon the same question. Here is what I figured out for myself - let me know if I'm wrong. Before we ponder why the area under the ROC curve (AUC) can be high while accuracy is low, let's first recapitulate the meanings of these terms. The

Reason of having high AUC and low accuracy in a balanced dataset

由 核能气质少年 提交于 2020-01-24 07:26:26
问题 Given a balanced dataset (size of both classes are the same), fitting it into an SVM model I yield a high AUC value (~0.9) but a low accuracy (~0.5). I have totally no idea why would this happen, can anyone explain this case for me? 回答1: I recently stumbled upon the same question. Here is what I figured out for myself - let me know if I'm wrong. Before we ponder why the area under the ROC curve (AUC) can be high while accuracy is low, let's first recapitulate the meanings of these terms. The

推荐系统评价指标：AUC和GAUC

由 白昼怎懂夜的黑 提交于 2020-01-21 23:59:03
AUC是推荐系统中最常用的模型评价指标。基础概念要常看常新，最近复习了一遍AUC的概念，在此做个笔记。本文力求简洁系统地理解AUC的概念和计算方法，AUC在推荐/广告领域的局限性以及解决这一问题的另一个指标：Group AUC(GAUC) 1. 分类任务与混淆矩阵 认识auc的第一步，是看懂混淆矩阵： 预测\真实 1 0 1 TP FP 0 FN TN True/False代表预测的正确/错误； Positive/Negative代表预测值为1/0. TP是真1；FP是假1；FN是假0； TN是真0。 真阳率： T P R = T P T P + F N TPR = \frac{TP}{TP+FN} T P R = T P + F N T P ​ ，正样本被预测为1的概率； 假阳率： F P R = F P F P + T N FPR = \frac{FP}{FP+TN} F P R = F P + T N F P ​ ，负样本被预测为1的概率； 2. ROC曲线与AUC 以x轴为FPR, y轴为TPR，做出图称为ROC曲线 AUC的定义：Area Under ROC Curve，即ROC曲线下的面积 AUC的意义：随机抽取一对正负样本，AUC是 把正样本预测为1的概率大于把负样本预测为1的概率的概率 。这句话有点拗口，用公式写就是： A U C = P ( P 正 > P 负 )

rWeka how to calculate ROC AUC?

由 坚强是说给别人听的谎言 提交于 2020-01-16 16:49:30
问题 I am using rWeka package to compare the performance of different machine learning algorithms such as: # KNN: (resultIBk <- IBk(postScore~., data_train)) # Naive Bayes: NB <- make_Weka_classifier("weka/classifiers/bayes/NaiveBayes") # Default settings Weka (resultNB <- NB(postScore~., data_train)) # Decision Tree J48 (resultJ48 <- J48(postScore~., data_train)) Can anyone please advise on how to calculate ROC AUC for the various machine learning algorithms in Weka? I understand that for

rWeka how to calculate ROC AUC?

由 守給你的承諾、 提交于 2020-01-16 16:49:17
问题 I am using rWeka package to compare the performance of different machine learning algorithms such as: # KNN: (resultIBk <- IBk(postScore~., data_train)) # Naive Bayes: NB <- make_Weka_classifier("weka/classifiers/bayes/NaiveBayes") # Default settings Weka (resultNB <- NB(postScore~., data_train)) # Decision Tree J48 (resultJ48 <- J48(postScore~., data_train)) Can anyone please advise on how to calculate ROC AUC for the various machine learning algorithms in Weka? I understand that for

How to calculate AUC for One Class SVM in python?

由 爷，独闯天下 提交于 2020-01-13 05:59:08
问题 I have difficulty in plotting OneClassSVM's AUC plot in python (I am using sklearn which generates confusion matrix like [[tp, fp],[fn,tn]] with fn=tn=0 . from sklearn.metrics import roc_curve, auc fpr, tpr, thresholds = roc_curve(y_test, y_nb_predicted) roc_auc = auc(fpr, tpr) # this generates ValueError[1] print "Area under the ROC curve : %f" % roc_auc plt.plot(fpr, tpr, label='ROC curve (area = %0.2f)' % roc_auc) I want to handle error [1] and plot AUC for OneClassSVM . [1] ValueError:

How to calculate AUC with tensorflow?

由 丶灬走出姿态 提交于 2020-01-12 03:38:50
问题 I've built a binary classifier using Tensorflow and now I would like to evaluate the classifier using AUC and accuracy. As far as accuracy is concerned, I can easily do like this: X = tf.placeholder('float', [None, n_input]) y = tf.placeholder('float', [None, n_classes]) pred = mlp(X, weights, biases, dropout_keep_prob) correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1)) accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float")) When calculating AUC I use the following:

How to calculate AUC with tensorflow?

由 廉价感情. 提交于 2020-01-12 03:38:07
问题 I've built a binary classifier using Tensorflow and now I would like to evaluate the classifier using AUC and accuracy. As far as accuracy is concerned, I can easily do like this: X = tf.placeholder('float', [None, n_input]) y = tf.placeholder('float', [None, n_classes]) pred = mlp(X, weights, biases, dropout_keep_prob) correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1)) accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float")) When calculating AUC I use the following:

AUC，准确率，敏感性，特异性计算

由 被刻印的时光 ゝ 提交于 2020-01-11 06:44:31
import numpy as np from sklearn.metrics import confusion_matrix,accuracy_score,f1_score,roc_auc_score,recall_score,precision_score def calculate_metric(gt, pred): pred[pred>0.5]=1 pred[pred<1]=0 confusion = confusion_matrix(gt,pred) TP = confusion[1, 1] TN = confusion[0, 0] FP = confusion[0, 1] FN = confusion[1, 0] print('Accuracy:',(TP+TN)/float(TP+TN+FP+FN)) print('Sensitivity:',TP / float(TP+FN)) print('Specificity:',TN / float(TN+FP)) path = "OS3_val_result.csv" with open(path, 'r') as f: file_list = f.read() file_list = file_list.split('\n')[1:-1] file_list = [file.split(',') for file in

python画ROC曲线

由 穿精又带淫゛_ 提交于 2020-01-08 10:50:30
import numpy as np from sklearn import metrics from matplotlib import pyplot path = "PFS3_val_result.csv" with open(path, 'r') as f: file_list = f.read() file_list = file_list.split('\n')[1:-1] file_list = [file.split(',') for file in file_list] y = np.array([float(y[1]) for y in file_list]) scores1 = np.array([float(preds[2]) for preds in file_list]) scores2 = np.array([float(preds[3]) for preds in file_list]) scores3 = np.array([float(preds[4]) for preds in file_list]) fpr1, tpr1, thresholds1 = metrics.roc_curve(y, scores1, pos_label=1) fpr2, tpr2, thresholds2 = metrics.roc_curve(y, scores2,
订阅 auc