precision-recall

问题 I trained a supervised model in FastText using the Python interface and I'm getting weird results for precision and recall. First, I trained a model: model = fasttext.train_supervised("train.txt", wordNgrams=3, epoch=100, pretrainedVectors=pretrained_model) Then I get results for the test data: def print_results(N, p, r): print("N\t" + str(N)) print("P@{}\t{:.3f}".format(1, p)) print("R@{}\t{:.3f}".format(1, r)) print_results(*model.test('test.txt')) But the results are always odd, because

问题 I'm working in a very unbalanced classification problem, and I'm using AUPRC as metric in caret. I'm getting very differents results for the test set in AUPRC from caret and in AUPRC from package PRROC. In order to make it easy, the reproducible example uses PimaIndiansDiabetes dataset from package mlbench: rm(list=ls()) library(caret) library(mlbench) library(PRROC) #load data, renaming it to 'datos' data(PimaIndiansDiabetes) datos=PimaIndiansDiabetes[,1:9] # training and test set.seed(998)

问题 I'm working in a very unbalanced classification problem, and I'm using AUPRC as metric in caret. I'm getting very differents results for the test set in AUPRC from caret and in AUPRC from package PRROC. In order to make it easy, the reproducible example uses PimaIndiansDiabetes dataset from package mlbench: rm(list=ls()) library(caret) library(mlbench) library(PRROC) #load data, renaming it to 'datos' data(PimaIndiansDiabetes) datos=PimaIndiansDiabetes[,1:9] # training and test set.seed(998)

问题 I'm working in a very unbalanced classification problem, and I'm using AUPRC as metric in caret. I'm getting very differents results for the test set in AUPRC from caret and in AUPRC from package PRROC. In order to make it easy, the reproducible example uses PimaIndiansDiabetes dataset from package mlbench: rm(list=ls()) library(caret) library(mlbench) library(PRROC) #load data, renaming it to 'datos' data(PimaIndiansDiabetes) datos=PimaIndiansDiabetes[,1:9] # training and test set.seed(998)

问题 I'm working in a very unbalanced classification problem, and I'm using AUPRC as metric in caret. I'm getting very differents results for the test set in AUPRC from caret and in AUPRC from package PRROC. In order to make it easy, the reproducible example uses PimaIndiansDiabetes dataset from package mlbench: rm(list=ls()) library(caret) library(mlbench) library(PRROC) #load data, renaming it to 'datos' data(PimaIndiansDiabetes) datos=PimaIndiansDiabetes[,1:9] # training and test set.seed(998)

问题 I try to calculate the f1_score but I get some warnings for some cases when I use the sklearn f1_score method. I have a multilabel 5 classes problem for a prediction. import numpy as np from sklearn.metrics import f1_score y_true = np.zeros((1,5)) y_true[0,0] = 1 # => label = [[1, 0, 0, 0, 0]] y_pred = np.zeros((1,5)) y_pred[:] = 1 # => prediction = [[1, 1, 1, 1, 1]] result_1 = f1_score(y_true=y_true, y_pred=y_pred, labels=None, average="weighted") print(result_1) # prints 1.0 result_2 = f1

问题 I try to calculate the f1_score but I get some warnings for some cases when I use the sklearn f1_score method. I have a multilabel 5 classes problem for a prediction. import numpy as np from sklearn.metrics import f1_score y_true = np.zeros((1,5)) y_true[0,0] = 1 # => label = [[1, 0, 0, 0, 0]] y_pred = np.zeros((1,5)) y_pred[:] = 1 # => prediction = [[1, 1, 1, 1, 1]] result_1 = f1_score(y_true=y_true, y_pred=y_pred, labels=None, average="weighted") print(result_1) # prints 1.0 result_2 = f1

问题 I was wondering how sklearn decides how many thresholds to use in precision_recall_curve. There is another post on this here: How does sklearn select threshold steps in precision recall curve?. It mentions the source code where I found this example import numpy as np from sklearn.metrics import precision_recall_curve y_true = np.array([0, 0, 1, 1]) y_scores = np.array([0.1, 0.4, 0.35, 0.8]) precision, recall, thresholds = precision_recall_curve(y_true, y_scores) which then gives >>>precision

来源： https://stackoverflow.com/questions/46518041/what-evaluation-metric-should-i-use-to-compare-knowledge-based-approach-with-gen

来源： https://stackoverflow.com/questions/46518041/what-evaluation-metric-should-i-use-to-compare-knowledge-based-approach-with-gen