F1分数可以衡量模型对少类用户的预测效果。

实际上，遇到unbalanced data，就是一种分类下小的可怜，例如5%的，1%的，理解Precision、F1-Score、KS的作用，关键在于\(FP\)， 具体混淆矩阵可以看这里， KS函数 - A Hugo website。

直接说干货。 比如，一个分类下，\(y=1\)占有99%的样本，拍脑袋的话，全部\(\hat y = 1\)也有99%的\(Acc\)了。

\[Acc = \frac{TP+TN}{TP+FN+TN+FP}\]

但是，这么中和下这种偏度呢？ 关键之处是，\(FP\)太多了，太多的\(TN\)被搞成了\(FP\)，因此\(FP \uparrow\)。

就是Precision含有这一项，

\[Precision = \frac{TP}{TP+FP}\]，当\(FP \uparrow\)时，显然在下降了。

另外，F1-Score是Recall和Precision的调和平均数，

\[F1= \frac{1}{\frac{1}{Recall} + \frac{1}{Precision}} = \frac{1}{\frac{1}{\frac{TP}{TP+FN}} + \frac{1}{\frac{TP}{TP+FP}}}\]

有句话说得好，Recall更客观一些，描述了总的\(P\)，一共有多少被预测了，因为\(FN\)就是锅啊，被预测错了。 然而，Precision更主观一些，描述了总的\(P\)，模型有多自信能够预测成\(P\)，因为\(FP\)衡量了模型太暴力了，多预测了\(P\)的代价，所以不要大力出奇迹。

显然KS为

\[KS = \frac{TP}{TP+FN} - \frac{FP}{FP+TN}\]

\(FP \uparrow\)时，KS下降，因此KS也能抓取这种暴力模式。

---

• 机器学习中分类器的评价指标：召回率(recall), 精度(precision), 准确率(accuracy), F1分数(F1-score) - CSDN博客 只能说，这个写的解释的一塌糊涂。

• 机器学习 F1-Score, recall, precision - 万水千山 - CSDN博客 解释的更好些，搞清楚。

• classification - How to interpret F-measure values? - Cross Validated

The importance of the F1 score is different based on the scenario. Lets assume the target variable is a binary label.

Balanced class: In this situation, the F1 score can effectively be ignored, the mis-classification rate \(1-Acc\) is key. Unbalanced class, but both classes are important: If the class distribution is highly skewed (such as 80:20 or 90:10), then a classifier can get a low mis-classification rate simply by choosing the majority class. In such a situation, I would choose the classifier that gets high F1 scores on both classes, as well as low mis-classification rate. A classifier that gets low F1-scores should be overlooked. Unbalanced class, but one class if more important that the other. For e.g. in Fraud detection, it is more important to correctly label an instance as fraudulent, as opposed to labeling the non-fraudulent one. In this case, I would pick the classifier that has a good F1 score only on the important class. Recall that the F1-score is available per class.

说到点子上了。

• ROC101.dvi ROC和F1的关系。

---