分类问题中基于信息论度量的客观评价研究
doi: 10.3724/SP.J.1004.2012.01169
Information-theoretic Measures for Objective Evaluation of Classifications
-
摘要: 基于信息论度量而系统性地研究了拒识分类中客观评价问题. 定义了客观评价度量为一类无自由参数描述的函数. 该定义是为了从技术上可以简单地判别度量或评价在“客观性”或“主观性”中的归属. 建议了应用并考察24个信息度量. 它们分别来自于互信息、散度、交叉熵的定义. 不同于传统的性能类度量是基于经验公式或用户直觉上的定义, 信息类度量是构建在更为普适的理论基础上. 该类度量可以对二值分类中的“误差类别”与“拒识类别”进行区分, 而不需求人们输入代价信息. 针对拒识分类评价中更为关注的需求, 提出了三个“元度量(meta-measure)”用于考察度量. 由此用户可以在更高知识层面上测评度量的各自优缺点. 应用数值实例比较了24个信息度量. 对其中最优的信息度量进行了在“误差”与“拒识”代价性质方面的解析分析.Abstract: This work presents a systematic study of objective evaluations of abstaining classifications using information-theoretic measures (ITMs). First, we define objective measures as the ones which do not depend on any free parameter. According to this definition, technical simplicity for examining "objectivity" or "subjectivity" is directly provided for classification evaluations. Second, we propose 24 normalized ITMs for investigation, which are derived from either mutual information, divergence, or cross-entropy. Contrary to conventional performance measures that apply empirical formulas based on users0 intuitions or preferences, the ITMs are theoretically more general for realizing objective evaluations of classifications. They are able to distinguish "error types" and "reject types" in binary classifications without the need to inputting data of cost terms. Third, to better understand and select the ITMs, we suggest three desirable features for classification assessment measures, which appear more crucial and appealing from the viewpoint of classification applications. Using these features as "meta-measures", we can reveal the advantages and limitations of ITMs from a higher level of evaluation knowledge. Numerical examples are given to demonstrate our claims and compare the differences among the proposed measures. The best measure is
-
Key words:
- Abstaining classifications /
- error types /
- reject types /
- entropy /
- similarity /
- objectivity
点击查看大图
计量
- 文章访问数: 1544
- HTML全文浏览量: 41
- PDF下载量: 734
- 被引次数: 0