-
摘要: 深度学习是一门依赖于数据的科学, 传统深度学习方法假定在平衡数据集上训练模型, 然而, 现实世界中大规模数据集通常表现出长尾分布现象, 样本数量众多的少量头部类主导模型训练, 而大量尾部类样本数量过少, 难以得到充分学习. 近年来, 长尾学习掀起学术界的研究热潮, 涌现出大量先进的工作. 本文综合梳理和分析了近年来发表在高水平会议或期刊上的文献, 对长尾学习进行全面的综述. 具体而言, 根据深度学习模型设计流程, 将图像识别领域的长尾学习算法分为丰富样本数量与语义信息的优化样本空间方法, 关注特征提取器、分类器、logits和损失函数这四个基本组成部分的优化模型方法以及通过引入帮助模型训练的辅助任务, 在多个空间共同优化长尾学习模型的辅助任务学习3大类, 并根据提出的分类方法综合对比分析每类长尾学习方法的优缺点. 然后, 进一步将基于样本数量的狭义长尾学习概念推广至多尺度广义长尾学习. 此外, 本文对文本数据、语音数据等其它数据形式下的长尾学习算法进行简要评述. 最后, 讨论了目前长尾学习面临的可解释性较差、数据质量较低等挑战, 并展望了如多模态长尾学习、半监督长尾学习等未来具有潜力的发展方向.Abstract: Deep learning is a science that depends on data. Traditional deep learning methods unrealistically assume that the training models are on balanced datasets. In real-world large-scale datasets, a long-tailed distribution often occurs, with a few head classes having many samples dominating model training, while many tail classes have too few samples to be adequately learned. In recent years, the long-tailed learning has set off a research upsurge in academic circles. In this paper, we synthesize and analyze the literature published in high-level conferences or journals to provide a comprehensive survey of long-tailed learning. Specifically, we categorize long-tailed learning algorithms in the field of image recognition according to the design process of deep learning models into three main types: optimizing the sample space by enriching the quantity and semantic information of samples, optimizing the model by focusing on the four fundamental components of feature extractor, classifier, logits, and loss function, and auxiliary task learning, which involves introducing auxiliary tasks to aid model training and jointly optimizing long-tailed learning models across multiple spaces. Additionally, a comprehensive comparative analysis of the strengths and weaknesses of each category is conducted based on the proposed classification method. We further extend the concept of narrow long-tail learning based on the number of samples to multi-scale generalized long-tailed learning. In addition, we briefly review long-tailed learning algorithms in other data forms such as text data. Finally, we discussed the current challenges faced by long-tailed learning, such as poor interpretability and low data quality, and explored promising future directions such as multimodal long-tailed learning and semi-supervised long-tailed learning.
-
身份鉴别是个人利益和国家安全的重要保证, 生物特征作为身份鉴别的一种重要工具, 因其不可代替性和便携性而受到学者与产业界的青睐[1-2]. 例如, 生物特征识别系统被广泛应用于国防安全、互联网金融、海关出入境等多个领域[3-4]. 随着生物特征识别系统应用的普及, 生物特征存在一旦丢失无法重新发布的隐患逐渐显现出来. 为此生物特征模板保护成为身份认证领域的研究热点.
双因子可撤销生物特征模板保护方法是生物特征模板保护的主流方法之一, 需要附加用户特定参数(通常以密码或令牌的形式出现)与生物特征一起作为输入[1, 5]. 该方法需要用户引入额外的输入因子, 存在一些问题, 例如: 保留令牌或记忆密码给用户带来了不便, 以及外部因子可能被盗、丢失或遗忘等[6]. 基于单因子的可撤销生物特征模板保护是一种新的生物特征模板保护方法[7], 将生物特征作为唯一的输入因子, 解决了上述双因子可撤销生物特征模板保护中外部因子产生的问题.
本文基于文献[7]中单因子的可撤销生物特征认证系统的框架, 提出了一种新的解决方法, 即滑动提取窗口哈希(Window sliding and extracting Hashing, WSE) 算法. 与文献[7]中方法相比, 该方法改进了滑动窗口取值与哈希函数模块, 并以指纹模板的二进制矢量形式[8]为例, 在FVC2002和FVC2004的两个公共指纹数据集中的4个数据库上进行实验. 实验结果表明, 本文提出的方法不仅满足可撤销生物识别技术设计的4个标准, 而且能抵御3种安全攻击.
本文方法的主要贡献如下:
1) 建立了WSE哈希算法的单因子可撤销生物特征认证模型, 提高了可撤销模板的精确性;
2) 采用了跳位取值的滑动窗口哈希算法技术, 提高了可撤销模板的安全性;
3) 增加了一个评价维度, 即唯密文攻击, 更加全面的评价本文方法的安全性.
本文的其余部分安排如下: 第1节介绍相关工作, 第2节描述了一种单因子的可撤销生物特征认证方法, 第3节给出了性能分析, 第4节对安全性进行了分析和讨论, 最后, 第5节给出了总结与展望.
1. 相关工作
生物特征模板保护通常可分为两大类[1]: 可撤销的生物特征识别和生物特征加密系统. 前者包括生物特征加盐法和不可逆形变法, 后者包括密钥绑定系统和密钥生成方案. 本文的研究重点是可撤销的生物特征识别.
一般来说, 可撤销的方案通常被设计为参数化认证机制, 要求用户提供生物特征标识符和密钥. 用户特定的密钥通常存储在令牌的外部存储器(例如, 个人的存储器或物理硬件)中; 因此, 可撤销的方案也通常被称为"双因子"或"令牌化"生物特征认证方案. 另一方面, "单因子"或"无标记"方案要求用户仅提供用于认证的生物特征标识符, 并且单因子方案的工作量非常少. 单因子方案中的服务器负责存储注册模板和密钥. 在单因子方法中, 即使模板和密钥被破坏, 攻击者也很难获得原始模板. 不考虑转换策略(即盐基和不可逆转换)和生物统特征的形式, 本节介绍了双因子和单因子可撤销的生物特征识别方案的相关工作.
Biohashing[5]利用用户特定的令牌生成可撤销的模板(参见bioCode ${\bm b}$). 如图 1, 通常, Biohashing方法把生物特征向量${\bm x} \in {\bf R} ^{{n}} $和正交随机矩阵${R} \in {\bf R} ^{{n} \times {q} }$作为输入, 其中${q}\le {n}$. Biohashing方法中的可撤销生成过程如下: 1)通过计算${\bm x} ={{R}^{\rm T}{\bm x}}$形成内积矢量${\bm y}$; 2)根据预先定义的阈值$\tau$将${\bm y}$进行行二值化运算, 生成bioCode $ {\bm b}\in [0, 1]^{{q}}$, 如式(1)所示:
$$ \begin{align} {{b}}_{{i}}= \begin{cases} 0, &\mbox{若}~ {y_i }\ge \tau \\ 1, & \mbox{否则} \end {cases} \end{align} $$ (1) 其中, $i=1, \cdots , q$. Biohashing方法可以推广到其他生物特征形式, 例如, 面部、虹膜、手掌等. BioHashing是一种典型的生物特征加盐法, 其他的加盐法如文献[9-10], 这些基于加盐的方法具有共同的特征, 即它们利用外部用户特定因子来生成转换矩阵并与生物特征模板相乘或卷积, 当模板受到攻击时, 可以通过改变令牌从而撤销已有模板并生成新模板. 此外, 文献[11]阐述了利用折衷的可撤销模板和正交矩阵获得原始生物特征模板的可行性.
Wang和Hu[12]提出了一种可撤销的指纹生物识别技术, 即"Densely infinite-to-one mapping (DITOM)"映射. 该方案在匹配过程中不需要对准过程, 利用多对一转换机制, 来生成用于匹配的可撤销模板. 简而言之, 该方案将每个细节点对量化为二进制字符串, 之后进行离散傅里叶变换(Discrete Fourier transformation, DFT)将二进制串转换为复数向量${{C}}$. 再通过将随机生成的参数密钥${R}$与复向量组合来生成可撤销模板${{T}}$. 组合函数描述如式(2)所示.
$$ \begin{align} {T}={R}{C} \end{align} $$ (2) 与将生物特征数据和密钥组合以生成模板的Biohashing不同, 该方法从生物特征数据生成不可逆实例, 然后将不可逆实例与密钥组合以生成可撤销模板.
布隆过滤方法(Bloom filter)是由Rathgeb等[13]提出的可撤销生物识特征别技术, 首先应用于虹膜模板保护, 后来被推广到面部、指纹和多模态生物特征识别等多种生物特征形式. 在文献[13]中, Bloom filter ${\bm b}$是一个长度为${n}$的用0初始化的bit数组. 然后, 应用${K}$个独立的哈希函数根据输入项生成一组十进制值${\bm h}\in [0, {n}-1]^{{K}}$. 之后, 通过增加${\bm b}$中元素的值来形成最终${\bm b}$, 其中${\bm h}$中的值表示要增加的元素的位置. 该技术中不是使用哈希函数, 而是提出二进制到十进制映射函数来生成用于匹配的${\bm b}\in [0, 1]$[13]. 基于Bloom filter转换的过程如图 2所示[13]: 1)给出一个具有${H}\times {W}$维度的IrisCode, 将IrisCode细分为维度为${H}\times {l}$的多个子矩阵${{B}}$, 其中${l}={W}/{K}$, ${K}$是子矩阵的数量; 2)对于每一个${ {B}}_{{i}}$, 其中${i}=1, 2, \cdots , {K}$, 当${w}={H} $时, 用0来初始化Bloom filter ${\bm b}_{{i}}\in [0, 1]^{2^{w}}$; 3)在每个${{B}}_{{i}}$中, 逐列对元素执行二进制到十进制转换以生成一组十进制数; 4)根据在3)中转换得到的十进制数, 将Bloom filter ${\bm b}_{{i}}$中的元素设置为1, 其中十进制数的值表示${\bm b}_{{i}}$中元素的索引; 5)重复步骤2) $\sim$ 4), 直到形成${K}$ Bloom filter ${\bm b}_{{i}}$. 注意, 在步骤4) 中, 如果转换了两个相同的十进制数, 则${\bm b}_{{i}}$中的元素仍设置为1, 因此, 实现了多对一映射. 为了实现可撤销性, 在基于Bloom filter转换之前, 将原始的IrisCode和特定的秘密${T}$做异或(XOR)运算. 尽管Bloom filter方法具有良好的不可逆性(多对一映射), 但它不能满足不可链接性标准[14]. Hermans等[14]说明了由相同的IrisCode和不同的${T}$生成的两个Bloom filters之间的高匹配分数(最高96 %). 此外, Bringer等[15]指出当密钥空间(${T}$)太小时, Bloom filter方法容易受到暴力攻击.
Cappelli等[16]提出了一种指纹细节点描述符(Minutia cylinder code, MCC). MCC是将细节点集${M}=\{{\bm{m}}_{{1}}, { \bm{m}}_{{2}}, \cdots , { \bm{m}}_{{n}}\}$转换成一组圆柱数据${C}= \{{\bm{c}}_{{1}}, { \bm{c}}_{{2}}, \cdots , { \bm{c}}_{{n}}\}$的技术, 其中每个${\bm m }=\{x$, $y, \theta\}$, ${n}$是提取的细节点的数量, 圆柱是指在固定半径${r}$内记录中心细节点与其邻域细节点之间的方向(导向)和空间(位置)关系的数据结构. 尽管MCC具有较高的匹配性能, 但可以从MCC模板中获得原始细节点集[17], 因此, 文献[17] 提出了P-MCC (Protected minutia cylinder code)来保护MCC模板. 在P-MCC方法中, 通过单向转换函数B-KL投影(B-KL projection) 将MCC模板${C}=\{{\bm{c}}_{{1}}, { \bm{c}}_{{2}}, \cdots , { \bm{c}}_{{n}}\}$转换为P-MCC模板${V}=$ $\{{\bm{v}}_{{1}}$, ${ \bm{v}}_{{2}}, \cdots , { \bm{v}}_{{n}}\}$[17]. B-KL投影概述如下: 1)在训练过程中, 从MCC模板计算出一个平均向量$\bar{\bm x}$和${k}$个最大特征值${\Phi}$; 2)使用参数$\bar{\bm x}$和${\Phi}_{k}$, 在MCC模板上执行KL投影以生成2P-MCC模板; 3)使用单位阶跃函数对2P-MCC进行二值化. 尽管P-MCC生成了不可逆的实例, 保证了MCC模板的安全性, 但P-MCC的用户无法使用相同的指纹来重新发布P-MCC模板. 针对可撤销性问题, 提出了2P-MCC (Two-factor protected minutia cylinder code). 在2P-MCC中, 使用用户特定的密钥${\pmb s}$对在P-MCC模板进行部分置换, 生成2P-MCC模板[18]. 然而, 2P-MCC方案不能推广到其他生物特征模式, 因为它是专门为点集数据结构(即MCC模板)设计的.
Ouda等[19]提出了无标记的可撤销生物特征识别方法, 即"BioEncoding", 来保护IrisCode. BioEncoding方法中的两个基本输入是: ${n}$位二进制向量$\bm c$ (生物特征数据)和随机生成密钥${S}$ $\in$ $[0, 1]^{2^{{m}-1}}$, 其中${m}$是系统参数. 从两个输入导出BioCode $\bm b \in[0, 1]^{{n}/{{m}}} $的过程是: 1) 将$\bm c$分割成具有${m}$位的多个块; 2) 将这些块转换成一组整数值${\bm x}$ $=$ $\{x_{{1}}$, $x_{{2}}, \cdots , x_{{n/m}}\}$; 3)通过执行布尔函数$f(x)$将整数值转换为二进制值, 该函数定义式为
$$ \begin{align} f(x)={S}[x_i], \quad f(x)\in {0, 1} \end{align} $$ (3) 其中, ${S}[x_i]$表示${S}$中的第${x}_i$个二进制, $i=1, \cdots$, ${n/m}$. 因此, 输出二进制值形式的BioCode $\bm b$用于匹配. 虽然文献[19]表明${S}$可以作为公共信息存储在数据库中, 但如果${S}$泄露, 原始生物特征模板可以恢复.
表 1总结了各种可撤销的生物特征认证方案在转换方式、相似性和缺点方面的比较结果.
表 1 各种生物特征模板保护算法的比较结果Table 1 Comparative result of various biometric template protection methods可撤销方案 转换方式 相似性 缺点 Biohashing[5] 随机投影+二值化处理 汉明距离 原始模板可由折衷密钥推算出来 Wang等[12] 离散傅里叶变换+随机投影 汉明距离 性能下降 Bloom filter[13] Bloom filter (十进制到二进制映射) 汉明距离 易受暴力攻击 P-MCC[17] KL投影+二值化 汉明距离 可撤销性弱 2P-MCC[18] 完全/部分置换 汉明距离 用户需要管理密钥 GRP-based IoM Hashing[10] 多重随机投影+记录最大值索引 欧氏距离 性能下降 URP-based IoM Hashing[10] 置换+记录最大值索引 欧氏距离 性能下降 BioEncoding[19] 布尔函数 汉明距离 易受ARM攻击 在认证阶段, 双因子可撤销生物认证方法依赖于其他认证因素, 在转换过程中需要用户特定的令牌, 转换过程复杂, 且需要大量的存储空间存放额外的令牌化随机数据.单因子可撤销生物认证方法不依赖于其他独立的认证因素, 在不影响性能精度的前提下, 满足了不可逆性、可撤销性和不可链接性的要求, 且转换过程简单, 所需存储空间降低. 两者的应用场景都是身份认证, 但是单因子方法只需个人生物特征, 而双因子方法还需要令牌.
另外, 尽管双因子可撤销生物认证方法是生物特征模板保护的主要方法, 但这种方法还是存在不足, 例如, 该方法需要用户的额外输入, 而且外部因素可能遗忘, 被盗或丢失, 这导致了文献[6]中被盗令牌场景的不利情况. 被盗令牌场景是指真实用户的令牌(参数)受到攻击并被攻击者利用以发起零努力错误接受攻击的事件. 此外, 用户公开特定参数可能会产生转换模板入侵的风险, 特别是对于生物特征加盐法的方案. 单因子可撤销生物认证方法可以有效避免这些不足.
2. 一种单因子的可撤销生物认证方法
2.1 方法框架
局部敏感哈希(Locality sensitive Hashing, LSH)主要通过将原始数据投影到更少数量的"桶" (buckets)来降低高维数据的维度. LSH的目标是以最大的概率将类似的物体映射到相同的"桶"中[10].
定义1. LSH是一族哈希函数$\mathcal{H}=\{h_i:{\bf R}^d \to B \} $, 将数据点从${\bf R}^d $映射到"桶" ${b} \in {B}$, 并且任何两个给定点${X}, {Y}\in {\bf R}^d$满足的条件:
$$ \begin{align} & \mathbb {P}_{h\in\mathcal{H}}(h_i({X})=h_i({Y}))\le \gamma, &&{s}({X}, {Y})<\alpha\notag\\ &\mathbb {P}_{h\in\mathcal{H}}(h_i({X})=h_i({Y}))\ge \delta, &&{s}({X}, {Y})>\beta \end{align} $$ (4) 其中, $\delta > \gamma$, ${s}(\cdot)$是相似函数. LSH确保具有高相似性的数据点${X}$和${Y}$在在经过哈希函数后有较高的哈希碰撞概率, 即将${X}$和${Y}$映射到同一个"桶"中; 相反, 彼此相似度低的数据点发生哈希碰撞概率较低, 即两个数据点映射到不同的"桶"中.
双因子的可撤销生物特征认证方法将令牌化随机数作为外部因子带来一些问题, 本文针对这些问题提出一种单因子的可撤销生物特征认证方法, 即滑动提取窗口哈希(Window sliding and extracting Hashing)算法, 简称WSE哈希算法. 该方法实际上应用了LSH的理论, 在本文指纹匹配的场景中, 通过复制原始特征向量, 尽可能增加有用特征的提取数量, 经过哈希(滑动窗口跳位取值)后, 相似物体的哈希值碰撞的几率一定也高, 所以匹配成功. 与文献[7]中方法相比, 该方法改进了滑动窗口取值与哈希函数模块, 目的是提取更多有用的特征向量, 增强不可逆性, 以提高可撤销模板的性能和安全性. 本文提出的单因子的可撤销生物特征认证方法框架如图 3所示, 该方法只需要生物特征(以指纹为例)作为唯一的输入因子, 与二进制随机数生成器生成的密钥$\bm r$做运算生成可撤销的模板. 具体来说, 在注册阶段, 首先由二值生物特征向量$\bm x$生成置换种子(Permutation seed), 然后置换密钥$\bm r$, 得到可撤销模板$\bm w$. 该阶段存储编码随机二进制向量$\bm v$ (密文)和模板$\bm w$. 在验证阶段, 从密文中解码和置换密钥生成用于匹配的查询向量$\bm w^\prime $, 其中置换种子由查询生物特征确定. 最后$\bm w$和$\bm w', $进行匹配, 判断是否匹配成功.
2.2 滑动提取窗口哈希算法
设${\bm x }\in [0, 1]^{l}$是一个具有长度${l}$的二元生物特征向量, 则WSE哈希算法的实现描述如下:
1) 将${\bm x}\in [0, 1]^{l}$复制${m}$倍, 形成扩展的特征向量$\bar{\bm x} \in [0, 1]^{lm}$, 其中${m}$是系统参数; 该步骤增加了二元生物特征向量的长度, 为接下来的精度要求和安全性分析做准备.
2) 对于每个元素$\bar{x_{i}}\in \bar{\bm x}$, 它附加来自于$\bar{\bm x}\in$ $[0, 1]^{lm}$对应的${k}-1$个元素, 其中${k}$是系统参数, 我们将其命名为窗口大小, 生成一个子位块$\bar{x}_{{{b}}_i}=$ $[\bar{x}_i|\bar{x}_{i+2}|\bar{x}_{i+4}| \cdots |\bar{x}_{i+2({k}-1)}]$, 这种方法称之为滑动窗口跳位取值, 其中$|$表示连接操作, 按照此方法可以由$\bar{\bm x}$转换成子位块的形式$\bar{\bm x}_b$. 例如, 令$\bar{\bm x}= [\bar{x}_1$, $\bar{x}_2, \cdots , \bar{x}_{lm}]$, ${k}=2$, 则, 每个$\bar{x_{i}}$附加来自$\bar{\bm x}$的$ (2-1)$个元素; 如果$\bar{x_{i}}$是$\bar{\bm x}$的倒数第二个元素$ (\bar {x}_{{lm}-1})$, $\bar{\bm x}$的第一个元素将会被追加, 若$\bar{x_{i}}$是$\bar{\bm x}$的最后一个元素$ (\bar{x}_{{lm}})$, $\bar{\bm x}$的第二个元素将会被追加, 即, $\bar{\bm x}_{{b}}=[\bar{x}_1|\bar{x}_3, \bar{x}_2|\bar{x}_4, \cdots , \bar{x}_{lm-1}|\bar{x}_1, \bar{x}_{lm}|\bar{x}_2]$, 该操作保护数据$\bm x$.
3) 将$\bar{\bm x}$的每个子位块$\bar{x}_{{{b}}_i}$转化为整数$\hat {x}_i \in {\bf Z}$. 此时若$\hat {x}_i$为0, 则令$\hat {x}_i=1$. 此操作进一步处理$\bar{\bm x}$, 将每个子块由二进制转为十进制.
4) 根据哈希函数$y_i=(i^{\hat {x}_i})~{\rm mod}~(lm+1)$, 变换$\hat {x}_i$以确保$y_i$的最大值等于${lm}$. 如果求模运算结果为0, 则设置$y_i=1$. 此过程产生整数向量${\bm y}$ $=$ $[0, 1]^{lm}$. 这一步由$\bm x$生成置换种子$\bm y$.
5) 设${\bm r }\in [0, 1]^{lm}$是用户/应用程序特有的随机字符串, 作为密钥key, 它是由伪随机二进制数发生器生成的向量, 将置换种子$\bm y$作为$r \in {\bm r}$的索引置换$\bm y$生成可撤销的模板$\bm w$, ${\bm w}=[{r}_{{{y}}_1}, \cdots , {r}_{{{y}}_{lm}}]\in [0, 1]^{{lm}}$. 算法1展示了滑动提取窗口哈希算法.
算法1. 滑动提取窗口(WSE) 哈希算法
输入.二进制生物特征向量${\bm x}\in [0, 1]^{l}$, 复制的倍数${m}$, 窗口大小${k}$
步骤1.扩增二进制生物特征向量
$~{\bf for}~ {i}=1: {m}$
将$\bm x$扩大${m}$倍, 赋值给$\bar{\bm x}$
即, 令$\bm x=\bar{\bm x}$
${\bf end ~for}$
步骤2. 生成子位块
${\bf for }~{i}=1: {lm}$
令$\bar{x}_{{{b}}_i}=[\bar{x}_i|\bar{x}_{i+2}|\bar{x}_{i+4}| \cdots |\bar{x}_{i+2({k}-1)}]$,
其中$|$表示连接操作
$~{\bf end~ for}$
步骤3. 二进制转换为十进制
$~{\bf for }~{i}=1: {lm}$
将$[\bar{x}_i|\bar{x}_{i+2}|\bar{x}_{i+4}| \cdots |\bar{x}_{i+2({k}-1)}]$转换成整数值$\hat {x}_i$
当$\hat {x}_i==0$时, 令$\hat {x}_i=1$
$~{\bf end ~for}~$
步骤4. 生成置换种子$\bm y$
$~{\bf for}~ {i}=1: {lm}$
将$\hat {x}_i$进行$y_i=(i^{\hat {x}_i})~{\rm mod}~(lm+1))$转换
当$y_i==0$时, 令$y_i=1$
$~{\bf end ~for}~$
步骤5. $\bm r$置换构造$\bm w$
随机二进制向量${\bm r}\in [0, 1]^{l}$
$~{\bf for}~ {i}=1: {lm}$
${\bm w}={P}_{y}(\bm r)$, 其中$\bm y$被视为$\bm r$的索引
当$y_i==0$时, 令$y_i=1$
$~{\bf end ~for}~$
输出. 可撤销模板${\bm w}\in [0, 1]^{l}$
如图 4所示, 以长度${l}=6$的二元生物特征向量, 复制倍数${m}=2$, 窗口大小${k}=2$为例, 演示步骤1~4, 通过WSE哈希算法生成置换种子${\bm y }$ $=$ $[1$, $12]^{12}$的过程.
图 4 WSE哈希算法生成置换种子示意图(${l}=6$, ${m}=2$, ${k}=2)$Fig. 4 Diagram of generated permutation seed by WSE Hashing algorithm (${l}=6$, ${m}=2$, ${k}=2)$本文提出的一种单因子可撤销生物特征方案中的WSE哈希算法流程图如图 5所示. 在注册阶段, 输入用户的生物特征向量${\bm x }$, ${\bm x }\in [0, 1]^{l}$, 与密钥${\bm r }$经过WSE哈希算法, 生成扩展向量$\bar{\bm x}$和隐藏了真实信息的整数向量${\bm y} = [1, lm]^{lm}$, ${\bm y }$为置换种子. 然后, 将$\bar{\bm x}$与${\bm r }$进行异或生成二进制编码的随机向量${\bm v }$, ${\bm v }={\bm r }$ $\oplus$ $\bar{\bm x}$; 将${\bm y} $作为${\bm r }$的索引, 并置换${\bm r }$生成可撤销的生物特征模板$\bm w$, ${\bm w}=[{r}_{{{y}}_1}, \cdots , {r}_{{{y}}_{lm}}]\in [0, 1]^{{lm}}$, 可以简写为${\bm w}={P}_{y}(\bm r)$, 其中${P}(\, )$是置换函数. 最后只将${\bm v }$和$\bm w$存储在数据库中, 这样做有助于保护用户的真实信息, 增强不可逆性, 提高安全性. 一方面, 因为${\bm r }$和$\bar{\bm x}$都未保存, 攻击者不能从${\bm v }$中轻易的得到${\bm r }$, 必须同时猜测${\bm x }$ (或$\bar{\bm x}$)和${\bm r }$ (从${\bm v }$中推出), 另一方面, ${\bm w }$是由真正的用户${\bm x }$ (或$\bar{\bm x}$)解码生成的, 要得到正确的${\bm w }$必须是正确的生物特征输入.
图 5 WSE哈希算法流程图($l=6, m=2, k=2$)Fig. 5 The flowchart of WSE Hashing algorithm ($l=6, m=2, k=2)$在验证阶段, 给出查询二进制生物特征向量${\bm x}' $, 让其也经过WSE哈希算法, 得到扩展向量$\bar{\bm x}^\prime $和$\bm y^\prime $. 给定${\bm v }$, 通过逆运算${\bm r^\prime }={\bm v }\oplus {\bar{\bm x}^\prime }$得到${\bm r^\prime }$, 然后置换${\bm r^\prime }$, ${\bm w^\prime }={P}_{{y}^\prime }(\bm r^\prime )$获得查询可撤销模板${\bm w^\prime }$, 换而言之, ${\bm r^\prime }$是由数据库中的${\bm v }$解码得到的.
该方案是单因子的, 在验证阶段身份检验的唯一输入是生物特征, 而不是像基于双因子的置换方案那样由第二个因子计算得出[20]. 在双因子方案中, 如果在注册期间和验证期间的置换种子是相同的, 则${\bm r }$置换前后的性能将被精确保留.然而, 在本文提出的方法中, 因为两种置换种子都来自于唯一的注册生物特征和查询生物特征, 注册生物特征和查询生物特征在实际中是不相同的. 这一点可以类比对称加密系统(见第4.3节), ${\bm r }$可以根据需要进行撤销和替换.
3. 性能分析
3.1 实验环境
本文实验均在MATLAB R2017b上运行, 运行环境为Intel$\circledR$ Core(TM) i5-7500 CPU @ 3.40 GHz, Intel$\circledR$ HD Graphics 630 (1 024 MB), 内存16.00 GB的台式电脑.
本文用长度为256位的二进制指纹向量$\bm x$作为输入[8], 在4个公共指纹数据集(FVC2002 (DB1, DB2)[21] FVC2004 (DB1, DB2)[22])上进行实验. 每个数据集包含100个手指的采样图像, 相当于100个用户, 每个手指采样8次, 得到有8个样本, 因此总计有800个指纹图像样本. 因为文献[8]是基于学习的方法, 所以每个用户的8个样本中有3个用于训练, 有5个样本可以用于测试. 通过比较汉明距离获得匹配结果, 因为注册和查询标识符均是二进制向量.
本文中, 评价指纹识别系统性能准确性的参数是真/假匹配得分(Genuine/Imposter matching score)和等错误率(Equal error rate, EER). 评价标准是文献[23]中的测试协议. 在每个数据集中, 可以生成真匹配得分1 000个($100\times C_5^2$), 假匹配得分4 950个($C_{100}^2$). 为了无偏差地评估所提出的方案, 本文基于五个不同密钥key ($\bm r$)的实验来计算平均EER. 该方案是单因子可撤销方案, 因此不需要对盗令牌的场景进行评估.
文中处理时间是指注册阶段和验证阶段的总计, 其中前者包括密钥key($\bm r$)生成, 可撤销模板生成和密钥编码; 而后者包括密钥解码, 查询可撤销模板生成和匹配. 表 2说明了当${m}=1 000$和${k}=3$的WSE哈希算法处理时间. 从表 2中可以看出, WSE哈希算法两个阶段的平均处理时间约等于0.035 s.
表 2 WSE哈希处理效率(s) (${m}=1 000$, ${k}=3$)Table 2 Processing efficiency of WSE Hashing (s) (${m}=1 000$, $k=3$)平均时间 FVC2002-DB1 FVC2002-DB2 FVC2004-DB1 FVC2004-DB2 注册阶段 0.035031 0.034884 0.034481 0.033151 验证阶段 0.034896 0.034874 0.034621 0.034647 3.2 认证性能
本节分析内部各个参数的不同取值对认证性能(EER)的影响, 以及比较对比实验和本文方法的认证性能.
3.2.1 各个参数对认证性能的影响
方案中有两个系统参数, 分别是扩大的倍数${m}$ $({m}$ $\ge$ $1)$和窗口大小${k}$ $({k}\ge 2)$. 本节通过实验来分析${m}$和${k}$对所提方法的认证性能的影响, 用等错误率EER(%)表示, EER越低, 说明性能越好.
图 6显示了WSE哈希算法在数据库FVC2002 DB1上的"EER-vs-${k}$"的曲线, 其中窗口大小${k}$从2, 3, 4到5的变化, 而${m}$从1, 5, 10到15的变化. 我们观察单个线条, ${m}$值固定不变且${k}$变大时, EER (%)会变高. 如算法1中所述, ${k}$表示子位块, 因此当${k}$值增大时, 需要附加更多的比特, 增加了子比特块之间的噪声影响, 所以EER (%)变高. 图 6的另一个观察结果是当${m}$变大时EER (%)变低.
根据图 6的观察, 为了进一步研究${m}$对认证性能的影响, 进行了如下实验研究. 实验时, 使用控制单一变量法, 将${k}$固定为2, 通过改变${m}$的取值来观察EER (%)的变化, ${m}$的取值分别为1, 5, 10, 15, 20, 40, 100, 200, 500, 800, 1 000. 图 7显示了FVC2002 DB1上的"EER-vs-${m}$"曲线. 增大${m}$则EER相对较低, 认证性能提高; ${m}$在1, 5, 10和20之间变化较明显, 而认证性能在${m}$ $({m}\ge 40)$时以较慢的速度改变. 值得注意的是, ${m}$较大时可以减少由注册和查询生物特征生成的两个置换序列$\bm r$与$\bm r ^\prime $的冲突, 但是${m}$也不能一味的增大, 因为${m}$过大时, 会造成资源浪费及攻击者易盗取的安全隐患.
图 7 EER-vs-${m}$曲线图(FVC2004 DB1/DB2)Fig. 7 Curves of "EER (%)-vs-${m}$" (FVC2004 DB1/DB2)3.2.2 对比实验的认证性能
本文WSE哈希算法在${m}=1 000$和${k}=3$时的性能精度与原始生物特征识别方法、4种经典的双因子指纹可撤销生物识别技术以及文献[7]的单因子方法EFV Hashing比较, 如表 3所示. 据观察, WSE哈希算法在数据库FVC2002 DB1和FVC2004 DB1上等错误率EER均最低, 在其他数据库上也展现了良好的性能. 除此之外, 在与双因子方案比较中, WSE哈希算法优于文献[18]、文献[24]和基于URP的IoM[10]. 在与单因子方案EFV哈希算法[7]的比较中, 由于WSE哈希算法改进了滑动窗口取值与哈希函数模块, 在4个数据库上性能精度均有所提升, 且不可链接性也有提升(参见第3.5节).
表 3 不同方法的性能精度对比(EER) (%)Table 3 EER comparison between proposed method and other methods (%)方法 FVC2002-DB1 FVC2002-DB2 FVC2004-DB1 FVC2004-DB2 WSE Hashing 0.2 0.62 2.6 7.13 Binary fingerprint vector (Baseline)[11] 0.26 0.12 1.58 4.39 URP-based IoM Hashing[10] 0.46 2.1 4.51 8.02 GRP-based IoM Hashing[10] 0.22 0.47 4.74 4.1 Bloom filter[24] 2.3 1.8 13.4 8.1 2P-MCC$_{64, 64}$[18] 3.3 1.8 6.3 _ EFV Hashing[7] 0.32 0.63 2.62 7.14 3.3 不可逆性
本文方法在数据库中存储的只有$\bm v$和$\bm w$, 若能逆推出$\bm x$或$\bar{\bm x}$则说明不满足不可逆性. 假设攻击者已经盗取$\bm v$, 根据${\bm v }={\bm r }\oplus \bar{\bm x}$, 我们如果知道${\bm r }$或$\bar{\bm x}$都可以经过逆运算得到另外一个, 但是由于${\bm r }$和$\bar{\bm x}$在数据库中均未存储, 所以无法恢复$\bar{\bm x}$或${\bm r }$.
假设攻击者盗取了$\bm w$, 即使已知${\bm w}={P}_{y}(\bm r)$, 但由于$\bm x$未存储不可知, 所以置换种子$\bm y$不可知, 则从$\bm w$中恢复密钥$\bm r$的枚举次数是$2^{{lm}}$次, 并且${l}=$ $256$, ${m}=1 000$, 这在实际计算中也是不可行的, 因而无法恢复$\bm x$或$\bar{\bm x}$.
3.4 可撤销性
根据可撤销性的要求, 一旦模板被破坏, 就应该生成一个新的模板并替换受损模板. 为了验证方案的可撤销性, 计算和评价了来自每个数据集真匹配得分(Genuine match score)、假匹配得分(Imposter match score)和配对真匹配得分(Mated-genuine match score)分布. 计算Mated-genuine分数分布的步骤是: 1)对于每个用户, 使用51个不同的${\bm r }$和用户的第一个特征向量生成51个不同的模板; 2)将第一个模板(假设为已泄露的模板)与其余50个模板(假定为更新的模板)匹配, 从而为每个用户生成50个Mated-genuine分数. 因此, 共有5 000 ($50\times 100$个用户) Mated-genuine得分. 图 8显示了FVC2002的DB1和DB2、FVC2002的DB1和DB2这4个数据库的可撤销性分析, 其中Mated-genuine和Imposter得分分布在很大程度上重叠. 这表示对于相同的用户, 用不同的密钥${\bm r }$生成的模板彼此之间不能区分, 所以WSE哈希算法是满足可撤销性的.
3.5 不可链接性
根据不可链接性的要求, 同一个生物特征向量$\bm x$或$\bar{\bm x}$与不同的密钥${\bm r }s$生成的多个模板${\bm w }s$, 这些${\bm r }$之间不能链接. 本文遵循文献[25]的基准框架来验证WSE哈希算法的不可链接性. 方法如下:
1) 计算WSE哈希算法模板与配对/非配对样本得分分布(Mated/non-mated samples score distributions)的模型交叉匹配. 其中, 配对样本分数分布(Mated samples score distributions)是由同一用户通过不同密钥产生的模板之间的相似性匹配来计算. 非配对样本得分分布(Non-mated samples score distributions)是指由不同用户利用相同密钥导出的模板之间的相似性匹配.
2) 计算局部度量$D_\leftrightarrow(s)$和全局度量$D_{\underset\longleftrightarrow{{\rm sys}}}$的值[25], 并根据计算的值判断转换模板的不可链接性.
具体来说, 局部度量$D_\leftrightarrow(s)$和全局度量$D_{\underset\longleftrightarrow{{\rm sys}}}$是为了定量评估转化模板的不链接性, 而引入的两种不同的度量[25], 它们是根据配对和非配对样本得分分布计算的. 局部度量$D_\leftrightarrow(s)\in [0, 1]$是依赖于配对和非配对样本得分分布之间的似然比的局部得分测度. $D_\leftrightarrow(s)$的值从0到1表示转换后的模板在得分基础上的可链接性程度. 全局度量$D_{\underset\longleftrightarrow{{\rm sys}}}\in [0, 1]$评估整个系统的不可链接性, 并且可以更公平地与其他可撤销方案的不可链接性水平进行比较. $D_{\underset\longleftrightarrow{{\rm sys}}}$越接近0, 转换模板集的不可链接性越好.
实验在所有数据集上进行了测试, 其中最佳参数集为${m}=1 000$, ${k}=3$. 为了公平地评估转换模板的不可链接性, 将$\omega$设置为1, 并且$\omega $是计算$D_\leftrightarrow(s)$和$D_{\underset\longleftrightarrow{{\rm sys}}}$的参数. 根据文献[25], $\omega =1$是不可链接性评估标准的最坏情况.
图 9显示了4个数据库(FVC2002 (DB1, DB2), FVC2004 (DB1, DB2))的不可链接性的分析. 正如图 9所示, 配对和非配对样本的得分分布曲线是重叠的, 这表示源自同一用户或不同用户的模板无法区分. 因此, WSE哈希算法满足不可链接性标准.
表 4列出了WSE哈希算法和EFV哈希算法[7]所有测试数据集的$D_{\underset\longleftrightarrow{{\rm sys}}}$的详细值, 表中WSE哈希算法$D_{\underset\longleftrightarrow{{\rm sys}}}$的最大值= 0.03 (接近0), 这表明WSE哈希算法接近完全不可链接的情况. 并且我们可以观察到, EFV哈希算法$D_{\underset\longleftrightarrow{{\rm sys}}}$的最大值$=$ $0.05$ $>$ $0.03$, 这说明WSE哈希的不可链接性比EFV哈希算法的不可链接性高, 安全性和隐私性也高于EFV哈希算法.
表 4 不可链接性的全局度量($D_{\underset\longleftrightarrow{{\rm sys}}}$) (${m}=1 000$, ${k}=3$)Table 4 Global measure ($D_{\underset\longleftrightarrow{{\rm sys}}}$) of unlinkability (${m}=1 000$, ${k}=3$)方法 FVC2002-DB1 FVC2002-DB2 FVC2004-DB1 FVC2004-DB2 WSE Hashing 0.0257 0.0235 0.0271 0.0250 EFV Hashing[7] 0.0404 0.0473 0.0465 0.0459 4. 安全性分析
本文是单因子的可撤销生物特征模板保护方法, 所以基于双因子的可撤销生物特征模板保护中第2个因子的安全性问题, 在这里将不再分析. 在本节中, 我们从暴力攻击、字典攻击和唯密文攻击3个方面来分析本文方法的安全性.
4.1 暴力攻击
暴力攻击(Brute force attack)作为安全攻击的一个经典方法, 指的是用穷举法试图随机使用非法访问生成转换的查询实例. 在本文中, 暴力攻击是通过猜测来衡量的WSE哈希算法的复杂性在代码中耗尽了代码$\bm w^\prime $方式. 由于$\bm w^\prime $是具有长度${lm}$的二进制向量, 因此需要总共$2^{{lm}}$的猜测复杂度. 本文实验设置${m}=1 000$, ${l}=256$, 其猜测复杂性为$2^{256 000}$. 因此, 暴力攻击对本文方法是不可行的.
4.2 字典攻击
与暴力攻击中对整个散列代码的盲目猜测不同, 字典攻击(错误接受攻击) (False accept attack)需要更少的尝试来获得非法访问[26]. 实际上, 基于阈值的决策方案通常应用于生物识别系统, 因此这种攻击是可行的. 换句话说, 只要匹配分数超过预定阈值$\tau$, 就可以授予访问权限, 这可以显著减少攻击的次数.
选择FVC2002 DB1作为评估实例. 令参数${m}$ $=$ $1 000$, ${k}=3$和${l}=256$, 实验结果如图 10所示, 阈值$\tau=0.56$. 这说明字典攻击需要的密码序列的最小匹配是$lm\tau=143 360$. 因此, 字典攻击复杂度为$2^{lm\tau}=2^{143 360}$. 尽管比暴力攻击小得多, 但是在现实操作中也是不可行的.
图 10 真匹配-假匹配曲线(FVC2002 DB1, ${m}=1 000$, ${k}=3$)Fig. 10 Genuine-imposter curve on FVC2002 DB1 (${m}=1 000$, ${k}=3$)4.3 唯密文攻击
从另外一个角度看, 本文提出的WSE哈希算法可以看作是一种特殊的对称加密[4]. 对称加密(也称私钥加密)是指加密和解密使用相同密钥(或是两个密钥之间可以进行简单的转换)的加密算法. 在本文方法中, 生物特征信息${\bm x }$ (或$\bar{\bm x}$)对应于对称加密系统中的明文, 随机的二进制向量${\bm r }/{\bm r }^\prime $对应于加密/解密密钥, ${\bm v}$对应于密文. 因此, 我们还可以考虑针对对称加密算法的安全攻击, 如唯密文攻击(Cipher-text only attack, COA).
唯密文攻击(COA)是指攻击者仅仅知道密文, 来得到相应的明文信息. 在文中, 密文对应于存储在数据库中的${\bm v}$, 若已知${\bm v}$, 攻击者可以用$2^l$种可能的组合来枚举${\bm x}$ (或者$\bar{\bm x}$). 在第3.1节, 猜测正确${\bm x}$的平均时间是($ (\frac{2^l}{2})\times0.035$)\, s, 其中0.035 s是验证所花费的平均时间. 在本文中, ${l}=256$, 因此这需要平均$ (\frac{2^l}{2})\times0.035$\, s $\approx 6.43\times{10^{67}}$ year来猜测正确的${\bm x}$. 这表明猜测${\bm x}$是计算不可行的. 另一方面, 虽然${\bm w}$ $=$ ${P}_{y}(\bm r)$, 如果${\bm x}$ (来源于生物特征的置换种子)未知, 则从${\bm w}$中恢复${\bm r}$同样在计算上是不可行的, 因为${\bm r}$的暴力攻击猜测是$2^{lm}$个组合.
5. 总结与展望
双因子可撤销的生物特征认证方法引入额外因子即令牌化因子带来了隐私和安全威胁问题.本文提出了一种单因子可撤销生物识别解决方法, 即WSE哈希算法. 针对这一问题, 本文提出了一种唯一二值数据生物特征作为输入因子的单因子可撤销生物识别方法, 即WSE哈希算法. WSE哈希算法满足不可逆性, 可撤销性, 不可链接性以及精确性这4个可撤销的生物特征模板保护标准, 也抵御了3种方式的安全性攻击测试. 同时WSE哈希算法也可以扩展到二值向量形式表示的虹膜、面部特征、掌纹和静脉等生物特征识别. 另外, 算法的安全性, 如碰撞攻击、差分攻击等攻击方式, 也是我们未来研究方向.
-
图 1 深度长尾学习研究综述组织结构图
Fig. 1 Organizational Structure Diagram of a Survey on Deep Long-Tail Learning Research
图 8 优化样本空间各方法关系示意图
Fig. 8 Diagram of Relationships Among Various Methods for Optimizing Sample Space
图 14 辅助任务学习各方法关系示意图
Fig. 14 Diagram of the Relationships Among Various Methods in Auxiliary Task Learning
图 20 类间样本数量长尾分布与类内属性长尾分布示例图
Fig. 20 Example Diagram of Long-Tail Distribution of Inter-Class Sample Counts and Intra-Class Attributes
表 1 常见长尾数据集基本信息
Table 1 Basic Information of Common Long-Tail Datasets
类型 数据集 类别数量 训练集样本数量 测试集样本数量 最大类样本数量 最小类样本数量 图像分类 CIFAR10-LT[13] 10 50000 10000 5000 5($ \rho$=100), 50($ \rho$=10) 图像分类 CIFAR100-LT[13] 100 50000 10000 500 5($ \rho$=100), 50($ \rho$=10) 目标检测 ImageNet-LT[62] 1000 115846 50000 1280 5 场景识别 Places-LT[62] 365 62500 36500 4980 5 人脸识别 MS1M-LT[62] 74500 (ID)887530 3530 598 1 目标检测 iNaturalist2017[63] 5089 579184 182707 196613 381 目标检测 iNaturalist 2018[63] 8142 437513 24426 127551 19 实例分割 LVIS v0.5[64] 1230 57000 20000 26148 1 实例分割 LVIS v1[64] 1203 100170 19822 50552 1 场景理解 SUN-LT[65] 397 4084 2868 12 2 目标检测 AWA-LT[65] 50 6713 6092 720 2 鸟类识别 CUB-LT[65] 200 2945 2348 43 2或3 图像分类 STL10-LT[66] 10 5000 8000 500 5($ \rho$=100), 50($ \rho$=10) 目标检测 VOC-LT[67] 20 1142 4952 775 4 视频分类 VideoLT[68] 1004 179352 51244 1912 44 
下载: 导出CSV
表 2 长尾图像识别方法比较
Table 2 Comparison of Long-Tail Image Recognition Methods
分类 代表性文献 优点 缺点 优化样本空间 重采样 [1, 2, 56, 80, 70, 82, 30, 169] 简单通用, 理论直观, 易于操作 1)会丢弃大量头部类有效信息
2)重复采样尾部类不能增加有效信息, 并容易引发过拟合
3)易引入其它噪声数据增强 [2, 8, 9, 15, 76, 88, 89, 94, 95] 样本变换法成本较低, 易与其它方法结合, 灵活性较高. 语义增强法丰富尾部样本的语义信息, 生成具有现实意义的新样本 1)样本变换法引入大量新数据, 增加模型训练成本, 且可能生成毫无意义的样本, 鲁棒性较差.
2)语义增强方法需设计专门的模型结构, 操作复杂. 并过于依赖于头部类数据质量, 易给模型带来新的偏置.优化模型空间 优化特征提取器 [107, 108, 109, 111, 112, 170] 有效增强样本上下文语义特征帮助模型学到无偏的特征表示 1)引入大量参数, 占用内存, 降低训练效率
2)可解释性较差优化分类器 [1, 16, 26, 113, 115, 116, 118, 119] 计算开销小, 训练稳定无需设计额外的损失函数或存储单元 1)对超参数和优化器的选择敏感, 试错代价高
2)灵活性较低, 在目标检测与实例分割任务上表现不佳logits调整 [12, 28, 30, 55, 71, 120, 122] 既能优化训练过程, 又能进行事后修正. 计算开销较低, 泛化性能良好, 易与其它方法结合. 1)依赖于数据集的先验分布
2)修正后的边际分布可能不满足期望分布.代价敏感加权损失函数 [11, 12, 54, 72, 127, 129, 133] 操作简单, 易于实现, 计算开销较小, 适应于实际应用场景 1)优化困难, 参数敏感, 难以处理大规模真实场景
2)头尾性能像“跷跷板”, 无法从本质上解决信息缺乏的问题辅助任务学习 解耦学习 [1, 14, 134, 135, 138, 139] 利用大量头部类数据生成泛化能力良好的特征表示能够有效提升模型性能, 且计算成本较低. 1)两阶段方法不利于端到端的模型训练与部署
2)对数据依赖性较强
3)与其它算法结合使用时需重新设计, 实用性不强度量学习 [40, 58, 59, 127, 145, 149, 151] 便于公式化与计算构建一个正样本接近, 负样本远离的特征空间, 优化决策边界. 1)尾部类样本极少的情况下性能很差.
2)依赖于度量损失函数的设计知识蒸馏 [17, 19, 36, 145, 147, 154] 重用模型资源, 充分利用数据集蕴含的知识. 稳定尾部类学习过程 1)计算开销大, 优化成本相对过高, 对超参数敏感
2)易出现师生不匹配问题, 整体性能过于依赖教师模型的学习情况集成学习 [18, 19, 20, 158, 159, 161] 在头部类和尾部类上都能保持良好性能泛化能力良好, 能够处理未知分布的测试集 1)计算和存储负担过大, 框架部署复杂
2)专家之间存在相互影响的情况, 难以有效整合专家层次学习 [23, 24, 25, 162] 对数据间的关系进行多粒度建模, 捕捉类间隐式语义关系有助于头尾知识迁移 1)模型设计复杂, 训练成本较高
2)依赖于高质量数据, 有时需要数据集提供外部信息
3)层次划分步骤对后续训练产生过大影响
下载: 导出CSV
-
[1] Kang B, Xie S, Rohrbach M, et al. Decoupling representation and classifier for long-tailed recognition[J]. arXiv preprint arXiv: 1910.09217, 2019. [2] Zhang Y, Wei X S, Zhou B, et al. Bag of tricks for long-tailed visual recognition with deep convolutional neural networks[C]//Proceedings of the AAAI conference on artificial intelligence. 2021, 35(4): 3447−3455. [3] Wang J, Zhang W, Zang Y, et al. Seesaw loss for long-tailed instance segmentation[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 9695−9704. [4] Fu Y, Xiang L, Zahid Y, et al. Long-tailed visual recognition with deep models: A methodological survey and evaluation. Neurocomputing, 2022 [5] Yang L, Jiang H, Song Q, et al. A survey on long-tailed visual recognition. International Journal of Computer Vision, 2022, 130(7): 1837−1872 doi: 10.1007/s11263-022-01622-8 [6] Drummond C, Holte R C. C4. 5, class imbalance, and cost sensitivity: why under-sampling beats over-sampling[C]//Workshop on learning from imbalanced datasets II. 2003, 11: 1−8. [7] Shen L, Lin Z, Huang Q. Relay backpropagation for effective learning of deep convolutional neural networks[C]//Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part VII 14. Springer International Publishing, 2016: 467−482. [8] Chou H P, Chang S C, Pan J Y, et al. Remix: rebalanced mixup[C]//Computer Vision–ECCV 2020 Workshops: Glasgow, UK, August 23–28, 2020, Proceedings, Part VI 16. Springer International Publishing, 2020: 95−110. [9] Kim J, Jeong J, Shin J. M2m: Imbalanced classification via major-to-minor translation[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020: 13896−13905. [10] Chu P, Bian X, Liu S, et al. Feature space augmentation for long-tailed data[C]//Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXIX 16. Springer International Publishing, 2020: 694−710. [11] Cui Y, Jia M, Lin T Y, et al. Class-balanced loss based on effective number of samples[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019: 9268−9277. [12] Tan J, Wang C, Li B, et al. Equalization loss for long-tailed object recognition[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020: 11662−11671. [13] Cao K, Wei C, Gaidon A, et al. Learning imbalanced datasets with label-distribution-aware margin loss. Advances in neural information processing systems, 201932 [14] Zhou B, Cui Q, Wei X S, et al. Bbn: Bilateral-branch network with cumulative learning for long-tailed visual recognition[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020: 9719−9728. [15] Zhou A, Tajwar F, Robey A, et al. Do deep networks transfer invariances across classes?[J]. arXiv preprint arXiv: 2203.09739, 2022. [16] Liu B, Li H, Kang H, et al. Gistnet: a geometric structure transfer network for long-tailed recognition[C]//Proceedings of the IEEE/CVF international conference on computer vision. 2021: 8209−8218. [17] Zhang S, Chen C, Hu X, et al. Balanced knowledge distillation for long-tailed learning. Neurocomputing, 2023, 527: 36−46 doi: 10.1016/j.neucom.2023.01.063 [18] Sharma S, Yu N, Fritz M, et al. Long-tailed recognition using class-balanced experts[C]//Pattern Recognition: 42nd DAGM German Conference, DAGM GCPR 2020, Tübingen, Germany, September 28–October 1, 2020, Proceedings 42. Springer International Publishing, 2021: 86−100. [19] Xiang L, Ding G, Han J. Learning from multiple experts: Self-paced knowledge distillation for long-tailed classification[C]//Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part V 16. Springer International Publishing, 2020: 247−263. [20] Cai J, Wang Y, Hwang J N. Ace: Ally complementary experts for solving long-tailed recognition in one-shot[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021: 112−121. [21] Cai J, Wang Y, Hsu H M, et al. Luna: Localizing unfamiliarity near acquaintance for open-set long-tailed recognition[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2022, 36(1): 131−139. [22] Liu X, Zhang J, Hu T, et al. Inducing Neural Collapse in Deep Long-tailed Learning[C]//International Conference on Artificial Intelligence and Statistics. PMLR, 2023: 11534−11544. [23] Wu J, Song L, Zhang Q, et al. Forestdet: Large-vocabulary long-tailed object detection and instance segmentation. IEEE Transactions on Multimedia, 2021, 24: 3693−3705 [24] Ouyang W, Wang X, Zhang C, et al. Factors in finetuning deep model for object detection with long-tail distribution[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 864−873. [25] Li B. Adaptive Hierarchical Representation Learning for Long-Tailed Object Detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 2313−2322. [26] Tang K, Huang J, Zhang H. Long-tailed classification by keeping the good and removing the bad momentum causal effect. Advances in Neural Information Processing Systems, 2020, 33: 1513−1524 [27] Zhu B, Niu Y, Hua X S, et al. Cross-domain empirical risk minimization for unbiased long-tailed classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2022, 36(3): 3589−3597. [28] Wu T, Liu Z, Huang Q, et al. Adversarial robustness under long-tailed distribution[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 8659−8668. [29] Wang Y X, Ramanan D, Hebert M. Learning to model the tail. Advances in neural information processing systems, 201730 [30] Ren J, Yu C, Ma X, et al. Balanced meta-softmax for long-tailed visual recognition. Advances in neural information processing systems, 2020, 33: 4175−4186 [31] Dong B, Zhou P, Yan S, et al. Lpt: Long-tailed prompt tuning for image classification[J]. arXiv preprint arXiv: 2210.01033, 2022. [32] Tang K, Tao M, Qi J, et al. Invariant feature learning for generalized long-tailed classification[C]//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 709−726. [33] Zhang R, Haihong E, Yuan L, et al. MBNM: multi-branch network based on memory features for long-tailed medical image recognition. Computer Methods and Programs in Biomedicine, 2021, 212: 106448 doi: 10.1016/j.cmpb.2021.106448 [34] Ju L, Yu Z, Wang L, et al. Hierarchical Knowledge Guided Learning for Real-world Retinal Disease Recognition. IEEE Transactions on Medical Imaging, 2023 [35] Yang Z, Pan J, Yang Y, et al. Proco: Prototype-aware contrastive learning for long-tailed medical image classification[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer Nature Switzerland, 2022: 173−182. [36] Zhao W, Liu J, Liu Y, et al. Teaching teachers first and then student: Hierarchical distillation to improve long-tailed object recognition in aerial images. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1−12 [37] Li G, Pan L, Qiu L, et al. A Two-Stage Shake-Shake Network for Long-Tailed Recognition of SAR Aerial View Objects[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 249−256. [38] Jiao W, Zhang J. Sonar Images Classification While Facing Long-Tail and Few-Shot. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1−20 [39] Guo S, Liu R, Wang M, et al. Exploiting the Tail Data for Long-Tailed Face Recognition. IEEE Access, 2022, 10: 97945−97953 doi: 10.1109/ACCESS.2022.3206040 [40] Zhang X, Fang Z, Wen Y, et al. Range loss for deep face recognition with long-tail[J]. arXiv preprint arXiv: 1611.08976, 2016. [41] Moon W J, Seong H S, Heo J P. Minority-Oriented Vicinity Expansion with Attentive Aggregation for Video Long-Tailed Recognition[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2023, 37(2): 1931−1939. [42] Zhang C, Ren L, Wang J, et al. Making Pre-trained Language Models Good Long-tailed Learners[J]. arXiv preprint arXiv: 2205.05461, 2022. [43] Li Y, Shen T, Long G, et al. Improving long-tail relation extraction with collaborating relation-augmented attention[J]. arXiv preprint arXiv: 2010.03773, 2020. [44] Huang Y, Giledereli B, Köksal A, et al. Balancing methods for multi-label text classification with long-tailed class distribution[J]. arXiv preprint arXiv: 2109.04712, 2021. [45] Li X, Sun X, Meng Y, et al. Dice loss for data-imbalanced NLP tasks[J]. arXiv preprint arXiv: 1911.02855, 2019. [46] Conde M V, Choi U J. Few-shot long-tailed bird audio recognition[J]. arXiv preprint arXiv: 2206.11260, 2022. [47] Chen Z, Chen J, Xie Z, et al. Multi-expert Attention Network with Unsupervised Aggregation for long-tailed fault diagnosis under speed variation. Knowledge-Based Systems, 2022, 252: 109393 doi: 10.1016/j.knosys.2022.109393 [48] Sreepada R S, Patra B K. Mitigating long tail effect in recommendations using few shot learning technique. Expert Systems with Applications, 2020, 140: 112887 doi: 10.1016/j.eswa.2019.112887 [49] Chaudhary A, Gupta H P, Shukla K K. Real-Time Activities of Daily Living Recognition Under Long-Tailed Class Distribution. IEEE Transactions on Emerging Topics in Computational Intelligence, 2022, 6(4): 740−750 doi: 10.1109/TETCI.2022.3150757 [50] Zhang Y, Kang B, Hooi B, et al. Deep long-tailed learning: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023 [51] Pareto V. Cours d'économie politique[M]. Librairie Droz, 1964. [52] Zipf G K. The meaning-frequency relationship of words. The Journal of general psychology, 1945, 33(2): 251−256 doi: 10.1080/00221309.1945.10544509 [53] Hitt M A. The long tail: Why the future of business is selling less of more[J]. 2007. [54] Tan J, Lu X, Zhang G, et al. Equalization loss v2: A new gradient balance approach for long-tailed object detection[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 1685−1694. [55] Zhang S, Li Z, Yan S, et al. Distribution alignment: A unified framework for long-tail visual recognition[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 2361−2370. [56] Sinha S, Ohashi H, Nakamura K. Class-difficulty based methods for long-tailed visual recognition. International Journal of Computer Vision, 2022, 130(10): 2517−2531 doi: 10.1007/s11263-022-01643-3 [57] Cui J, Zhong Z, Liu S, et al. Parametric contrastive learning[C]//Proceedings of the IEEE/CVF international conference on computer vision. 2021: 715−724. [58] Kang B, Li Y, Xie S, et al. Exploring balanced feature spaces for representation learning[C]//International Conference on Learning Representations. 2020. [59] Li T, Cao P, Yuan Y, et al. Targeted supervised contrastive learning for long-tailed recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 6918−6928. [60] 叶志飞, 文益民, 吕宝粮. 不平衡分类问题研究综述. 智能系统学报, 2009, 4(002): 148−156 doi: 10.3969/j.issn.1673-4785.2009.02.010 [61] 赵凯琳, 靳小龙, 王元卓. 小样本学习研究综述. 软件学报, 2020, 32(2): 349−369 [62] Liu Z, Miao Z, Zhan X, et al. Large-scale long-tailed recognition in an open world[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019: 2537−2546. [63] Van Horn G, Mac Aodha O, Song Y, et al. The inaturalist species classification and detection dataset[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 8769−8778. [64] Gupta A, Dollar P, Girshick R. Lvis: A dataset for large vocabulary instance segmentation[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019: 5356−5364. [65] Samuel D, Atzmon Y, Chechik G. From generalized zero-shot learning to long-tail with class descriptors[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2021: 286−295. [66] Oh, Y., Kim, D. J., & Kweon, I. S. (2022). Daso: Distribution-aware semantics-oriented pseudo-label for imbalanced semi-supervised learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 9786−9796). [67] Wu T, Huang Q, Liu Z, et al. Distribution-balanced loss for multi-label classification in long-tailed datasets[C]//Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IV 16. Springer International Publishing, 2020: 162−178. [68] Zhang X, Wu Z, Weng Z, et al. Videolt: Large-scale long-tailed video recognition[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021: 7960−7969. [69] Feng C, Zhong Y, Huang W. Exploring classification equilibrium in long-tailed object detection[C]//Proceedings of the IEEE/CVF International conference on computer vision. 2021: 3417−3426. [70] Shrivastava A, Gupta A, Girshick R. Training region-based object detectors with online hard example mining[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 761−769. [71] Zhao Y, Chen W, Tan X, et al. Adaptive logit adjustment loss for long-tailed visual recognition[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2022, 36(3): 3472−3480. [72] Chen Z, Casser V, Kretzschmar H, et al. GradTail: learning long-tailed data using gradient-based sample weighting[J]. arXiv preprint arXiv: 2201.05938, 2022. [73] Wah C, Branson S, Welinder P, et al. The caltech-ucsd birds-200-2011 dataset[J]. 2011. [74] Zhou B, Lapedriza A, Khosla A, et al. Places: A 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence, 2017, 40(6): 1452−1464 [75] Coates A, Ng A, Lee H. An analysis of single-layer networks in unsupervised feature learning[C]//Proceedings of the fourteenth international conference on artificial intelligence and statistics. JMLR Workshop and Conference Proceedings, 2011: 215−223. [76] Zang Y, Huang C, Loy C C. Fasa: Feature augmentation and sampling adaptation for long-tailed instance segmentation[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021: 3457−3466. [77] Park S, Hong Y, Heo B, et al. The majority can help the minority: Context-rich minority oversampling for long-tailed classification[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 6887−6896. [78] Li B, Han Z, Li H, et al. Trustworthy long-tailed classification[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 6970−6979. [79] Wang T, Zhu Y, Chen Y, et al. C2am loss: Chasing a better decision boundary for long-tail object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 6980−6989. [80] Buda M, Maki A, Mazurowski M A. A systematic study of the class imbalance problem in convolutional neural networks. Neural networks, 2018, 106: 249−259 doi: 10.1016/j.neunet.2018.07.011 [81] Haixiang G, Yijing L, Shang J, et al. Learning from class-imbalanced data: Review of methods and applications. Expert systems with applications, 2017, 73: 220−239 doi: 10.1016/j.eswa.2016.12.035 [82] Chawla N V, Bowyer K W, Hall L O, et al. SMOTE: synthetic minority over-sampling technique. Journal of artificial intelligence research, 2002, 16: 321−357 doi: 10.1613/jair.953 [83] Jaiswal A, Babu A R, Zadeh M Z, et al. A survey on contrastive self-supervised learning. Technologies, 2020, 9(1): 2 doi: 10.3390/technologies9010002 [84] Zhang H, Cisse M, Dauphin Y N, et al. mixup: Beyond empirical risk minimization[J]. arXiv preprint arXiv: 1710.09412, 2017. [85] Yun S, Han D, Oh S J, et al. Cutmix: Regularization strategy to train strong classifiers with localizable features[C]//Proceedings of the IEEE/CVF international conference on computer vision. 2019: 6023−6032. [86] Verma V, Lamb A, Beckham C, et al. Manifold mixup: Better representations by interpolating hidden states[C]//International conference on machine learning. PMLR, 2019: 6438−6447. [87] Zhang S, Chen C, Zhang X, et al. Label-occurrence-balanced mixup for long-tailed recognition[C]//ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022: 3224−3228. [88] Zhang C, Pan T Y, Li Y, et al. Mosaicos: a simple and effective use of object-centric images for long-tailed object detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021: 417−427. [89] Liu B, Li H, Kang H, et al. Breadcrumbs: Adversarial class-balanced sampling for long-tailed recognition[C]//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 637−653. [90] Liu J, Li W, Sun Y. Memory-based jitter: Improving visual recognition on long-tailed data with diversity in memory[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2022, 36(2): 1720−1728. [91] Kingma D P, Welling M. An introduction to variational autoencoders. Foundations and Trends® in Machine Learning, 2019, 12(4): 307−392 [92] Rangwani H, Jaswani N, Karmali T, et al. Improving GANs for Long-Tailed Data Through Group Spectral Regularization[C]//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 426−442. [93] Rodriguez M G, Balduzzi D, Schölkopf B. Uncovering the temporal dynamics of diffusion networks[J]. arXiv preprint arXiv: 1105.0697, 2011. [94] Liu B, Li H, Kang H, et al. Semi-supervised long-tailed recognition using alternate sampling[J]. arXiv preprint arXiv: 2105.00133, 2021. [95] Wei C, Sohn K, Mellina C, et al. Crest: A class-rebalancing self-training framework for imbalanced semi-supervised learning[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 10857−10866. [96] Wang W, Zhao Z, Wang P, et al. Attentive feature augmentation for long-tailed visual recognition. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32(9): 5803−5816 doi: 10.1109/TCSVT.2022.3161427 [97] Wang Y, Pan X, Song S, et al. Implicit semantic data augmentation for deep networks. Advances in Neural Information Processing Systems, 201932 [98] Li S, Gong K, Liu C H, et al. Metasaug: Meta semantic augmentation for long-tailed visual recognition[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 5212−5221. [99] Zhao Y, Chen W, Huang K, et al. Feature Re-Balancing for Long-Tailed Visual Recognition[C]//2022 International Joint Conference on Neural Networks (IJCNN). IEEE, 2022: 1−8. [100] Vigneswaran R, Law M T, Balasubramanian V N, et al. Feature generation for long-tail classification[C]//Proceedings of the twelfth Indian conference on computer vision, graphics and image processing. 2021: 1−9. [101] Liu J, Sun Y, Han C, et al. Deep representation learning on long-tailed data: A learnable embedding augmentation perspective[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020: 2970−2979. [102] LeCun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998, 86(11): 2278−2324 doi: 10.1109/5.726791 [103] He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770−778. [104] Xie S, Girshick R, Dollár P, et al. Aggregated residual transformations for deep neural networks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 1492−1500. [105] He K, Gkioxari G, Dollár P, et al. Mask r-cnn[C]//Proceedings of the IEEE international conference on computer vision. 2017: 2961−2969. [106] Ren S, He K, Girshick R, et al. Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems, 201528 [107] Long A, Yin W, Ajanthan T, et al. Retrieval augmented classification for long-tail visual recognition[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022: 6959−6969. [108] Zhou J, Li J, Yan Y, et al. Mixing Global and Local Features for Long-Tailed Expression Recognition. Information, 2023, 14(2): 83 doi: 10.3390/info14020083 [109] Zhao W, Su Y, Hu M, et al. Hybrid ResNet based on joint basic and attention modules for long-tailed classification. International Journal of Approximate Reasoning, 2022, 150: 83−97 doi: 10.1016/j.ijar.2022.08.007 [110] Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need. Advances in neural information processing systems, 201730 [111] Chen J, Agarwal A, Abdelkarim S, et al. Reltransformer: A transformer-based long-tail visual relationship recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 19507−19517. [112] Hou Z, Yu B, Tao D. Batchformer: Learning to explore sample relationships for robust representation learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 7256−7266. [113] Ye H J, Chen H Y, Zhan D C, et al. Identifying and compensating for feature deviation in imbalanced deep learning[J]. arXiv preprint arXiv: 2001.01385, 2020. [114] Djouadi A, Bouktache E. A fast algorithm for the nearest-neighbor classifier. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1997, 19(3): 277−282 doi: 10.1109/34.584107 [115] Wei X S, Xu S L, Chen H, et al. Prototype-based classifier learning for long-tailed visual recognition. Science China Information Sciences, 2022, 65(6): 160105 doi: 10.1007/s11432-021-3489-1 [116] Parisot S, Esperança P M, McDonagh S, et al. Long-tail recognition via compositional knowledge transfer[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 6939−6948. [117] Wu T Y, Morgado P, Wang P, et al. Solving long-tailed recognition with deep realistic taxonomic classifier[C]//Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VIII 16. Springer International Publishing, 2020: 171−189. [118] Jia Y, Peng X, Wang R, et al. Long-tailed Partial Label Learning by Head Classifier and Tail Classifier Cooperation[J]. 2024. [119] Duggal R, Freitas S, Dhamnani S, et al. Elf: An early-exiting framework for long-tailed classification[J]. arXiv preprint arXiv: 2006.11979, 2020. [120] Menon A K, Jayasumana S, Rawat A S, et al. Long-tail learning via logit adjustment[J]. arXiv preprint arXiv: 2007.07314, 2020. [121] Wang Y, Zhang B, Hou W, et al. Margin calibration for long-tailed visual recognition[C]//Asian Conference on Machine Learning. PMLR, 2023: 1101−1116. [122] Li M, Cheung Y, Lu Y. Long-tailed visual recognition via gaussian clouded logit adjustment[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 6929−6938. [123] Hong Y, Han S, Choi K, et al. Disentangling label distribution for long-tailed visual recognition[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 6626−6636. [124] Xu Z, Yang S, Wang X, et al. Rethink Long-Tailed Recognition with Vision Transforms[C]//ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023: 1−5. [125] He Y Y, Zhang P, Wei X S, et al. Relieving long-tailed instance segmentation via pairwise class balance[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 7000−7009. [126] Long H, Zhang X, Liu Y, et al. Mutual Exclusive Modulator for Long-Tailed Recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023: 4890−4899. [127] Huang C, Li Y, Loy C C, et al. Learning deep representation for imbalanced classification[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 5375−5384. [128] Jamal M A, Brown M, Yang M H, et al. Rethinking class-balanced methods for long-tailed visual recognition from a domain adaptation perspective[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020: 7610−7619. [129] Hsieh T I, Robb E, Chen H T, et al. Droploss for long-tail instance segmentation[C]//Proceedings of the AAAI conference on artificial intelligence. 2021, 35(2): 1549−1557. [130] Park S, Lim J, Jeon Y, et al. Influence-balanced loss for imbalanced visual classification[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021: 735−744. [131] Lin T Y, Goyal P, Girshick R, et al. Focal loss for dense object detection[C]//Proceedings of the IEEE international conference on computer vision. 2017: 2980−2988. [132] Smith L N. Cyclical focal loss[J]. arXiv preprint arXiv: 2202.08978, 2022. [133] Li B, Yao Y, Tan J, et al. Equalized focal loss for dense long-tailed object detection[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022: 6990−6999. [134] Wang T, Li Y, Kang B, et al. The devil is in classification: A simple framework for long-tail instance segmentation[C]//Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XIV 16. Springer International Publishing, 2020: 728−744. [135] Li Y, Wang T, Kang B, et al. Overcoming classifier imbalance for long-tail object detection with balanced group softmax[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020: 10991−11000. [136] Zhong Z, Cui J, Liu S, et al. Improving calibration for long-tailed recognition[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 16489−16498. [137] Fan S, Zhang X, Song Z, et al. Cumulative dual-branch network framework for long-tailed multi-class classification. Engineering Applications of Artificial Intelligence, 2022, 114: 105080 doi: 10.1016/j.engappai.2022.105080 [138] Guo H, Wang S. Long-tailed multi-label visual recognition by collaborative training on uniform and re-balanced samplings[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021: 15089−15098. [139] Bengio Y, Lamblin P, Popovici D, et al. Greedy layer-wise training of deep networks. Advances in neural information processing systems, 200619 [140] Alshammari S, Wang Y X, Ramanan D, et al. Long-tailed recognition via weight balancing[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 6897−6907. [141] Zhu Z, Xing H, Xu Y. Easy balanced mixing for long-tailed data. Knowledge-Based Systems, 2022, 248: 108816 doi: 10.1016/j.knosys.2022.108816 [142] Yang Y, Xu Z. Rethinking the value of labels for improving class-imbalanced learning. Advances in neural information processing systems, 2020, 33: 19290−19301 [143] Liu X, Hu Y S, Cao X S, et al. Long-tailed class incremental learning[C]//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 495−512. [144] Deng J, Dong W, Socher R, et al. Imagenet: A large-scale hierarchical image database[C]//2009 IEEE conference on computer vision and pattern recognition. Ieee, 2009: 248−255. [145] Wang Y, Gan W, Yang J, et al. Dynamic curriculum learning for imbalanced data classification[C]//Proceedings of the IEEE/CVF international conference on computer vision. 2019: 5017−5026. [146] Wei T, Shi J X, Tu W W, et al. Robust long-tailed learning under label noise[J]. arXiv preprint arXiv: 2108.11569, 2021. [147] Li T, Wang L, Wu G. Self supervision to distillation for long-tailed visual recognition[C]//Proceedings of the IEEE/CVF international conference on computer vision. 2021: 630−639. [148] Chen T, Kornblith S, Norouzi M, et al. A simple framework for contrastive learning of visual representations[C]//International conference on machine learning. PMLR, 2020: 1597−1607. [149] Wang P, Han K, Wei X S, et al. Contrastive learning based hybrid networks for long-tailed image classification[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 943−952. [150] Fu S, Chu H, He X, et al. Meta-prototype Decoupled Training for Long-Tailed Learning[C]//Proceedings of the Asian Conference on Computer Vision. 2022: 569−585. [151] Zhong Z, Cui J, Li Z, et al. Rebalanced Siamese Contrastive Mining for Long-Tailed Recognition[J]. arXiv preprint arXiv: 2203.11506, 2022. [152] Zhu J, Wang Z, Chen J, et al. Balanced contrastive learning for long-tailed visual recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 6908−6917. [153] Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network[J]. arXiv preprint arXiv: 1503.02531, 2015. [154] Iscen A, Araujo A, Gong B, et al. Class-balanced distillation for long-tailed visual recognition[J]. arXiv preprint arXiv: 2104.05279, 2021. [155] He Y Y, Wu J, Wei X S. Distilling virtual examples for long-tailed recognition[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021: 235−244. [156] Xia Y, Zhang S, Wang J, et al. One‐stage self‐distillation guided knowledge transfer for long‐tailed visual recognition. International Journal of Intelligent Systems, 2022, 37(12): 11893−11908 doi: 10.1002/int.23068 [157] Yang C Y, Hsu H M, Cai J, et al. Long-tailed recognition of sar aerial view objects by cascading and paralleling experts[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021: 142−148. [158] Cui J, Liu S, Tian Z, et al. Reslt: Residual learning for long-tailed recognition. IEEE transactions on pattern analysis and machine intelligence, 2022, 45(3): 3695−3706 [159] Wang X, Lian L, Miao Z, et al. Long-tailed recognition by routing diverse distribution-aware experts[J]. arXiv preprint arXiv: 2010.01809, 2020. [160] Li J, Tan Z, Wan J, et al. Nested collaborative learning for long-tailed visual recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022: 6949−6958. [161] Zhang Y, Hooi B, Hong L, et al. Self-supervised aggregation of diverse experts for test-agnostic long-tailed recognition. Advances in Neural Information Processing Systems, 2022, 35: 34077−34090 [162] Chen Q, Liu Q, Lin E. A knowledge-guide hierarchical learning method for long-tailed image classification. Neurocomputing, 2021, 459: 408−418 doi: 10.1016/j.neucom.2021.07.008 [163] Li Z, Zhao H, Lin Y. Multi-task convolutional neural network with coarse-to-fine knowledge transfer for long-tailed classification. Information Sciences, 2022, 608: 900−916 doi: 10.1016/j.ins.2022.07.015 [164] Wen Y, Zhang K, Li Z, et al. A discriminative feature learning approach for deep face recognition[C]//Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part VII 14. Springer International Publishing, 2016: 499−515. [165] Cao D, Zhu X, Huang X, et al. Domain balancing: Face recognition on long-tailed domains[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 5671−5679. [166] Ma Y, Jiao L, Liu F, et al. Delving into Semantic Scale Imbalance[J]. arXiv preprint arXiv: 2212.14613, 2022. [167] Park B, Kim J, Cho S, et al. Balancing Domain Experts for Long-Tailed Camera-Trap Recognition[J]. arXiv preprint arXiv: 2202.07215, 2022. [168] Wang W, Wang M, Wang S, et al. One-shot learning for long-tail visual relation detection[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2020, 34(07): 12225−12232. [169] Chang N, Yu Z, Wang Y X, et al. Image-level or object-level? a tale of two resampling strategies for long-tailed detection[C]//International conference on machine learning. PMLR, 2021: 1463−1472. [170] Zhang C, Lin G, Lai L, et al. Calibrating Class Activation Maps for Long-Tailed Visual Recognition[J]. arXiv preprint arXiv: 2108.12757, 2021. [171] Cao Y, Kuang J, Gao M, et al. Learning relation prototype from unlabeled texts for long-tail relation extraction. IEEE Transactions on Knowledge and Data Engineering, 2021 [172] Zhang G, Liang R, Yu Z, et al. Rumour detection on social media with long-tail strategy[C]//2022 International Joint Conference on Neural Networks (IJCNN). IEEE, 2022: 1−8. [173] Mottaghi A, Sarma P K, Amatriain X, et al. Medical symptom recognition from patient text: An active learning approach for long-tailed multilabel distributions[J]. arXiv preprint arXiv: 2011.06874, 2020. [174] Shi C, Hu B, Zhao W X, et al. Heterogeneous information network embedding for recommendation. IEEE Transactions on Knowledge and Data Engineering, 2018, 31(2): 357−370 [175] Zhao T, Zhang X, Wang S. Graphsmote: Imbalanced node classification on graphs with graph neural networks[C]//Proceedings of the 14th ACM international conference on web search and data mining. 2021: 833−841. [176] Park J, Song J, Yang E. Graphens: Neighbor-aware ego network synthesis for class-imbalanced node classification[C]//International Conference on Learning Representations. 2021. [177] Yun S, Kim K, Yoon K, et al. Lte4g: long-tail experts for graph neural networks[C]//Proceedings of the 31st ACM International Conference on Information & Knowledge Management. 2022: 2434−2443. [178] Hu Z, Dong Y, Wang K, et al. Gpt-gnn: Generative pre-training of graph neural networks[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2020: 1857−1867. [179] Liu Z, Nguyen T K, Fang Y. Tail-gnn: Tail-node graph neural networks[C]//Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2021: 1109−1119. [180] Perrett T, Sinha S, Burghardt T, et al. Use Your Head: Improving Long-Tail Video Recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023: 2415−2425. [181] Tian C, Wang W, Zhu X, et al. Vl-ltr: Learning class-wise visual-linguistic representation for long-tailed visual recognition[C]//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 73−91. [182] Ma T, Geng S, Wang M, et al. A simple long-tailed recognition baseline via vision-language model[J]. arXiv preprint arXiv: 2111.14745, 2021. [183] Wang R, Yu G, Domeniconi C, et al. Meta Cross-Modal Hashing on Long-Tailed Data[J]. arXiv preprint arXiv: 2111.04086, 2021. [184] Wang P, Wang X, Wang B, et al. Long-Tailed Time Series Classification via Feature Space Rebalancing[C]//International Conference on Database Systems for Advanced Applications. Cham: Springer Nature Switzerland, 2023: 151−166. [185] Deng J, Chen X, Jiang R, et al. St-norm: Spatial and temporal normalization for multi-variate time series forecasting[C]//Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining. 2021: 269−278. [186] Craw S, Horsburgh B, Massie S. Music recommendation: audio neighbourhoods to discover music in the long tail[C]//Case-Based Reasoning Research and Development: 23rd International Conference, ICCBR 2015, Frankfurt am Main, Germany, September 28-30, 2015. Proceedings 23. Springer International Publishing, 2015: 73−87. [187] Deng K, Cheng G, Yang R, et al. Alleviating asr long-tailed problem by decoupling the learning of representation and classification. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021, 30: 340−354 [188] Winata G I, Wang G, Xiong C, et al. Adapt-and-adjust: Overcoming the long-tail problem of multilingual speech recognition[J]. arXiv preprint arXiv: 2012.01687, 2020. [189] Peng P, Lu J, Tao S, et al. Progressively balanced supervised contrastive representation learning for long-tailed fault diagnosis. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 1−12 [190] Deng S, Lei Z, Liu J, et al. A Cost-Sensitive Dense Network for Fault Diagnosis under Data Imbalance[C]//2022 International Conference on Sensing, Measurement & Data Analytics in the era of Artificial Intelligence (ICSMD). IEEE, 2022: 1−6. [191] Jiao W, Zhang J. Sonar images classification while facing long-tail and few-shot. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1−20 [192] Shao J, Zhu K, Zhang H, et al. DiffuLT: How to Make Diffusion Model Useful for Long-tail Recognition[J]. arXiv preprint arXiv: 2403.05170, 2024. [193] Shi J X, Wei T, Zhou Z, et al. Parameter-Efficient Long-Tailed Recognition[J]. arXiv preprint arXiv: 2309.10019, 2023. [194] Kabir H M. Reduction of Class Activation Uncertainty with Background Information[J]. arXiv preprint arXiv: 2305.03238, 2023. [195] Du F, Yang P, Jia Q, et al. Global and Local Mixture Consistency Cumulative Learning for Long-tailed Visual Recognitions[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023: 15814−15823. [196] Chen X, Liang C, Huang D, et al. Symbolic discovery of optimization algorithms[J]. arXiv preprint arXiv: 2302.06675, 2023. [197] Cui J, Zhong Z, Tian Z, et al. Generalized parametric contrastive learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023 [198] Hendrycks D, Gimpel K. A baseline for detecting misclassified and out-of-distribution examples in neural networks[J]. arXiv preprint arXiv: 1610.02136, 2016. [199] Liu W, Wang X, Owens J, et al. Energy-based out-of-distribution detection. Advances in neural information processing systems, 2020, 33: 21464−21475 [200] Yang Y, Wang H, Katabi D. On multi-domain long-tailed recognition, imbalanced domain generalization and beyond[C]//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 57−75. [201] Kim C D, Jeong J, Kim G. Imbalanced continual learning with partitioning reservoir sampling[C]//Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XIII 16. Springer International Publishing, 2020: 411−428. [202] Ditzler G, Polikar R, Chawla N. An incremental learning algorithm for non-stationary environments and class imbalance[C]//2010 20th International Conference on Pattern Recognition. IEEE, 2010: 2997−3000. [203] Shi J X, Wei T, Li Y F. Residual diverse ensemble for long-tailed multi-label text classification. Science CHINA Information Science, 2024 [204] Kharbanda S, Gupta D, Schultheis E, et al. Learning label-label correlations in Extreme Multi-label Classification via Label Features[J]. arXiv preprint arXiv: 2405.04545, 2024. [205] Zhang Y, Cao S, Mi S, et al. Learning sample representativeness for class-imbalanced multi-label classification. Pattern Analysis and Applications, 20241−12 [206] Du C, Han Y, Huang G. SimPro: A Simple Probabilistic Framework Towards Realistic Long-Tailed Semi-Supervised Learning[J]. arXiv preprint arXiv: 2402.13505, 2024. [207] Ma C, Elezi I, Deng J, et al. Three heads are better than one: Complementary experts for long-tailed semi-supervised learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2024, 38(13): 14229−14237. [208] Shang X, Lu Y, Huang G, et al. Federated learning on heterogeneous and long-tailed data via classifier re-training with federated features[J]. arXiv preprint arXiv: 2204.13399, 2022. [209] Kou X, Xu C, Yang X, et al. Attention-guided Contrastive Hashing for Long-tailed Image Retrieval[C]//IJCAI. 2022: 1017−1023. [210] Geifman Y, El-Yaniv R. Deep active learning over the long tail[J]. arXiv preprint arXiv: 1711.00941, 2017. 期刊类型引用(8)
1. 刘文璇，钟忺，徐晓玉，周卓，江奎，王正，白翔. 空—地多视角行为识别的判别信息增量学习方法. 中国图象图形学报. 2025(01): 130-147 . 
百度学术2. 苏本跃，郭梦娟，朱邦国，盛敏. 顺序主导和方向驱动下基于点边特征的人体动作识别方法. 控制与决策. 2024(09): 3090-3098 . 百度学术3. 乔迤，曲毅. 基于自适应融合权重的人体行为识别方法. 计算机工程与设计. 2023(03): 845-851 . 百度学术4. 沈加炜，陆一鸣，陈晓艺，钱美玲，陆卫忠. 基于深度学习的人体行为检测方法研究综述. 计算机与现代化. 2023(09): 1-9 . 百度学术5. 曾明如，熊嘉豪，祝琴. 基于T-Fusion的TFP3D人体行为识别算法. 计算机集成制造系统. 2023(12): 4032-4039 . 百度学术6. 凌永标，毛峰，杨岚岚，邱兴卫，张志锐，张杰. 基于混合注意力网络的安全工器具检测. 计算机技术与发展. 2022(06): 209-214 . 百度学术7. 张海超，张闯. 融合注意力的轻量级行为识别网络研究. 电子测量与仪器学报. 2022(05): 173-179 . 百度学术8. 杨观赐，李杨，赵乐，刘赛赛，何玲，刘丹. 基于传感器数据的用户行为识别方法综述. 包装工程. 2021(18): 94-102+133+11 . 百度学术其他类型引用(19)
-
下载:
计量
- 文章访问数: 712
- HTML全文浏览量: 602
- 被引次数: 27
