大模型参数高效微调方法综述: 技术、趋势与挑战

唐岸达; 林宙辰

doi:10.16383/j.aas.c250451

大模型参数高效微调方法综述: 技术、趋势与挑战

doi: 10.16383/j.aas.c250451 cstr: 32138.14.j.aas.c250451

唐岸达^1,,
林宙辰^{1, 2, 3, 4,}

1.
北京大学智能学院北京 100871
2.
北京大学跨媒体通用人工智能全国重点实验室北京 100871
3.
北京大学人工智能研究院北京 100871
4.
琶洲实验室(黄埔) 广州 510335

基金项目: 国家自然科学基金(62276004), 北京市自然科学基金(L257007)资助

详细信息

作者简介:
唐岸达：北京大学智能学院博士后. 2024年获得中国科学院大学数学科学学院理学博士学位. 主要研究方向为机器学习、深度学习和优化. E-mail: tanganda@pku.edu.cn

林宙辰：北京大学智能学院教授. 2000年获得北京大学博士学位. 主要研究方向为机器学习、数值优化. 本文通信作者. E-mail: zlin@pku.edu.cn

计量
- 文章访问数: 1491
- HTML全文浏览量: 3203
- PDF下载量: 89
- 被引次数: 0
出版历程
- 收稿日期: 2025-09-04
- 录用日期: 2026-01-19
- 网络出版日期: 2026-03-19
- 刊出日期: 2026-08-20

A Survey on Parameter-efficient Fine-tuning of Large Models: Techniques, Trends, and Challenges

TANG An-Da^1
,,
LIN Zhou-Chen^{1, 2, 3, 4
,}

1.
School of Intelligence Science and Technology, Peking University, Beijing 100871
2.
State Key Laboratory of General AI, Peking University, Beijing 100871
3.
Institute for Artificial In-telligence, Peking University, Beijing 100871
4.
Pazhou Laboratory (Huangpu), Guangzhou 510335

Funds: Supported by National Natural Science Foundation of China (62276004) and Beijing Natural Science Foundation (L257007)

More Information

Author Bio:
TANG An-Da　Postdoctor at the School of Intelligence Science and Technology, Peking University. He received his Ph.D. degree in Science from the School of Mathematical Sciences, University of Chinese Academy of Sciences in 2024. His research interests include machine learning, deep learning, and optimization

LIN Zhou-Chen　Professor at the School of Intelligence Science and Technology, Peking University. He received his Ph.D. degree from Peking University in 2000. His research interests include machine learning and numerical optimization. Corresponding author of this paper

摘要

摘要: 大规模预训练模型在自然语言处理等多个领域表现优异. 为更好地适配下游任务, 微调预训练模型是一个常用的方法, 但全量微调面临高昂计算与存储成本. 参数高效微调 (PEFT) 通过仅更新极少量参数, 在降低开销的同时保持模型性能. 该综述系统梳理PEFT领域的主流方法. 首先, 将现有方法归纳为添加式、局部式、重参数化式和融合式四大范式, 并深入剖析各类方法的核心机理、性能特征、应用场景及策略优势. 进而, 探讨PEFT的技术演进, 总结出PEFT方法从单一方法创新走向存储、计算、性能的三元权衡, 并向自动化、智能化、软硬件协同等统一框架发展. 更进一步, 该综述对各类代表性PEFT方法进行性能与参数效率的定量比较. 此外, 本综述还涵盖PEFT在视觉、语音及跨模态模型等领域的拓展应用. 最后, 总结并探讨未来研究方向, 以推动更高效、更适应多样化任务的大模型微调技术的发展.
- 大语言模型 /
- 大规模预训练模型 /
- 参数高效微调 /
- 机器学习 /
- 深度学习
Abstract: Large-scale pre-trained models have demonstrated remarkable capabilities in various fields such as natural language processing. To better adapt these models to downstream tasks, fine-tuning these models is a commonly adopted approach. However, full fine-tuning faces severe challenges, including high computational costs and substantial storage requirements. Parameter-efficient fine-tuning (PEFT) addresses these issues by updating only a minimal number of parameters, thereby reducing overhead while preserving model performance. This survey provides a systematic review of the mainstream methodologies in the PEFT field. First, we categorize existing methods into four major paradigms: Additive, selective, reparameterized, and hybrid, offering an in-depth analysis of their core mechanisms, performance characteristics, application scenarios, and strategic advantages. Furthermore, we explore the technical evolution of PEFT, summarizing a clear trajectory. The field is moving from isolated method innovations towards a tri-balance among storage, computation, and performance, and further advancing into unified frameworks emphasizing automation, intelligence, and hardware-software co-design. Moreover, we conduct a quantitative comparison of representative PEFT methods in terms of performance and parameter efficiency. In addition, this survey also covers the extended applications of PEFT in fields such as vision, speech, and cross-modal models. Finally, we summarize and discuss promising future research directions to facilitate the development of more efficient and adaptable fine-tuning techniques for large-scale models across diverse tasks.
- large language models /
- large-scale pre-trained models /
- parameter-efficient fine-tuning /
- machine learning /
- deep learning

HTML全文

图 1 不同类型PEFT方法示意图

Fig. 1 Schematic diagram of different types of PEFT methods

下载: 全尺寸图片幻灯片

图 3 适配器结构图

Fig. 3 Adapter structure diagram

下载: 全尺寸图片幻灯片

图 2 PEFT方法分类图

Fig. 2 Classification diagram of PEFT methods

下载: 全尺寸图片幻灯片

图 4 LoRA模块的计算示意图

Fig. 4 Computational schematic diagram of the LoRA module

下载: 全尺寸图片幻灯片

表 1 PEFT方法分类对比

Table 1 Comparison of PEFT method classification

方法类别	核心思想	分类依据	参数可合并	推理延迟	典型代表方法
添加式	引入全新的可训练单元(模块/参数/前缀等), 与原始主干并行或串联工作	是否在原始模型结构外增加新的可训练计算单元到原计算图; 训练与推理均依赖该新增单元	否	低到高, 或无	Sequential Adapter, Prefix-tuning, Prompt-tuning, (IA)³
局部	激活或选择模型的一部分进行更新	是否修改网络结构, 是否通过门控、掩码或选择机制来动态决定使用模型的哪些部分; 计算图基本不变	是	低或无	Diff, BitFit, PaFi
重参数化	对原始参数进行低秩或结构性变换, 训练后可与原模型合并	是否通过一个参数化的变换(如矩阵分解、张量分解)来间接更新权重, 且推理时能还原为原始架构	是	低或无	LoRA, LoRTA, LoraHub, MultiLoRA, LoRAPrune, QLoRA
融合	结合上述多种策略以发挥各自优势	是否明确地融合了两种及以上不同类别的PEFT技术	视情况而定	视情况而定	UniPELT, MAM-Adapter, AUTOPEFT

下载: 导出CSV

表 2 PEFT代表性方法在RoBERTa-Large模型上的性能对比(%)

Table 2 Performance comparison of representative PEFT methods on RoBERTa-Large model (%)

类别	方法	作用位置	推理延迟	可训练参数	数据集					平均值
类别	方法	作用位置	推理延迟	可训练参数	SST-2	MRPC	CoLA	QNLI	RTE	平均值
—	全量微调	所有参数	无	100.00	96.1	90.2	68.0	94.2	86.0	87.1
添加式	Prefix-Tuning	注意力	有	0.11	95.6	86.8	59.1	94.6	74.8	82.2
	Prompt-Tuning	输入	有	0.30	94.6	73.0	61.1	89.1	60.3	75.6
	Sequential Adapter	前馈层	有	4.72	96.0	89.2	65.4	94.5	84.1	85.8
	(IA)³	注意力、前馈	无	0.34	94.6	86.5	61.1	94.2	91.2	85.5
局部	BitFit	注意力	无	0.41	96.1	90.9	68.0	94.5	87.7	87.4
	Child-tuning_D	—	无	0.10	95.1	90.7	63.1	93.1	86.3	85.7
	SparseGrad	前馈	无	47.32	96.8	90.5	63.2	93.3	64.7	81.7
	RoCoFT_1row	注意力、前馈	无	0.06	96.6	90.0	65.7	94.2	85.3	86.4
	RoCoFT_3row	注意力、前馈	无	0.18	96.7	91.1	67.4	94.9	87.8	87.6
	RoCoFT_1col	注意力、前馈	无	0.06	96.6	89.1	64.9	94.1	85.7	86.1
	RoCoFT_3col	注意力、前馈	无	0.18	96.7	89.9	67.2	94.8	87.8	87.3
重参数化	LoRA	注意力	无	0.24	96.2	90.2	68.2	94.8	85.2	86.9
	LoRA-FA	注意力	无	1.10	96.0	90.0	68.0	94.4	86.1	86.9
	AdaLoRA	注意力	无	0.24	95.0	90.4	66.9	94.6	84.5	86.3
	PiSSA	注意力	无	0.24	95.5	86.9	61.1	92.1	56.8	78.5
	FourierFT	注意力	无	0.01	96.0	90.9	67.1	94.4	87.4	87.2
	VeRA	注意力、前馈	无	0.02	96.1	90.9	68.0	94.4	85.9	87.1
	RoseLoRA	注意力、前馈	无	0.02	95.2	90.2	69.2	94.7	89.2	87.7
	LoRA-PRO	注意力	无	0.24	95.9	90.9	66.7	93.0	60.5	81.4
	LoRA-dropout	注意力	无	0.12	96.2	89.9	68.5	94.9	88.8	87.7
	WeGeFT	注意力	无	0.02	95.0	75.7	64.0	93.7	53.6	76.4
	EFlat-LoRA	注意力	无	0.24	96.3	90.3	68.0	94.8	89.3	87.7
	LoRETTA	注意力	无	0.04	96.2	90.5	69.5	94.1	53.0	80.7
	FoRA-UA	注意力	无	0.01	96.6	91.2	69.0	93.9	86.9	87.5
融合	MAM-Adapter	—	有	12.30	95.8	90.1	67.3	94.3	86.6	86.8
	ProPETL-Adapter	注意力、前馈	有	1.50	96.3	89.7	65.6	95.2	88.9	87.1
	ProPETL-Prefix	注意力	有	7.60	96.2	90.0	62.2	94.7	79.7	84.6
	ProPETL-LoRA	注意力	无	1.20	95.9	89.1	61.9	94.9	83.6	85.1

下载: 导出CSV

参考文献(263)

[1]	Li J Y, Tang T Y, Zhao W X, Nie J Y, Wen J R. Pre-trained language models for text generation: A survey. ACM Computing Surveys, 2024, 56(9): Article No. 230
[2]	Xia X, Zhang D, Liao Z B, Hou Z Y, Sun T R, Li J, et al. SceneGenAgent: Precise industrial scene generation with coding agent. In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Vienna, Austria: Association for Computational Linguistics, 2025. 17847−17875
[3]	Zhu W H, Liu H Y, Dong Q X, Xu J J, Huang S J, Kong L P, et al. Multilingual machine translation with large language models: Empirical results and analysis. In: Proceedings of the Findings of the Association for Computational Linguistics. Mexico City, Mexico: Association for Computational Linguistics, 2024. 2765−2781
[4]	Zheng L M, Chiang W L, Sheng Y, Zhuang S Y, Wu Z H, Zhuang Y H, et al. Judging LLM-as-a-judge with MT-bench and chatbot arena. In: Proceedings of the 37th International Conference on Neural Information Processing Systems. New Orleans, USA: Curran Associates Inc., 2023. Article No. 2020
[5]	Kim J K, Chua M, Rickard M, Lorenzo A. ChatGPT and large language model (LLM) chatbots: The current state of acceptability and a proposal for guidelines on utilization in academic medicine. Journal of Pediatric Urology, 2023, 19(5): 598−604 doi: 10.1016/j.jpurol.2023.05.018
[6]	Wang J J. The power of AI-assisted diagnosis. EAI Endorsed Transactions on e-Learning, 2022, 8(4): Article No. e3 doi: 10.4108/eetel.3772
[7]	Biswas S S. Role of ChatGPT in public health. Annals of Biomedical Engineering, 2023, 51(5): 868−869 doi: 10.1007/s10439-023-03172-7
[8]	Li H T, Chen J J, Yang J L, Ai Q Y, Jia W, Liu Y F, et al. LegalAgentBench: Evaluating LLM agents in legal domain. In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Vienna, Austria: Association for Computational Linguistics, 2025. 2322−2344
[9]	Xing F. Designing heterogeneous LLM agents for financial sentiment analysis. ACM Transactions on Management Information Systems, 2025, 16(1): Article No. 5 doi: 10.1145/3688399
[10]	Han Z Y, Gao C, Liu J Y, Zhang J, Zhang S Q. Parameter-efficient fine-tuning for large models: A comprehensive survey. arXiv preprint arXiv: 2403.14608, 2024.
[11]	Guo D M, Rush A, Kim Y. Parameter-efficient transfer learning with diff pruning. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Virtual Event: Association for Computational Linguistics, 2021. 4884−4896
[12]	Hu E, Shen Y L, Wallis P, Allen-Zhu Z, Li Y Z, Wang S A, et al. LoRA: Low-rank adaptation of large language models. arXiv preprint arXiv: 2106.09685, 2021.
[13]	Lialin V, Deshpande V, Rumshisky A. Scaling down to scale up: A guide to parameter-efficient fine-tuning. arXiv preprint arXiv: 2303.15647, 2023.
[14]	Vucetic D, Tayaranian M, Ziaeefard M, Clark J J, Meyer B H, Gross W J. Efficient fine-tuning of BERT models on the edge. In: Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS). Austin, USA: IEEE, 2022. 1838−1842
[15]	Fu Z H, Yang H R, So A M C, Lam W, Bing L D, Collier N. On the effectiveness of parameter-efficient fine-tuning. In: Proceedings of the 37th AAAI Conference on Artificial Intelligence. Washington, USA: AAAI Press, 2023. 12799−12807
[16]	Wang L P, Chen S, Jiang L N, Pan S, Cai R Z, Yang S, et al. Parameter-efficient fine-tuning in large models: A survey of methodologies. arXiv preprint arXiv: 2410.19878, 2024.
[17]	Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, et al. Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, USA: Curran Associates Inc., 2017. 6000−6010
[18]	Devlin J, Chang M W, Lee K, Toutanova K. BERT: Pre-training of deep bidirectional Transformers for language understanding. In: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Minneapolis, USA: Association for Computational Linguistics, 2019. 4171−4186
[19]	Achiam J, Adler S, Agarwal S, Ahmad L, Akkaya I, Aleman F L, et al. GPT-4 technical report. arXiv preprint arXiv: 2303.08774, 2023.
[20]	Wei J, Tay Y, Bommasani R, Raffel C, Zoph B, Borgeaud S, et al. Emergent abilities of large language models. arXiv preprint arXiv: 2206.0768, 2022.
[21]	Ouyang L, Wu J, Jiang X, Almeida D, Wainwright C L, Mishkin P, et al. Training language models to follow instructions with human feedback. In: Proceedings of the 36th International Conference on Neural Information Processing Systems. New Orleans, USA: Curran Associates Inc., 2022. Article No. 2011
[22]	Schulman J, Wolski F, Dhariwal P, Radford A, Klimov O. Proximal policy optimization algorithms. arXiv preprint arXiv: 1707.06347, 2017.
[23]	Rafailov R, Sharma A, Mitchell E, Ermon S, Manning C D, Finn C. Direct preference optimization: Your language model is secretly a reward model. In: Proceedings of the 37th International Conference on Neural Information Processing Systems. New Orleans, USA: Curran Associates Inc., 2023. Article No. 2338
[24]	Lee H, Phatale S, Mansoor H, Lu K, Mesnard T, Bishop C, et al. RLAIF: Scaling reinforcement learning from human feedback with AI feedback. arXiv preprint arXiv: 2309.00267, 2023.
[25]	Radford A, Narasimhan K, Salimans T, Sutskever I. Improving language understanding by generative pre-training [Online], available: https://www.mikecaptain.com/resources/pdf/GPT-1.pdf, July 1, 2026
[26]	Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I. Language models are unsupervised multitask learners. OpenAI Blog, 2019, 1(8): Article No. 9
[27]	Brown T B, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P, et al. Language models are few-shot learners. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. Vancouver, Canada: Curran Associates Inc., 2020. Article No. 159
[28]	Touvron H, Lavril T, Izacard G, Martinet X, Lachaux M A, Lacroix T, et al. LLaMA: Open and efficient foundation language models. arXiv preprint arXiv: 2302.13971, 2023.
[29]	Vavekanand R, Sam K. LLaMA 3.1: An in-depth analysis of the next-generation large language model [Online], available: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6139407, July 1, 2026
[30]	Bi X, Chen D L, Chen G T, Chen S H, Dai D M, Deng C Q, et al. DeepSeek LLM: Scaling open-source language models with longtermism. arXiv preprint arXiv: 2401.02954, 2024.
[31]	Liu A X, Feng B, Wang B, Wang B X, Liu B, Zhao C G, et al. DeepSeek-v2: A strong, economical, and efficient mixture-of-experts language model. arXiv preprint arXiv: 2405.04434, 2024.
[32]	Liu A X, Feng B, Xue B, Wang B X, Wu B C, Lu C D, et al. DeepSeek-v3 technical report. arXiv preprint arXiv: 2412.19437, 2024.
[33]	Guo D Y, Yang D J, Zhang H W, Song J X, Zhang R Y, Xu R X, et al. DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning. arXiv preprint arXiv: 2501.12948, 2025.
[34]	Bai J Z, Bai S, Chu Y F, Cui Z Y, Dang K, Deng X D, et al. Qwen technical report. arXiv preprint arXiv: 2309.16609, 2023.
[35]	Yang A, Li A F, Yang B S, Zhang B C, Hui B Y, Zheng B, et al. Qwen3 technical report. arXiv preprint arXiv: 2505.09388, 2025.
[36]	Ahmed I, Islam S, Datta P P, Kabir I, Chowdhury N U R, Haque A. Qwen 2.5: A comprehensive review of the leading resource-efficient LLM with potential to surpass all competitors [Online], available: https://www.techrxiv.org/doi/pdf/10.36227/techrxiv.174060306.65738406/v1, July 1, 2026
[37]	Caruccio L, Cirillo S, Polese G, Solimando G, Sundaramurthy S, Tortora G. Claude 2.0 large language model: Tackling a real-world classification problem with a new iterative prompt engineering approach. Intelligent Systems With Applications, 2024, 21: Article No. 200336 doi: 10.1016/j.iswa.2024.200336
[38]	Anil R, Borgeaud S, Alayrac J B, Yu J H, Soricut R, Schalkwyk J, et al. Gemini: A family of highly capable multimodal models. arXiv preprint arXiv: 2312.11805, 2023.
[39]	Groeneveld D, Beltagy I, Walsh E, Bhagia A, Kinney R, Tafjord O, et al. OLMo: Accelerating the science of language models. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Bangkok, Thailand: Association for Computational Linguistics, 2024. 15789−15809
[40]	Radford A, Kim J W, Hallacy C, Ramesh A, Goh G, Agarwal S, et al. Learning transferable visual models from natural language supervision. In: Proceedings of the 38th International Conference on Machine Learning (ICML). Virtual Event: PMLR, 2021. 8748−8763
[41]	Sun Q, Fang Y X, Wu L, Wang X L, Cao Y. EVA-CLIP: Improved training techniques for CLIP at scale. arXiv preprint arXiv: 2303.15389, 2023.
[42]	Kaplan J, McCandlish S, Henighan T, Brown T B, Chess B, Child R, et al. Scaling laws for neural language models. arXiv preprint arXiv: 2001.08361, 2020.
[43]	Hoffmann J, Borgeaud S, Mensch A, Buchatskaya E, Cai T, Rutherford E, et al. Training compute-optimal large language models. arXiv preprint arXiv: 2203.15556, 2022.
[44]	Wang A, Singh A, Michael J, Hill F, Levy O, Bowman S R. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In: Proceedings of the EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP. Brussels, Belgium: Association for Computational Linguistics, 2018. 353−355
[45]	Mihaylov T, Clark P, Khot T, Sabharwal A. Can a suit of armor conduct electricity? A new dataset for open book question answering. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Brussels, Belgium: Association for Computational Linguistics, 2018. 2381−2391
[46]	Bisk Y, Zellers R, le Bras R, Gao J F, Choi Y. PIQA: Reasoning about physical commonsense in natural language. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence. New York, USA: AAAI Press, 2020. 7432−7439
[47]	Sap M, Rashkin H, Chen D, le Bras R, Choi Y. Social IQa: Commonsense reasoning about social interactions. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Hong Kong, China: Association for Computational Linguistics, 2019. 4463−4473
[48]	Zellers R, Holtzman A, Bisk Y, Farhadi A, Choi Y. HellaSwag: Can a machine really finish your sentence? In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy: Association for Computational Linguistics, 2019. 4791−4800
[49]	Clark C, Lee K, Chang M W, Kwiatkowski T, Collins M, Toutanova K. BoolQ: Exploring the surprising difficulty of natural yes/no questions. In: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Minneapolis, USA: Association for Computational Linguistics, 2019. 2924−2936
[50]	Sakaguchi K, le Bras R, Bhagavatula C, Choi Y. WinoGrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 2021, 64(9): 99−106 doi: 10.1145/3474381
[51]	Clark P, Cowhey I, Etzioni O, Khot T, Sabharwal A, Schoenick C, et al. Think you have solved question answering? Try ARC, the AI2 reasoning challenge. arXiv preprint arXiv: 1803.05457, 2018.
[52]	Kay W, Carreira J, Simonyan K, Zhang B, Hillier C, Vijayanarasimhan S, et al. The kinetics human action video dataset. arXiv preprint arXiv: 1705.06950, 2017.
[53]	Lin T Y, Maire M, Belongie S, Hays J, Perona P, Ramanan D, et al. Microsoft COCO: Common objects in context. In: Proceedings of the 13th European Conference on Computer Vision (ECCV). Zurich, Switzerland: Springer, 2014. 740−755
[54]	Zhou B L, Zhao H, Puig X, Fidler S, Barriuso A, Torralba A. Scene parsing through ADE20K dataset. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE, 2017. 5122−5130
[55]	Everingham M, van Gool L, Williams C K I, Winn J, Zisserman A. The PASCAL visual object classes (VOC) challenge. International Journal of Computer Vision, 2010, 88(2): 303−338 doi: 10.1007/s11263-009-0275-4
[56]	Houlsby N, Giurgiu A, Jastrzebski S, Morrone B, de Laroussilhe Q, Gesmundo A, et al. Parameter-efficient transfer learning for NLP. In: Proceedings of the 36th International Conference on Machine Learning (ICML). Long Beach, USA: PMLR, 2019. 2790−2799
[57]	Pfeiffer J, Kamath A, Rücklé A, Cho K, Gurevych I. AdapterFusion: Non-destructive task composition for transfer learning. In: Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. Virtual Event: Association for Computational Linguistics, 2021. 487−503
[58]	Lin Z J, Madotto A, Fung P. Exploring versatile generative language model via parameter-efficient transfer learning. In: Proceedings of the Findings of the Association for Computational Linguistics. Virtual Event: Association for Computational Linguistics, 2020. 441−459
[59]	He J X, Zhou C T, Ma X Z, Berg-Kirkpatrick T, Neubig G. Towards a unified view of parameter-efficient transfer learning. In: Proceedings of the 10th International Conference on Learning Representations (ICLR). Virtual Event: OpenReview.net, 2022. 1−15
[60]	Zhu Y M, Feng J T, Zhao C Q, Wang M X, Li L. Counter-interference adapter for multilingual machine translation. In: Proceedings of the Findings of the Association for Computational Linguistics. Punta Cana, Dominican Republic: Association for Computational Linguistics, 2021. 2812−2823
[61]	Lei T, Bai J W, Brahma S, Ainslie J, Lee K, Zhou Y Q, et al. Conditional adapters: Parameter-efficient transfer learning with fast inference. In: Proceedings of the 37th International Conference on Neural Information Processing Systems. New Orleans, USA: Curran Associates Inc., 2023. Article No. 357
[62]	Chen Y Y, Fu Q, Fan G, Du L, Lou J G, Han S, et al. Hadamard adapter: An extreme parameter-efficient adapter tuning method for pre-trained language models. In: Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. Birmingham, UK: ACM, 2023. 276−285
[63]	He S, Fan R Z, Ding L, Shen L, Zhou T Y, Tao D C. MerA: Merging pretrained adapters for few-shot learning. arXiv preprint arXiv: 2308.15982, 2023.
[64]	Mahabadi R K, Ruder S, Dehghani M, Henderson J. Parameter-efficient multi-task fine-tuning for Transformers via shared hypernetworks. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Virtual Event: Association for Computational Linguistics, 2021. 565−576
[65]	Inoue N, Otake S, Hirose T, Ohi M, Kawakami R. ELP-adapters: Parameter efficient adapter tuning for various speech processing tasks. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2024, 32: 3867−3880 doi: 10.1109/TASLP.2024.3434445
[66]	Petrov A, Torr P H S, Bibi A. When do prompting and prefix-tuning work? A theory of capabilities and limitations. In: Proceedings of the 12th International Conference on Learning Representations (ICLR). Vienna, Austria: OpenReview.net, 2024. 1−24
[67]	Li X L, Liang P. Prefix-tuning: Optimizing continuous prompts for generation. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Virtual Event: Association for Computational Linguistics, 2021. 4582−4597
[68]	Lester B, Al-Rfou R, Constant N. The power of scale for parameter-efficient prompt tuning. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Punta Cana, Dominican Republic: Association for Computational Linguistics, 2021. 3045−3059
[69]	Liu X, Zheng Y N, Du Z X, Ding M, Qian Y J, Yang Z L, et al. GPT understands, too. AI Open, 2024, 5: 208−215 doi: 10.1016/j.aiopen.2023.08.012
[70]	Liu X, Ji K X, Fu Y C, Tam W L, Du Z X, Yang Z L, et al. P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks. arXiv preprint arXiv: 2110.07602, 2021.
[71]	Choi J Y, Kim J, Park J H, Mok W L, Lee S K. SMoP: Towards efficient and effective prompt tuning with sparse mixture-of-prompts. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Singapore: Association for Computational Linguistics, 2023. 14306−14316
[72]	Zhang Z R, Tan C Q, Xu H Y, Wang C Y, Huang J, Huang S F. Towards adaptive prefix tuning for parameter-efficient language model fine-tuning. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Toronto, Canada: Association for Computational Linguistics, 2023. 1239−1248
[73]	Wang Q F, Mao Y N, Wang J G, Yu H C, Nie S L, Wang S N, et al. APrompt: Attention prompt tuning for efficient adaptation of pre-trained language models. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Singapore: Association for Computational Linguistics, 2023. 9147−9160
[74]	Wu Z F, Wang S N, Gu J T, Hou R, Dong Y X, Vydiswaran V G V, et al. IDPG: An instance-dependent prompt generation method. In: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Seattle, USA: Association for Computational Linguistics, 2022. 5507−5521
[75]	Zhu W, Tan M. SPT: Learning to selectively insert prompts for better prompt tuning. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Singapore: Association for Computational Linguistics, 2023. 11862−11878
[76]	Liu X Y, Sun T X, Huang X J, Qiu X P. Late prompt tuning: A late prompt could be better than many prompts. In: Proceedings of the Findings of the Association for Computational Linguistics. Abu Dhabi, United Arab Emirates: Association for Computational Linguistics, 2022. 1325−1338
[77]	Ma F, Zhang C, Ren L, Wang J G, Wang Q F, Wu W, et al. XPrompt: Exploring the extreme of prompt tuning. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Abu Dhabi, United Arab Emirates: Association for Computational Linguistics, 2022. 11033−11047
[78]	Shi Z X, Lipani A. DePT: Decomposed prompt tuning for parameter-efficient fine-tuning. In: Proceedings of the 12th International Conference on Learning Representations (ICLR). Vienna, Austria: OpenReview.net, 2024. 1−21
[79]	Wu J D, Yu T, Wang R, Song Z, Zhang R Y, Zhao H D, et al. InfoPrompt: Information-theoretic soft prompt tuning for natural language understanding. In: Proceedings of the 37th International Conference on Neural Information Processing Systems. New Orleans, USA: Curran Associates Inc., 2023. Article No. 2669
[80]	Chen L C. PTP: Boosting Stability and Performance of Prompt Tuning With Perturbation-based Regularizer [Master thesis], University of Pittsburgh, USA, 2023.
[81]	Zeng R J, Han C, Wang Q F, Wu C S, Geng T, Huang L F, et al. Visual Fourier prompt tuning. In: Proceedings of the 38th International Conference on Neural Information Processing Systems. Vancouver, Canada: Curran Associates Inc., 2024. Article No. 180
[82]	He A L, Wu Y L, Wang Z H, Li T, Fu H Z. DVPT: Dynamic visual prompt tuning of large pre-trained models for medical image analysis. Neural Networks, 2025, 185: Article No. 107168 doi: 10.1016/j.neunet.2025.107168
[83]	Xiao Z H, Yan S L, Hong J, Cai J Y, Jiang X L, Hu Y, et al. DynaPrompt: Dynamic test-time prompt tuning. In: Proceedings of the 13th International Conference on Learning Representations (ICLR). Singapore: OpenReview.net, 2025. 1−17
[84]	Li Z P, Lin M H, Wang J, Wang S H. Fairness-aware prompt tuning for graph neural networks. In: Proceedings of the ACM on Web Conference. Sydney, Australia: ACM, 2025. 3586−3597
[85]	Liu H K, Tam D, Muqeeth M, Mohta J, Huang T H, Bansal M, et al. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. In: Proceedings of the 36th International Conference on Neural Information Processing Systems. New Orleans, USA: Curran Associates Inc., 2022. Article No. 142
[86]	Lian D Z, Zhou D Q, Feng J S, Wang X C. Scaling & shifting your features: A new baseline for efficient model tuning. In: Proceedings of the 36th International Conference on Neural Information Processing Systems. New Orleans, USA: Curran Associates Inc., 2022. Article No. 9
[87]	Yang X C, Huang J Y, Zhou W X, Chen M H. Parameter-efficient tuning with special token adaptation. In: Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics. Dubrovnik, Croatia: Association for Computational Linguistics, 2023. 865−872
[88]	Lu X M, Brahman F, West P, Jung J, Chandu K, Ravichander A, et al. Inference-time policy adapters (IPA): Tailoring extreme-scale LMs without fine-tuning. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Singapore: Association for Computational Linguistics, 2023. 6863−6883
[89]	Sung Y L, Cho J, Bansal M. LST: Ladder side-tuning for parameter and memory efficient transfer learning. In: Proceedings of the 36th International Conference on Neural Information Processing Systems. New Orleans, USA: Curran Associates Inc., 2022. Article No. 944
[90]	Cao J, Prakash C S, Hamza W. Attention Fusion: A light yet efficient late fusion mechanism for task adaptation in NLU. In: Proceedings of the Findings of the Association for Computational Linguistics. Seattle, USA: Association for Computational Linguistics, 2022. 857−866
[91]	Savadikar C, Song X, Wu T F. WeGeFT: Weight-generative fine-tuning for multi-faceted efficient adaptation of large models. In: Proceedings of the 42nd International Conference on Machine Learning (ICML). Vancouver, Canada: PMLR, 2025. 53019−53041
[92]	Sang M F, Hansen J H L. UniPET-SPK: A unified framework for parameter-efficient tuning of pre-trained speech models for robust speaker verification. IEEE Transactions on Audio, Speech and Language Processing, 2025, 33: 1402−1414 doi: 10.1109/TASLPRO.2025.3539005
[93]	Sang M F, Hansen J H L. Efficient adapter tuning of pre-trained speech models for automatic speaker verification. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Seoul, South Korea: IEEE, 2024. 12131−12135
[94]	Liu Y P, Yang X K, Zhang J Y, Xi Y L, Qu D. TAML-adapter: Enhancing adapter tuning through task-agnostic meta-learning for low-resource automatic speech recognition. IEEE Signal Processing Letters, 2025, 32: 636−640 doi: 10.1109/LSP.2024.3525399
[95]	Zhao M J, Lin T, Mi F, Jaggi M, Schütze H. Masking as an efficient alternative to finetuning for pretrained language models. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). Virtual Event: Association for Computational Linguistics, 2020. 2226−2241
[96]	Ben-Zaken E, Ravfogel S, Goldberg Y. BitFit: Simple parameter-efficient fine-tuning for Transformer-based masked language-models. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Dublin, Ireland: Association for Computational Linguistics, 2022. 1−9
[97]	Lawton N, Kumar A, Thattai G, Galstyan A, Steeg G V. Neural architecture search for parameter-efficient fine-tuning of large pre-trained language models. In: Proceedings of the Annual Meeting of the Association for Computational Linguistics. Toronto, Canada: Association for Computational Linguistics, 2023. 8506−8515
[98]	Liao B H, Meng Y, Monz C. Parameter-efficient fine-tuning without introducing new latency. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Toronto, Canada: Association for Computational Linguistics, 2023. 4242−4260
[99]	Sung Y L, Nair V, Raffel C. Training neural networks with fixed sparse masks. In: Proceedings of the 35th International Conference on Neural Information Processing Systems. Virtual Event: Curran Associates Inc., 2021. Article No. 1852
[100]	Das S S S, Zhang R H, Shi P, Yin W P, Zhang R. Unified low-resource sequence labeling by sample-aware dynamic sparse finetuning. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Singapore: Association for Computational Linguistics, 2023. 6998−7010
[101]	Ansell A, Ponti E M, Korhonen A, Vulić I. Composable sparse fine-tuning for cross-lingual transfer. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Dublin, Ireland: Association for Computational Linguistics, 2022. 1778−1796
[102]	Frankle J, Carbin M. The lottery ticket hypothesis: Finding sparse, trainable neural networks. In: Proceedings of the 7th International Conference on Learning Representations (ICLR). New Orleans, USA: OpenReview.net, 2019. 1−42
[103]	Malach E, Yehudai G, Shalev-Shwartz S, Shamir O. Proving the lottery ticket hypothesis: Pruning is all you need. In: Proceedings of the 37th International Conference on Machine Learning (ICML). Virtual Event: PMLR, 2020. 6682−6691
[104]	Xu R X, Luo F L, Zhang Z Y, Tan C Q, Chang B B, Huang S F, et al. Raise a child in large language model: Towards effective and generalizable fine-tuning. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Punta Cana, Dominican Republic: Association for Computational Linguistics, 2021. 9514−9528
[105]	Chekalina V A, Rudenko A, Mezentsev G, Mikhalev A, Panchenko A, Oseledets I. SparseGrad: A selective method for efficient fine-tuning of MLP layers. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Miami, USA: Association for Computational Linguistics, 2024. 14929−14939
[106]	Gheini M, Ren X, May J. Cross-attention is all you need: Adapting pretrained Transformers for machine translation. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Punta Cana, Dominican Republic: Association for Computational Linguistics, 2021. 1754−1765
[107]	He H Y, Cai J F, Zhang J, Tao D C, Zhuang B H. Sensitivity-aware visual parameter-efficient fine-tuning. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Paris, France: IEEE, 2023. 11791−11801
[108]	Kowsher M, Esmaeilbeig T, Yu C N, Chen C, Soltanalian M, Yousefi N. RoCoFT: Efficient finetuning of large language models with row-column updates. In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Vienna, Austria: Association for Computational Linguistics, 2025. 26659−26678
[109]	Aghajanyan A, Gupta S, Zettlemoyer L. Intrinsic dimensionality explains the effectiveness of language model fine-tuning. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Virtual Event: Association for Computational Linguistics, 2021. 7319−7328
[110]	Malladi S, Wettig A, Yu D L, Chen D Q, Arora S. A kernel-based view of language model fine-tuning. In: Proceedings of the 40th International Conference on Machine Learning (ICML). Honolulu, USA: PMLR, 2023. 23610−23641
[111]	Zeng Y C, Lee K. The expressive power of low-rank adaptation. In: Proceedings of the 12th International Conference on Learning Representations (ICLR). Vienna, Austria: OpenReview.net, 2024. 1−46
[112]	Jang U, Lee J D, Ryu E K. LoRA training in the NTK regime has no spurious local minima. In: Proceedings of the 41st International Conference on Machine Learning (ICML). Vienna, Austria: PMLR, 2024. 21306−21328
[113]	Zhu J C, Greenewald K H, Nadjahi K, Borde H S D O, Gabrielsson R B, Choshen L, et al. Asymmetry in low-rank adapters of foundation models. In: Proceedings of the 41st International Conference on Machine Learning (ICML). Vienna, Austria: PMLR, 2024. 62369−62385
[114]	Edalati A, Tahaei M, Kobyzev I, Nia V P, Clark J J, Rezagholizadeh M. KronA: Parameter-efficient tuning with Kronecker adapter. Enhancing LLM Performance: Efficacy, Fine-Tuning, and Inference Techniques. Cham: Springer, 2025. 49−65
[115]	Yang Y F, Zhou J J, Wong N, Zhang Z. LoRETTA: Low-rank economic tensor-train adaptation for ultra-low-parameter fine-tuning of large language models. In: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). Mexico City, Mexico: Association for Computational Linguistics, 2024. 3161−3176
[116]	Oseledets I V. Tensor-train decomposition. SIAM Journal on Scientific Computing, 2011, 33(5): 2295−2317 doi: 10.1137/090752286
[117]	Bershatsky D, Cherniuk D, Daulbaev T, Mikhalev A, Oseledets I. LoTR: Low tensor rank weight adaptation. arXiv preprint arXiv: 2402.01376, 2024.
[118]	Chen X Y, Liu J, Wang Y, Wang P P, Brand M, Wang G H, et al. SuperLoRA: Parameter-efficient unified adaptation of multi-layer attention modules. arXiv preprint arXiv: 2403.11887, 2024.
[119]	Anjum A, Eren M E, Boureima I, Alexandrov B, Bhattarai M. Tensor train low-rank approximation (TT-LoRA): Democratizing AI with accelerated LLMs. In: Proceedings of the International Conference on Machine Learning and Applications (ICMLA). Miami, USA: IEEE, 2024. 583−590
[120]	Hounie I, Kanatsoulis C, Tandon A, Ribeiro A. LoRTA: Low rank tensor adaptation of large language models. In: Proceedings of the 13th International Conference on Learning Representations (ICLR). Singapore: OpenReview.net, 2025. 1−20
[121]	Chang Y P, Chang Y, Wu Y. Bias-aware low-rank adaptation: Mitigating catastrophic inheritance of large language models. arXiv preprint arXiv: 2408.04556, 2024.
[122]	Biderman D, Portes J P, Ortiz J J G, Paul M, Greengard P, Jennings C, et al. LoRA learns less and forgets less. arXiv preprint arXiv: 2405.09673, 2024.
[123]	Valipour M, Rezagholizadeh M, Kobyzev I, Ghodsi A. DyLoRA: Parameter-efficient tuning of pre-trained models using dynamic search-free low-rank adaptation. In: Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics. Dubrovnik, Croatia: Association for Computational Linguistics, 2023. 3274−3287
[124]	Zhang Q R, Chen M S, Bukharin A, Karampatziakis N, He P C, Cheng Y, et al. AdaLoRA: Adaptive budget allocation for parameter-efficient fine-tuning. arXiv preprint arXiv: 2303.10512, 2023.
[125]	Zhang F Y, Li L Z, Chen J H, Jiang Z Q, Wang B W, Qian Y M. IncreLoRA: Incremental parameter allocation method for parameter-efficient fine-tuning. arXiv preprint arXiv: 2308.12043, 2023.
[126]	Ding N, Lv X T, Wang Q S, Chen Y L, Zhou B W, Liu Z Y, et al. Sparse low-rank adaptation of pre-trained language models. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Singapore: Association for Computational Linguistics, 2023. 4133−4145
[127]	Mao Y L, Huang K Y, Guan C H, Bao G L, Mo F R, Xu J N. DoRA: Enhancing parameter-efficient fine-tuning with dynamic rank distribution. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Bangkok, Thailand: Association for Computational Linguistics, 2024. 11662−11675
[128]	Lialin V, Muckatira S, Shivagunde N, Rumshisky A. ReLoRA: High-rank training through low-rank updates. In: Proceedings of the 12th International Conference on Learning Representations (ICLR). Vienna, Austria: OpenReview.net, 2024. 1−17
[129]	Xia W H, Qin C W, Hazan E. Chain of LoRA: Efficient fine-tuning of language models via residual learning. arXiv preprint arXiv: 2401.04151, 2024.
[130]	Ren P J, Shi C S, Wu S G, Zhang M Q, Ren Z C, de Rijke M, et al. MELoRA: Mini-ensemble low-rank adapters for parameter-efficient fine-tuning. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Bangkok, Thailand: Association for Computational Linguistics, 2024. 3052−3064
[131]	Hayou S, Ghosh N, Yu B. The impact of initialization on LoRA finetuning dynamics. In: Proceedings of the 38th International Conference on Neural Information Processing Systems. Vancouver, Canada: Curran Associates Inc., 2024. Article No. 3715
[132]	Meng F X, Wang Z H, Zhang M H. PiSSA: Principal singular values and singular vectors adaptation of large language models. In: Proceedings of the 38th International Conference on Neural Information Processing Systems. Vancouver, Canada: Curran Associates Inc., 2024. Article No. 3846
[133]	Wang H Q, Li Y X, Wang S, Chen G H, Chen Y. MiLoRA: Harnessing minor singular components for parameter-efficient LLM finetuning. In: Proceedings of the Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). Albuquerque, USA: Association for Computational Linguistics, 2025. 4823−4836
[134]	Hayou S, Ghosh N, Yu B. LoRA+: Efficient low rank adaptation of large models. In: Proceedings of the 41st International Conference on Machine Learning (ICML). Vienna, Austria: PMLR, 2024. 17783−17806
[135]	Shi S H, Huang S H, Song M H, Li Z J, Zhang Z H, Huang H Z, et al. ResLoRA: Identity residual mapping in low-rank adaption. In: Proceedings of the Findings of the Association for Computational Linguistics. Bangkok, Thailand: Association for Computational Linguistics, 2024. 8870−8884
[136]	Wen Z H, Zhang J, Fang Y. SIBO: A simple booster for parameter-efficient fine-tuning. In: Proceedings of the Findings of the Association for Computational Linguistics. Bangkok, Thailand: Association for Computational Linguistics, 2024. 1241−1257
[137]	Liu S Y, Wang C Y, Yin H X, Molchanov P, Wang Y C F, Cheng K T, et al. DoRA: Weight-decomposed low-rank adaptation. In: Proceedings of the 41st International Conference on Machine Learning (ICML). Vienna, Austria: PMLR, 2024. 32100−32121
[138]	Wang H Y, Liu T C, Li R R, Cheng M X, Zhao T, Gao J. RoseLoRA: Row and column-wise sparse low-rank adaptation of pre-trained language model for knowledge editing and fine-tuning. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Miami, USA: Association for Computational Linguistics, 2024. 996−1008
[139]	Wang Z B, Liang J, He R, Wang Z L, Tan T N. LoRA-pro: Are low-rank adapters properly optimized? In: Proceedings of the 13th International Conference on Learning Representations (ICLR). Singapore: OpenReview.net, 2025. 1−22
[140]	Deng J X, Zhu Q C, Pang J B, Yang L L, Fu Z Q, Zhang B C. EFlat-LoRA: Efficiently seeking flat minima for better generalization in fine-tuning large language models and beyond. arXiv preprint arXiv: 2508.00522, 2025.
[141]	Yang A X, Robeyns M, Wang X, Aitchison L. Bayesian low-rank adaptation for large language models. In: Proceedings of the 12th International Conference on Learning Representations (ICLR). Vienna, Austria: OpenReview.net, 2024. 1−31
[142]	Qi Z T, Tan X Y, Shi S J, Qu C, Xu Y H, Qi Y. PILLOW: Enhancing efficient instruction fine-tuning via prompt matching. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing: Industry Track. Singapore: Association for Computational Linguistics, 2023. 471−482
[143]	Zhang L H, Wu J L, Zhou D Y, Xu G Q. STAR: Constraint LoRA with dynamic active learning for data-efficient fine-tuning of large language models. In: Proceedings of the Findings of the Association for Computational Linguistics. Bangkok, Thailand: Association for Computational Linguistics, 2024. 3519−3532
[144]	Lin Y, Ma X Y, Chu X, Jin Y J, Yang Z B, Wang Y S. LoRA dropout as a sparsity regularizer for overfitting reduction. Knowledge-Based Systems, 2025, 329: Article No. 114241 doi: 10.1016/j.knosys.2025.114241
[145]	Zheng J J, Lu W L, Dong Y M, Ji C J, Cao Y K, Lin Z C. AdaMSS: Adaptive multi-subspace approach for parameter-efficient fine-tuning. In: Proceedings of the 39th Conference on Neural Information Processing Systems. San Diego, USA: Curran Associates Inc., 2025.
[146]	Wang X, Aitchison L, Rudolph M. LoRA ensembles for large language model fine-tuning. arXiv preprint arXiv: 2310.00035, 2023.
[147]	Zhao Z Y, Gan L L, Wang G Y, Zhou W C S, Yang H X, Kuang K, et al. LoraRetriever: Input-aware LoRA retrieval and composition for mixed tasks in the wild. In: Proceedings of the Findings of the Association for Computational Linguistics. Bangkok, Thailand: Association for Computational Linguistics, 2024. 4447−4462
[148]	Sun Y H, Li M K, Cao Y X, Wang K, Wang W X, Zeng X Y, et al. To be or not to be? An exploration of continuously controllable prompt engineering. arXiv preprint arXiv: 2311.09773, 2023.
[149]	Zhang J H, Chen S Q, Liu J T, He J X. Composing parameter-efficient modules with arithmetic operations. In: Proceedings of the 37th International Conference on Neural Information Processing Systems. New Orleans, USA: Curran Associates Inc., 2023. Article No. 552
[150]	Huang C S, Liu Q, Lin B Y, Pang T Y, Du C, Lin M. LoraHub: Efficient cross-task generalization via dynamic LoRA composition. arXiv preprint arXiv: 2307.13269, 2023.
[151]	Liu J L, Moreau A, Preuss M, Rapin J, Roziere B, Teytaud F, et al. Versatile black-box optimization. In: Proceedings of the Genetic and Evolutionary Computation Conference. Cancún, Mexico: ACM, 2020. 620−628
[152]	Tang A K, Shen L, Luo Y, Zhan Y B, Hu H, Du B, et al. Parameter-efficient multi-task model fusion with partial linearization. In: Proceedings of the 12th International Conference on Learning Representations (ICLR). Vienna, Austria: OpenReview.net, 2024. 1−31
[153]	Shen Y, Xu Z Y, Wang Q F, Cheng Y, Yin W P, Huang L F. Multimodal instruction tuning with conditional mixture of LoRA. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Bangkok, Thailand: Association for Computational Linguistics, 2024. 637−648
[154]	Buehler E L, Buehler M J. X-LoRA: Mixture of low-rank adapter experts, a flexible framework for large language models with applications in protein mechanics and molecular design. APL Machine Learning, 2024, 2(2): Article No. 026119 doi: 10.1063/5.0203126
[155]	Feng W F, Hao C Z, Zhang Y W, Han Y, Wang H. Mixture-of-LoRAs: An efficient multitask tuning method for large language models. In: Proceedings of the Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING). Torino, Italia: Association for Computational Linguistics, 2024. 11371−11380
[156]	Wang Y M, Lin Y, Zeng X D, Zhang G N. MultiLoRA: Democratizing LoRA for better multi-task learning. arXiv preprint arXiv: 2311.11501, 2023.
[157]	Yang Y Q, Jiang P T, Hou Q B, Zhang H, Chen J W, Li B. Multi-task dense prediction via mixture of low-rank experts. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE, 2024. 27927−27937
[158]	Agiza A, Neseem M, Reda S. MTLoRA: A low-rank adaptation approach for efficient multi-task learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE, 2024. 16196−16205
[159]	Gao C Y, Chen K Z, Rao J M, Sun B C, Liu R B, Peng D Y, et al. Higher layers need more LoRA experts. arXiv preprint arXiv: 2402.08562, 2024.
[160]	Chen S X, Jie Z Q, Ma L. LLaVA-MoLE: Sparse mixture of LoRA experts for mitigating data conflicts in instruction finetuning MLLMs. arXiv preprint arXiv: 2401.16160, 2024.
[161]	Zhu Y, Wichers N, Lin C C, Wang X Y, Chen T L, Shu L, et al. SiRA: Sparse mixture of low rank adaptation. arXiv preprint arXiv: 2311.09179, 2023.
[162]	Qing P J, Gao C Y, Zhou Y F, Diao X J, Yang Y Q, Vosoughi S. AlphaLoRA: Assigning LoRA experts based on layer training quality. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Miami, USA: Association for Computational Linguistics, 2024. 20511−20523
[163]	Wang J J, Yang G J, Chen W T, Yi H H, Wu X H, Lao Q C. MLAE: Masked LoRA experts for parameter-efficient fine-tuning. arXiv preprint arXiv: 2405.18897, 2024.
[164]	Fan C H, Lu Z Y, Liu S C, Gu C F, Qu X Y, Wei W, et al. Make LoRA great again: Boosting LoRA with adaptive singular values and mixture-of-experts optimization alignment. In: Proceedings of the 42nd International Conference on Machine Learning. Vancouver, Canada: PMLR, 2025. 15804−15832
[165]	Zhao Z Y, Zhou Y X, Yu X, Zhang Z, Zhu D D, Shen T, et al. Each rank could be an expert: Single-ranked mixture of experts LoRA for multi-task learning. In: Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Jeju Island, South Korea: ACM, 2026. 1998−2009
[166]	Liao M Q, Chen W, Shen J F, Guo S N, Wan H Y. HMoRA: Making LLMs more effective with hierarchical mixture of LoRA experts. In: Proceedings of the 13th International Conference on Learning Representations (ICLR). Singapore: OpenReview.net, 2025. 1−22
[167]	Mu B S, Wei K, Guo P C, Xie L. Mixture of LoRA experts with multi-modal and multi-granularity LLM generative error correction for accented speech recognition. IEEE Transactions on Audio, Speech and Language Processing, 2025, 33: 2973−2985 doi: 10.1109/TASLPRO.2025.3589856
[168]	Prabhakar A, Li Y Z, Narasimhan K, Kakade S, Malach E, Jelassi S. LoRA soups: Merging LoRAs for practical skill composition tasks. In: Proceedings of the 31st International Conference on Computational Linguistics: Industry Track. Abu Dhabi, United Arab Emirates: Association for Computational Linguistics, 2025. 644−655
[169]	Liang J, Huang W K, Wan G C, Yang Q, Ye M. LoRASculpt: Sculpting LoRA for harmonizing general and specialized knowledge in multimodal large language models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville, USA: IEEE, 2025. 26170−26180
[170]	Yang Y M, Muhtar D, Shen Y L, Zhan Y F, Liu J F, Wang Y J, et al. MTL-LoRA: Low-rank adaptation for multi-task learning. In: Proceedings of the 39th AAAI Conference on Artificial Intelligence. Philadelphia, USA: AAAI Press, 2025. 22010−22018
[171]	Zhang L T, Zhang L, Shi S H, Chu X W, Li B. LoRA-FA: Memory-efficient low-rank adaptation for large language models fine-tuning. In: Proceedings of the 12th International Conference on Learning Representations (ICLR). Vienna, Austria: OpenReview.net, 2024. 1−19
[172]	Wu Y C, Xiang Y F, Huo S N, Gong Y L, Liang P H. LoRA-SP: Streamlined partial parameter adaptation for resource efficient fine-tuning of large language models. In: Proceedings of the 3rd International Conference on Algorithms, Microchips, and Network Applications. Jinan, China: SPIE, 2024. Article No. 131711Z
[173]	Liu Z Y, Kundu S, Li A N, Wan J R, Jiang L H, Beerel P A. AFLoRA: Adaptive freezing of low rank adaptation in parameter efficient fine-tuning of large models. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Bangkok, Thailand: Association for Computational Linguistics, 2024. 161−167
[174]	Woo S, Park B, Kim B, Jo M, Kwon S J, Jeon D, et al. DropBP: Accelerating fine-tuning of large language models by dropping backward propagation. In: Proceedings of the 38th International Conference on Neural Information Processing Systems. Vancouver, Canada: Curran Associates Inc., 2024. Article No. 637
[175]	Balazy K, Banaei M, Aberer K, Tabor J. LoRA-XS: Low-rank adaptation with extremely small number of parameters. In: Proceedings of the 28th European Conference on Artificial Intelligence. Bologna, Italy: IOS Press, 2025. 3194−3201
[176]	Zhang M Y, Chen H, Shen C H, Yang Z, Ou L L, Yu X Y, et al. LoRAPrune: Structured pruning meets low-rank parameter-efficient fine-tuning. In: Proceedings of the Findings of the Association for Computational Linguistics. Bangkok, Thailand: Association for Computational Linguistics, 2024. 3013−3026
[177]	Zhou H Y, Lu X Y, Xu W, Zhu C H, Zhao T J, Yang M Y. LoRA-drop: Efficient LoRA parameter pruning based on output evaluation. In: Proceedings of the 31st International Conference on Computational Linguistics. Abu Dhabi, United Arab Emirates: Association for Computational Linguistics, 2025. 5530−5543
[178]	Ao S, Dong Y, Hu J W, Ramchurn S D. Safe pruning LoRA: Robust distance-guided pruning for safety alignment in adaptation of LLMs. Transactions of the Association for Computational Linguistics, 2025, 13: 1474−1487 doi: 10.1162/TACL.a.44
[179]	Miyano R, Arase Y. Adaptive LoRA merge with parameter pruning for low-resource generation. In: Proceedings of the Findings of the Association for Computational Linguistics. Vienna, Austria: Association for Computational Linguistics, 2025. 19353−19366
[180]	Liu Y J, Yang H R, Chen Y X, Zhang R Y, Wang M, Du Y, et al. PAT: Pruning-aware tuning for large language models. In: Proceedings of the 39th AAAI Conference on Artificial Intelligence. Philadelphia, USA: AAAI Press, 2025. 24686−24695
[181]	Zhang J, Wang J, Li H, Shou L D, Chen K, You Y, et al. Train small, infer large: Memory-efficient LoRA training for large language models. In: Proceedings of the 13th International Conference on Learning Representations (ICLR). Singapore: OpenReview.net, 2025. 1−30
[182]	Yu X, Xie C, Zhao Z Y, Fan T T, Xue L Z, Zhang Z. PrunedLoRA: Robust gradient-based structured pruning for low-rank adaptation in fine-tuning. In: Proceedings of the 14th International Conference on Learning Representations (ICLR). Rio de Janeiro, Brazil: OpenReview.net, 2026. 1−29
[183]	Kopiczko D J, Blankevoort T, Asano Y M. VeRA: Vector-based random matrix adaptation. In: Proceedings of the 12th International Conference on Learning Representations (ICLR). Vienna, Austria: OpenReview.net, 2024. 1−21
[184]	Li Y, Han S B, Ji S H. VB-LoRA: Extreme parameter efficient fine-tuning with vector banks. In: Proceedings of the 38th International Conference on Neural Information Processing Systems. Vancouver, Canada: Curran Associates Inc., 2024. Article No. 533
[185]	Gao Z Q, Wang Q C, Chen A C, Liu Z J, Wu B Z, Chen L, et al. Parameter-efficient fine-tuning with discrete Fourier transform. In: Proceedings of the 41st International Conference on Machine Learning (ICML). Vienna, Austria: PMLR, 2024. 14884−14901
[186]	Zhou Y H, Li R F, Zhou C H, Yang F, Pan A M. BSLoRA: Enhancing the parameter efficiency of LoRA with intra-layer and inter-layer sharing. In: Proceedings of the 42nd International Conference on Machine Learning (ICML). Vancouver, Canada: PMLR, 2025. 78883−78902
[187]	Zhao J M, Zhang X Y, Li J R, Niu J C, Hu Y L, Min E X, et al. Tiny budgets, big gains: Parameter placement strategy in parameter super-efficient fine-tuning. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Suzhou, China: Association for Computational Linguistics, 2025. 6315−6333
[188]	Dettmers T, Pagnoni A, Holtzman A, Zettlemoyer L. QLORA: Efficient finetuning of quantized LLMs. In: Proceedings of the 37th International Conference on Neural Information Processing Systems. New Orleans, USA: Curran Associates Inc., 2023. Article No. 441
[189]	Xu Y H, Xie L X, Gu X T, Chen X, Chang H, Zhang H H, et al. QA-LoRA: Quantization-aware low-rank adaptation of large language models. In: Proceedings of the 12th International Conference on Learning Representations (ICLR). Vienna, Austria: OpenReview.net, 2024. 1−18
[190]	Li Y X, Yu Y F, Liang C, He P C, Karampatziakis N, Chen W Z, et al. LoftQ: LoRA-fine-tuning-aware quantization for large language models. In: Proceedings of the 12th International Conference on Learning Representations (ICLR). Vienna, Austria: OpenReview.net, 2024. 1−16
[191]	Deng Y X, Zhang A Z, Gurses S, Wang N G, Yang Z, Yin P H. CLoQ: Enhancing fine-tuning of quantized LLMs via calibrated LoRA initialization. arXiv preprint arXiv: 2501.18475, 2025.
[192]	Lee G, Lee J, Hong S, Kim M, Ahn E, Chang D S, et al. RILQ: Rank-insensitive LoRA-based quantization error compensation for boosting 2-bit large language model accuracy. In: Proceedings of the 39th AAAI Conference on Artificial Intelligence. Philadelphia, USA: AAAI Press, 2025. 18091−18100
[193]	Ke W J, Li Z, Li D, Tian L, Barsoum E. DL-QAT: Weight-decomposed low-rank quantization-aware training for large language models. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing: Industry Track. Miami, USA: Association for Computational Linguistics, 2024. 113−119
[194]	Jeon H, Kim Y, Kim J J. L4Q: Parameter efficient quantization-aware fine-tuning on large language models. In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Vienna, Austria: Association for Computational Linguistics, 2025. 2002−2024
[195]	Ye Z M, Li D C, Tian J Q, Lan T F, Zuo J, Duan L, et al. ASPEN: High-throughput LoRA fine-tuning of large language models with a single GPU. arXiv preprint arXiv: 2312.02515, 2023.
[196]	Chen L Q, Ye Z H, Wu Y J, Zhuo D Y, Ceze L, Krishnamurthy A. Punica: Multi-tenant LoRA serving. In: Proceedings of the 7th Annual Conference on Machine Learning and Systems. Santa Clara, USA: mlsys.org, 2024. 1−13
[197]	Sheng Y, Cao S Y, Li D C, Hooper C, Lee N, Yang S, et al. S-LoRA: Serving thousands of concurrent LoRA adapters. arXiv preprint arXiv: 2311.03285, 2023.
[198]	Yan M H, Wang Z, Jia Z, Venkataraman S, Wang Y D. PLoRA: Efficient LoRA hyperparameter tuning for large models. arXiv preprint arXiv: 2508.02932, 2025.
[199]	Mao Y N, Mathias L, Hou R, Almahairi A, Ma H, Han J W, et al. UniPELT: A unified framework for parameter-efficient language model tuning. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Dublin, Ireland: Association for Computational Linguistics, 2022. 6253−6264
[200]	Hu S D, Zhang Z, Ding N, Wang Y D, Wang Y S, Liu Z Y, et al. Sparse structure search for delta tuning. In: Proceedings of the 36th International Conference on Neural Information Processing Systems. New Orleans, USA: Curran Associates Inc., 2022. Article No. 716
[201]	Chen J A, Zhang A, Shi X J, Li M, Smola A, Yang D Y. Parameter-efficient fine-tuning design spaces. In: Proceedings of the 11th International Conference on Learning Representations (ICLR). Kigali, Rwanda: OpenReview.net, 2023. 1−18
[202]	Zeng G T, Zhang P Y, Lu W. One network, many masks: Towards more parameter-efficient transfer learning. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Toronto, Canada: Association for Computational Linguistics, 2023. 7564−7580
[203]	Hu Z Q, Wang L, Lan Y H, Xu W Y, Lim E P, Bing L D, et al. LLM-adapters: An adapter family for parameter-efficient fine-tuning of large language models. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Singapore: Association for Computational Linguistics, 2023. 5254−5276
[204]	Zhang Y H, Zhou K Y, Liu Z W. Neural prompt search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025, 47(7): 5268−5280 doi: 10.1109/TPAMI.2024.3435939
[205]	Zhou H, Wan X C, Vulić I, Korhonen A. AutoPEFT: Automatic configuration search for parameter-efficient fine-tuning. Transactions of the Association for Computational Linguistics, 2024, 12: 525−542 doi: 10.1162/tacl_a_00662
[206]	Chai Y D, Liu Y, Zhou Y H, Xie J H, Zeng D D. A Bayesian hybrid parameter-efficient fine-tuning method for large language models. arXiv preprint arXiv: 2508.02711, 2025.
[207]	Xu L L, Xie H R, Qin S J, Tao X H, Wang F L. Parameter-efficient fine-tuning methods for pretrained language models: A critical review and assessment. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026, 48(6): 6107−6126 doi: 10.1109/TPAMI.2026.3657354
[208]	Wang L P, Chen S, Jiang L N, Pan S, Cai R Z, Yang S, et al. Parameter-efficient fine-tuning in large language models: A survey of methodologies. Artificial Intelligence Review, 2025, 58(8): Article No. 227 doi: 10.1007/s10462-025-11236-4
[209]	Meng L C, Li H D, Chen B C, Lan S Y, Wu Z X, Jiang Y G, et al. AdaViT: Adaptive vision Transformers for efficient image recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, USA: IEEE, 2022. 12299−12308
[210]	Chen S F, Ge C J, Tong Z, Wang J L, Song Y B, Wang J, et al. AdaptFormer: Adapting vision Transformers for scalable visual recognition. In: Proceedings of the 36th International Conference on Neural Information Processing Systems. New Orleans, USA: Curran Associates Inc., 2022. Article No. 1212
[211]	Chen Z, Duan Y C, Wang W H, He J J, Lu T, Dai J F, et al. Vision Transformer adapter for dense predictions. In: Proceedings of the 11th International Conference on Learning Representations (ICLR). Kigali, Rwanda: OpenReview.net, 2023. 1−20
[212]	Xu M D, Zhang Z, Wei F Y, Hu H, Bai X. Side adapter network for open-vocabulary semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver, Canada: IEEE, 2023. 2945−2954
[213]	Fu M H, Zhu K, Wu J X. DTL: Disentangled transfer learning for visual recognition. In: Proceedings of the 38th AAAI Conference on Artificial Intelligence. Vancouver, Canada: AAAI Press, 2024. 12082−12090
[214]	Chen H, Tao R, Zhang H, Wang Y D, Li X, Ye W, et al. Conv-adapter: Exploring parameter efficient transfer learning for ConvNets. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Seattle, USA: IEEE, 2024. 1551−1561
[215]	Chen Z, Zeng Y, Chen Z H, Gao H Z, Chen L, Liu J M, et al. VFM-adapter: Adapting visual foundation models for dense prediction with dynamic hybrid operation mapping. In: Proceedings of the 39th AAAI Conference on Artificial Intelligence. Philadelphia, USA: AAAI Press, 2025. 2385−2393
[216]	Pandey D S, Pyakurel S, Yu Q. Be confident in what you know: Bayesian parameter efficient fine-tuning of vision foundation models. In: Proceedings of the 38th International Conference on Neural Information Processing Systems. Vancouver, Canada: Curran Associates Inc., 2024. Article No. 1424
[217]	Yu Q H, He J, Deng X Q, Shen X H, Chen L C. Convolutions die hard: Open-vocabulary segmentation with single frozen convolutional CLIP. In: Proceedings of the 37th International Conference on Neural Information Processing Systems. New Orleans, USA: Curran Associates Inc., 2023. Article No. 1399
[218]	Yin D S, Yang Y R, Wang Z C, Yu H F, Wei K W, Sun X. 1% VS 100%: Parameter-efficient low rank adapter for dense predictions. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver, Canada: IEEE, 2023. 20116−20126
[219]	Hu Z X, Wei Y X, Shen L, Yuan C, Tao D C. LoRA recycle: Unlocking tuning-free few-shot adaptability in visual foundation models by recycling pre-tuned LoRAs. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville, USA: IEEE, 2025. 25026−25037
[220]	Mai Z D, Zhang P, Tu C H, Chen H Y, Nguyen Q H, Zhang L, et al. Lessons and insights from a unifying study of parameter-efficient fine-tuning (PEFT) in visual recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville, USA: IEEE, 2025. 14845−14857
[221]	Feng T T, Narayanan S. PEFT-SER: On the use of parameter efficient transfer learning approaches for speech emotion recognition using pre-trained speech models. In: Proceedings of the 11th International Conference on Affective Computing and Intelligent Interaction (ACII). Cambridge, USA: IEEE, 2023. 1−8
[222]	Lin T H, Wang H S, Weng H Y, Peng K C, Chen Z C, Lee H Y. PEFT for speech: Unveiling optimal placement, merging strategies, and ensemble techniques. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW). Seoul, South Korea: IEEE, 2024. 705−709
[223]	Chen S Y, Wang C Y, Chen Z Y, Wu Y, Liu S J, Chen Z, et al. WavLM: Large-scale self-supervised pre-training for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing, 2022, 16(6): 1505−1518 doi: 10.1109/JSTSP.2022.3188113
[224]	Ali M N, Falavigna D, Brutti A. EFL-PEFT: A communication efficient federated learning framework using PEFT sparsification for ASR. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Hyderabad, India: IEEE, 2025. 1−5
[225]	Meng L W, Hu S J, Kang J W, Li Z Q, Wang Y J, Wu W X, et al. Large language model can transcribe speech in multi-talker scenarios with versatile instructions. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Hyderabad, India: IEEE, 2025. 1−5
[226]	Lin T Q, Huang W P, Tang H, Lee H Y. Speech-FT: Merging pre-trained and fine-tuned speech representation models for cross-task generalization. arXiv preprint arXiv: 2502.12672, 2025.
[227]	Hsu W N, Bolte B, Tsai Y H H, Lakhotia K, Salakhutdinov R, Mohamed A. HuBERT: Self-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021, 29: 3451−3460 doi: 10.1109/TASLP.2021.3122291
[228]	Baevski A, Zhou H, Mohamed A, Auli M. wav2vec 2.0: A framework for self-supervised learning of speech representations. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. Vancouver, Canada: Curran Associates Inc., 2020. Article No. 1044
[229]	Ling S S, Liu Y Z. DeCoAR 2.0: Deep contextualized acoustic representations with vector quantization. arXiv preprint arXiv: 2012.06659, 2020.
[230]	Zhong Z S, Wang C Y, Liu Y Q, Yang S Q, Tang L X, Zhang Y C, et al. LYRA: An efficient and speech-centric framework for omni-cognition. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Honolulu, USA: IEEE, 2025. 3694−3704
[231]	Liang F, Wu B C, Dai X L, Li K P, Zhao Y N, Zhang H, et al. Open-vocabulary semantic segmentation with mask-adapted CLIP. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver, Canada: IEEE, 2023. 7061−7070
[232]	Gao P, Geng S J, Zhang R R, Ma T L, Fang R Y, Zhang Y F, et al. CLIP-adapter: Better vision-language models with feature adapters. International Journal of Computer Vision, 2024, 132(2): 581−595 doi: 10.1007/s11263-023-01891-x
[233]	Yan L Q, Han C, Xu Z L, Liu D F, Wang Q F. Prompt learns prompt: Exploring knowledge-aware generative prompt collaboration for video captioning. In: Proceedings of the 32nd International Joint Conference on Artificial Intelligence. Macao, China: ijcai.org, 2023. 1622−1630
[234]	Yang L X, Zhang R Y, Wang Y C, Xie X H. MMA: Multi-modal adapter for vision-language models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE, 2024. 23826−23837
[235]	Huang Z H, Guo Y C, Wang H R, Yi R, Ma L Z, Cao Y P, et al. Mv-adapter: Multi-view consistent image generation made easy. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Honolulu, USA: IEEE, 2025. 16377−16387
[236]	Li F D. MH-PEFT: Mitigating hallucinations in large vision-language models through the PEFT method. In: Proceedings of the 2nd International Conference on Generative Artificial Intelligence and Information Security. Hangzhou, China: ACM, 2025. 137−142
[237]	Xu Z Q, Peng Z L, Yang X K, Shen W. FATE: Feature-adapted parameter tuning for vision-language models. In: Proceedings of the 39th AAAI Conference on Artificial Intelligence. Philadelphia, USA: AAAI Press, 2025. 9014−9022
[238]	Zhang Y, Deng Y X, Guo M H, Hu S M. Adaptive parameter selection for tuning vision-language models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville, USA: IEEE, 2025. 4280−4290
[239]	Farina M, Mancini M, Iacca G, Ricci E. Rethinking few-shot adaptation of vision-language models in two stages. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville, USA: IEEE, 2025. 29989−29998
[240]	Zanella M, Ayed I B. Low-rank few-shot adaptation of vision-language models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Seattle, USA: IEEE, 2024. 1593−1603
[241]	Ornelas R, Chao A, Gupta S, Chao E, Greer R. Improving event-phase captions in multi-view urban traffic videos via prompt-aware LoRA tuning of vision language models. In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops (ICCVW). Honolulu, USA: IEEE, 2025. 1842−1848
[242]	Lu H D, Zhao C Y, Xue J, Yao L N, Moore K, Gong D. Adaptive rank, reduced forgetting: Knowledge retention in continual learning vision-language models with dynamic rank-selective LoRA. In: Proceedings of the 14th International Conference on Learning Representations (ICLR). Rio de Janeiro, Brazil: OpenReview.net, 2026. 1−26
[243]	Chitty-Venkata K T, Emani M, Vishwanath V. LangVision-LoRA-NAS: Neural architecture search for variable LoRA rank in vision language models. In: Proceedings of the IEEE International Conference on Image Processing (ICIP). Anchorage, USA: IEEE, 2025. 1330−1335
[244]	Peng Y Q, Wang P F, Liu J Z, Chen S F. GLAD: Generalizable tuning for vision-language models. In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops (ICCVW). Honolulu, USA: IEEE, 2025. 4369−4379
[245]	Nguyen T, Wu X B, Dong X S, Le K M, Hu Z Y, Nguyen C D, et al. READ-PVLA: Recurrent adapter with partial video-language alignment for parameter-efficient transfer learning in low-resource video-language modeling. In: Proceedings of the 38th AAAI Conference on Artificial Intelligence. Vancouver, Canada: AAAI Press, 2024. 18824−18832
[246]	Wang Y H, Ding P X, Li L X, Cui C, Ge Z R, Tong X Y, et al. VLA-adapter: An effective paradigm for tiny-scale vision-language-action model. In: Proceedings of the 40th AAAI Conference on Artificial Intelligence. Singapore: AAAI Press, 2026. 18638−18646
[247]	Zheng J L, Li J X, Wang Z H, Liu D X, Kang X R, Feng Y C, et al. X-VLA: Soft-prompted Transformer as scalable cross-embodiment vision-language-action model. In: Proceedings of the 14th International Conference on Learning Representations (ICLR). Rio de Janeiro, Brazil: OpenReview.net, 2026. 1−27
[248]	Kim M J, Finn C, Liang P. Fine-tuning vision-language-action models: Optimizing speed and success. arXiv preprint arXiv: 2502.19645, 2025.
[249]	Kim Y, Kim J, Mok W L, Park J H, Lee S K. Client-customized adaptation for parameter-efficient federated learning. In: Proceedings of the Findings of the Association for Computational Linguistics. Toronto, Canada: Association for Computational Linguistics, 2023. 1159−1172
[250]	Cai D Q, Wu Y Z, Wang S G, Xu M W. FedAdapter: Efficient federated learning for mobile NLP. In: Proceedings of the ACM Turing Award Celebration Conference. Wuhan, China: ACM, 2023. 27−28
[251]	Chen H K, Zhang Y, Krompass D, Gu J D, Tresp V. FedDAT: An approach for foundation model finetuning in multi-modal heterogeneous federated learning. In: Proceedings of the 38th AAAI Conference on Artificial Intelligence. Vancouver, Canada: AAAI Press, 2024. 11285−11293
[252]	Xiong B C, Yang X S, Song Y G, Wang Y W, Xu C S. Pilot: Building the federated multimodal instruction tuning framework. In: Proceedings of the 39th AAAI Conference on Artificial Intelligence. Philadelphia, USA: AAAI Press, 2025. 21716−21724
[253]	Saha P, Mishra D, Wagner F, Kamnitsas K, Noble J A. FedPIA-permuting and integrating adapters leveraging Wasserstein barycenters for finetuning foundation models in multi-modal federated learning. In: Proceedings of the 39th AAAI Conference on Artificial Intelligence. Philadelphia, USA: AAAI Press, 2025. 20228−20236
[254]	Wang R, Yu T, Zhang R Y, Kim S, Rossi R, Zhao H D, et al. Personalized federated learning for text classification with gradient-free prompt tuning. In: Proceedings of the Findings of the Association for Computational Linguistics. Mexico City, Mexico: Association for Computational Linguistics, 2024. 4597−4612
[255]	Yang F E, Wang C Y, Wang Y C F. Efficient model personalization in federated learning via client-specific prompt generation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Paris, France: IEEE, 2023. 19102−19111
[256]	Piao H M, Wu Y C, Wu D P, Wei Y. Federated continual learning via prompt-based dual knowledge transfer. In: Proceedings of the 41st International Conference on Machine Learning (ICML). Vienna, Austria: PMLR, 2024. 40725−40739
[257]	Bai S K, Zhang J, Guo S, Li S C, Guo J C, Hou J, et al. DiPrompT: Disentangled prompt tuning for multiple latent domain generalization in federated learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE, 2024. 27274−27283
[258]	Tan Y, Long G D, Ma J, Liu L, Zhou T Y, Jiang J. Federated learning from pre-trained models: A contrastive learning approach. In: Proceedings of the 36th International Conference on Neural Information Processing Systems. New Orleans, USA: Curran Associates Inc., 2022. Article No. 1405
[259]	Zhang Z, Yang Y H, Dai Y, Wang Q F, Yu Y, Qu L Z, et al. FedPETuning: When federated learning meets the parameter-efficient tuning methods of pre-trained language models. In: Proceedings of the Annual Meeting of the Association for Computational Linguistics. Toronto, Canada: Association for Computational Linguistics, 2023. 9963−9977
[260]	Su S C, Li B, Xue X Y. FedRA: A random allocation strategy for federated tuning to unleash the power of heterogeneous clients. In: Proceedings of the 18th European Conference on Computer Vision (ECCV). Milan, Italy: Springer, 2024. 342−358
[261]	Yang Y Y, Long G D, Shen T, Jiang J, Blumenstein M. Dual-personalizing adapter for federated foundation models. In: Proceedings of the 38th International Conference on Neural Information Processing Systems. Vancouver, Canada: Curran Associates Inc., 2024. Article No. 1245
[262]	Ghiasvand S, Yang Y F, Xue Z Y, Alizadeh M, Zhang Z, Pedarsani R. Communication-efficient and tensorized federated fine-tuning of large language models. In: Proceedings of the Findings of the Association for Computational Linguistics. Vienna, Austria: Association for Computational Linguistics, 2025. 24192−24207
[263]	Bian J M, Wang L, Zhang L T, Xu J. LoRA-FAIR: Federated LoRA fine-tuning with aggregation and initialization refinement. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Honolulu, USA: IEEE, 2025. 3737−3746