面向示意图问答的前瞻性多视角视觉推理框架

张新宇; 吴艳瑞; 张玲玲; 杨泽晟; 董宇轩; 武亚强; 郑庆华

doi:10.16383/j.aas.c260129

面向示意图问答的前瞻性多视角视觉推理框架

doi: 10.16383/j.aas.c260129 cstr: 32138.14.j.aas.c260129

张新宇^{1, 2,},
吴艳瑞^{1, 3,},
张玲玲^{1, 2,},
杨泽晟^{1, 3,},
董宇轩^{1, 3,},
武亚强^4,,
郑庆华^5,

1.
西安交通大学计算机科学与技术学院西安 710049
2.
西安交通大学智能网络与网络安全教育部重点实验室西安 710049
3.
西安交通大学陕西省大数据知识工程重点实验室西安 710049
4.
联想人工智能中心北京 100094
5.
同济大学上海 200092

基金项目: 教育部基础学科和交叉学科突破计划 (JYB2025XDXM116), 国家自然科学基金(62137002, 62293553, 62477036)资助

详细信息

作者简介:
张新宇：西安交通大学计算机科学与技术学院博士研究生. 主要研究方向为视觉推理, 视觉语言模型. E-mail: zhang1393869716@stu.xjtu.edu.cn

吴艳瑞：西安交通大学计算机科学与技术学院硕士研究生. 主要研究方向为示意图理解和逻辑推理. E-mail: yanrui.wu@stu.xjtu.edu.cn

张玲玲：西安交通大学计算机科学与技术学院副教授. 主要研究方向为智慧教育, 知识图谱和大模型推理. 本文通信作者. E-mail: zhanglling@xjtu.edu.cn

杨泽晟：西安交通大学计算机科学与技术学院硕士研究生. 主要研究方向为自然语言处理. E-mail: 2204215061@stu.xjtu.edu.cn

董宇轩：西安交通大学计算机科学与技术学院硕士研究生. 主要研究方向为多模态大模型和强化学习. E-mail: yuxuandong@stu.xjtu.edu.cn

武亚强：西安交通大学计算机学院兼职教授, 联想集团人工智能中心高级总监, 正高级工程师. 主要研究方向为企业级人工智能包括评估、推理与编排. E-mail: wuyqe@lenovo.com

郑庆华：中国工程院院士, 同济大学教授. 主要研究方向为大数据知识工程. E-mail: qhzheng@mail.xjtu.edu.cn

计量
- 文章访问数: 11
- HTML全文浏览量: 6
- 被引次数: 0
出版历程
- 收稿日期: 2026-02-14
- 录用日期: 2026-05-13
- 网络出版日期: 2026-07-07

A Look-ahead Multi-perspective Visual Reasoning Framework for Diagram Question Answering

ZHANG Xin-Yu^{1, 2
,},
WU Yan-Rui^{1, 3
,},
ZHANG Ling-Ling^{1, 2
,},
YANG Ze-Sheng^{1, 3
,},
DONG Yu-Xuan^{1, 3
,},
WU Ya-Qiang^4
,,
ZHENG Qing-Hua^5
,

1.
School of Computer Science and Technology, Xi＇an Jiaotong University, Xi＇an 710049
2.
Ministry of Education Key Laboratory of Intelligent Networks and Network Security, Xi＇an Jiaotong University, Xi＇an 710049
3.
Shaanxi Province Key Laboratory of Big Data Knowledge Engineering, Xi＇an Jiaotong University, Xi＇an, 710049
4.
Lenovo AI Technology Center, Beijing 100094
5.
Tongji University, Shanghai 200092

Funds: Supported by Fundamental and Interdisciplinary Disciplines Breakthrough Plan of the Ministry of Education of China (JYB2025XDXM116) and National Natural Science Foundation of China (62137002, 62293553, 62477036)

More Information

Author Bio:
ZHANG Xin-Yu　Ph. D. candidate at the School of Computer Science and Technology, Xi＇an Jiaotong University. His research interests include visual reasoning and vision-language models

WU Yan-Rui　Master student at the School of Computer Science and Technology, Xi＇an Jiaotong University. Her research interests include diagram understanding and logical reasoning

ZHANG Ling-Ling　Associate professor at the School of Computer Science and Technology, Xi＇an Jiaotong University. Her research interests include smart education, knowledge graphs, and LLM reasoning. Corresponding author of this paper

YANG Ze-Sheng　Master student at the School of Computer Science and Technology, Xi＇an Jiaotong University. His main research interest is natural language processing

DONG Yu-Xuan　Master student at the School of Computer Science and Technology, Xi＇an Jiaotong University. His research interests include multimodal large language models and reinforcement learning

WU Ya-Qiang　Adjunct professor at the School of Computer Science and Technology, Xi＇an Jiaotong University, Senior Director at the Lenovo AI Technology Center, Professor-level Senior Engineer. His research interests include enterprise artificial intelligence, covering evaluation, reasoning, and orchestration

ZHENG Qing-Hua　Academician of the Chinese Academy of Engineering, Professor of Tongji University. His research interests include big data knowledge engineering

摘要

摘要: 针对多模态大模型在抽象度示意图(如几何、电路、科学图表)理解中面临的视觉感知静态化挑战, 提出一种基于前瞻性机制的多视角视觉推理框架. 该方法旨在模拟人类“慢思考”的认知机理, 通过引入相对注意力与信息增益门控, 实现对关键视觉线索的主动搜索与去噪; 利用逻辑置信度与视觉相关性进行双维质量评估, 构建“观察–假设–验证”的动态闭环推理系统. 实验表明, 该方法在多学科的示意图推理数据集上显著优于现有主流方法, 在并未引入过多计算开销的同时, 大幅提升了复杂示意图问答的准确率与鲁棒性.
- 视觉语言模型 /
- 视觉推理 /
- 示意图问答 /
- 前瞻性机制
Abstract: To address the challenges of visual perception lag and logical hallucinations faced by Multimodal Large Language Models in complex educational reasoning tasks, a Look-ahead Multi-Perspective Framework is proposed. Mimicking the cognitive mechanism of human “System 2” thinking, this method achieves active search and denoising of key visual clues by introducing relative attention and information gain gating. Furthermore, it constructs a dynamic “observation-hypothesis-verification” closed-loop reasoning system by utilizing a dual-dimensional quality evaluation of logical confidence and visual relevance. Experiments on authoritative benchmarks such as mathematical geometry and scientific charts demonstrate that this method significantly outperforms existing mainstream approaches across models of different scales. It effectively enhances the reasoning accuracy and robustness of complex cross-modal tasks while significantly reducing computational overhead.
- vision-language models /
- visual reasoning /
- diagram question answering /
- look-ahead mechanism

HTML全文

图 1 不同视觉推理机制在求解几何问题上的对比

Fig. 1 Comparison of different visual reasoning mechanisms in solving geometry problem

下载: 全尺寸图片幻灯片

图 2 LaMP整体架构示意图

Fig. 2 The diagram of the overall architecture of LaMP

下载: 全尺寸图片幻灯片

图 3 消融实验结果

Fig. 3 Results of ablation study

下载: 全尺寸图片幻灯片

图 4 LaMP方法在示意图问题求解中的多视角探索与局部验证案例分析

Fig. 4 A case study on the LaMP for multi-perspective exploration and local verification in diagram problem-solving

下载: 全尺寸图片幻灯片

图 5 LaMP方法在示意图问题求解中的局限性分析

Fig. 5 Analysis of limitations of the LaMP in diagram problem-solving

下载: 全尺寸图片幻灯片

表 1 不同方法在两大类视觉推理基准上的性能对比(Pass@1, %)

Table 1 Performance comparison (Pass@1, %) of different methods on two major categories of visual reasoning benchmarks

	数理推理			多学科综合
模型方法	MathVista	MathVision	PhysReason	AI2D	SciQA	MMStar	M3CoT	Avg.
Qwen2.5VL-7B-Instruct
Baseline	68.2	18.09	14.35	80.51	74.42	63.9	59.62	54.16
MCTS	69.6	18.75	16.27	81.87	78.09	66.2	60.87	55.95
Pred. Dec.	69.9	19.73	18.65	82.67	78.68	65.7	61.34	56.67
ICoT	47.5	10.53	15.73	81.35	77.24	42.1	61.17	47.95
DyFo	68.4	16.78	14.75	80.93	76.95	64.5	61.26	54.80
LaMP	70.8	24.34	21.76	84.07	81.95	69.8	62.68	59.34
InternVL2.5-8B-Instruct
Baseline	64.4	22.00	11.58	81.61	76.15	60.5	57.16	53.34
MCTS	65.0	24.67	14.21	82.97	79.67	62.0	58.24	55.25
Pred. Dec.	65.4	25.00	17.16	83.74	80.37	62.4	58.76	56.12
ICoT	42.9	16.12	13.57	82.25	78.63	39.4	58.50	47.34
DyFo	64.7	20.39	12.08	81.96	77.89	61.5	58.58	53.87
LaMP	67.1	28.62	19.35	84.84	82.10	65.7	59.32	58.15
Qwen2.5VL-32B-Instruct
Baseline	74.7	25.33	19.42	83.29	78.83	69.5	62.81	59.13
MCTS	76.2	27.31	21.38	84.68	80.76	70.6	64.15	60.73
Pred. Dec.	76.6	27.96	22.76	85.17	81.95	71.1	64.62	61.45
ICoT	58.7	21.05	20.76	84.03	79.77	54.6	63.93	54.69
DyFo	75.6	24.67	20.35	83.52	78.93	70.1	64.32	59.64
LaMP	77.7	30.59	25.43	85.87	84.33	72.9	66.35	63.31

下载: 导出CSV

表 2 不同推理方法在各基座模型上的计算成本(FLOPs)对比

Table 2 A comparison of the computational cost (FLOPs) of different reasoning methods across different base models

基座模型	Baseline	MCTS	Pred. Dec.	ICoT	DyFo	LaMP
Qwen2.5-VL-7B	8.35e+12	4.05e+14	1.85e+14	1.88e+13	1.98e+13	1.54e+14
InternVL2.5-8B	1.02e+13	4.58e+14	2.09e+14	2.21e+13	2.38e+13	1.87e+14
Qwen2.5-VL-32B	3.59e+13	1.79e+15	8.17e+14	8.07e+13	8.63e+13	6.38e+14

下载: 导出CSV

表 3 各推理方法的计算成本、推理时间与准确率综合对比(Qwen2.5-VL-7B, A100)

Table 3 Comprehensive comparison of computational cost, inference time and accuracy for different reasoning methods (Qwen2.5-VL-7B, A100)

方法	FLOPs	时间开销(s)	平均准确率(%)	准确率增益
Baseline	8.35e+12	~ 3	54.16	—
ICoT	1.88e+13	~ 8	47.95	−6.21
DyFo	1.98e+13	~ 12	54.80	+0.64
LaMP	1.54e+14	~ 68	59.34	+5.18
Pred. Dec.	1.85e+14	~ 81	56.67	+2.51
MCTS	4.05e+14	~ 170	55.95	+1.79

下载: 导出CSV

表 4 LaMP各阶段计算开销与时间分解(Qwen2.5-VL-7B, 4样本/迭代× 5轮有效迭代)

Table 4 Computational cost and time breakdown for each stage of LaMP (Qwen2.5-VL-7B, 4 samples/iteration × 5 effective iterations)

阶段	FLOPs	FLOPs占比	单卡(s)	四卡(s)
阶段一: 视觉多样性采样	1.48e+13	9.6%	~ 7	~ 7
阶段二: 自适应门控	—	—	$< $ 1	$< $ 1
阶段三: 多视角生成与质量评估	1.39e+14	90.4%	~ 58	~ 21
阶段四: 推理链动态扩展与决策	—	—	$< $ 1	$< $ 1
I/O与图像预处理	—	—	~ 2	~ 2
合计	1.54e+14	100%	~ 68	~ 31

下载: 导出CSV

表 5 变体级组合消融实验结果(Qwen2.5-VL-7B, Pass@1 %)

Table 5 Results of variant-level combinatorial ablation study (Qwen2.5-VL-7B, Pass@1 %)

变体A	变体B	MathVista	M3CoT
$\checkmark $	$\checkmark $	70.8	62.68
$\checkmark $	×	69.4	60.91
×	$\checkmark $	69.8	61.82
×	×	68.3	59.80

下载: 导出CSV

表 6 跨方法组件替换实验结果(Qwen2.5-VL-7B, Pass@1 %)

Table 6 Results of cross-method component replacement experiments (Qwen2.5-VL-7B, Pass@1 %)

组合方式	MathVista	M3CoT
DyFo + Predictive Decoding	69.0	60.96
DyFo + LaMP变体B(前瞻评估)	69.7	61.73
LaMP变体A(视觉感知)+ Predictive Decoding	69.4	61.30
Full LaMP(变体A + 变体B)	70.8	62.68

下载: 导出CSV

表 7 前瞻参数$ L $和采样数量$ K $对实验结果的影响分析

Table 7 Analysis of the impact of lookahead parameter $ L $ and sampling number $ K $ on the experimental results

前瞻步数	MathVista	M3CoT	FLOPs	采样数	MathVista	M3CoT	FLOPs
$ L=3 $	69.4	60.65	9.50e+13	$ K=1 $	68.9	60.48	8.70e+13
$ L=4 $	70.2	61.91	1.25e+14	$ K=2 $	70.1	61.73	1.22e+14
$ L=5 $	70.8	62.68	1.54e+14	$ K=3 $	70.8	62.68	1.54e+14
$ L=6 $	71.0	63.07	1.79e+14	$ K=4 $	71.2	63.55	1.83e+14
$ L=7 $	71.1	63.42	2.02e+14	$ K=5 $	71.5	64.06	2.10e+14

下载: 导出CSV

表 8 逻辑置信度$ w_p $与视觉相关性$ w_v $在不同数据集上的敏感性分析(Qwen2.5-VL-7B)

Table 8 Sensitivity analysis of logical confidence $ w_p $ and visual relevance $ w_v $ on different datasets (Qwen2.5-VL-7B)

$ w_v $	$ w_p $	MathVista (数学)	PhysReason (物理)	SciQA-IMG (多学科)
0.1	0.9	70.2	20.21	80.48
0.2	0.8	70.5	21.05	81.23
0.3	0.7	70.8	21.76	81.95
0.4	0.6	70.6	21.28	81.61
0.5	0.5	70.1	20.47	80.83

下载: 导出CSV

表 9 候选区域生成核心参数敏感性分析(Qwen2.5-VL-7B, MathVista)

Table 9 Sensitivity analysis of core parameters for candidate region generation (Qwen2.5-VL-7B, MathVista)

候选数$ M $	3	5	10	15	20
Acc. (%)	69.3	69.9	70.8	70.6	70.3
裁剪比例$ \rho_{\min} $	0.1	0.2	0.3	0.4	0.5
Acc. (%)	69.5	70.2	70.8	70.1	69.4

下载: 导出CSV

表 10 信息增益门控阈值$ \theta $的敏感性分析(Qwen2.5-VL-7B, $ K=3 $)

Table 10 Sensitivity analysis of information gain gating threshold $ \theta $ (Qwen2.5-VL-7B, $ K=3 $)

$ \theta $	MathVista	M3CoT	FLOPs
0.6	70.1	61.95	1.76e+14
0.8	70.5	62.34	1.64e+14
1.0	70.8	62.68	1.54e+14
1.2	70.3	62.12	1.38e+14
1.4	69.5	61.52	1.15e+14

下载: 导出CSV

表 11 不同注意力提取策略的性能对比(Qwen2.5-VL-7B)

Table 11 Performance comparison of different attention extraction strategies (Qwen2.5-VL-7B)

注意力提取策略	MathVista	M3CoT
前1/3层平均	69.3	61.04
中间1/3层平均	69.8	61.65
后1/3层平均	70.3	62.17
全体层平均	70.0	61.82
最后一层Top-4头	70.5	62.34
最后一层平均	70.8	62.68

下载: 导出CSV

表 12 多视图输入策略对比(Qwen2.5-VL-7B, $ K=3 $, $ \theta=1.0 $)

Table 12 Comparison of multi-view input strategies (Qwen2.5-VL-7B, $ K=3 $, $ \theta=1.0 $)

策略	MathVista	M3CoT
多图联合输入	69.7	61.73
单视图独立推理	70.8	62.68

下载: 导出CSV

参考文献(37)

[1]	缪青海, 王兴霞, 杨静, 赵勇, 王雨桐, 陈圆圆, 等. 从基础智能到通用智能: 基于大模型的GenAI和AGI之现状与展望. 自动化学报, 2024, 50(4): 674−687 doi: 10.16383/j.aas.c240156 Miao Qing-Hai, Wang Xing-Xia, Yang Jing, Zhao Yong, Wang Yu-Tong, Chen Yuan-Yuan, et al. From foundation intelligence to general intelligence: The state-of-art and perspectives of GenAI and AGI based on foundation models. Acta Automatica Sinica, 2024, 50(4): 674−687 doi: 10.16383/j.aas.c240156
[2]	Liu H T, Li C Y, Wu Q Y, Lee Y J. Visual instruction tuning. In: Proceedings of Advances in Neural Information Processing Systems (NeurIPS). Vancouver, BC, Canada: Curran Associates, 2024. 34892−34916
[3]	Cui C, Ma Y S, Cao X, Ye W Q, Zhou Y, Liang K Z, et al. A survey on multimodal large language models for autonomous driving. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA, USA: IEEE, 2024. 958−979
[4]	Lu P, Bansal H, Xia T, Liu J C, Li C Y, Hajishirzi H, et al. MathVista: Evaluating mathematical reasoning of foundation models in visual contexts. In: Proceedings of the International Conference on Learning Representations (ICLR). Vienna, Austria: OpenReview. net, 2024. 23439−23554.
[5]	Yue X, Ni Y S, Zhang K, Zheng T Y, Liu R Q, Zhang G, et al. MMMU: A massive multi-discipline multimodal understanding and reasoning benchmark for expert AGI. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA, USA: IEEE, 2024. 9556−9567
[6]	Guan T R, Liu F X, Wu X Y, Xian R Q, Li Z X, Liu X Y, et al. HallusionBench: An advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA, USA: IEEE, 2024. 14375−14385
[7]	Zhang X Y, Zhang L L, Wu Y R, Wang S W, Wu W J, Huang M Y, et al. Memory-enriched thought-by-thought framework for complex diagram question answering. Computer Vision and Image Understanding, 2025Article No. 104608
[8]	Wang S W, Zhang L L, Zhu L J, Qin T, Yap K H, Zhang X Y, et al. CoG-DQA: Chain-of-guiding learning with large language models for diagram question answering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA, USA: IEEE, 2024. 13969−13979.
[9]	Wang L, Hu Y, He J B, Xu X, Liu N, Liu H, et al. T-SciQ: Teaching multimodal chain-of-thought reasoning via large language model signals for science question answering. In: Proceedings of the AAAI Conference on Artificial Intelligence (AAAI). Vancouver, BC, Canada: AAAI Press, 2024. 19162−19170
[10]	Li Y F, Du Y F, Zhou K, Wang J P, Zhao W X, Wen J R. Evaluating object hallucination in large vision-language models. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). Singapore: Association for Computational Linguistics, 2023. 249−260
[11]	Yin S K, Fu C Y, Zhao S R, Xu T, Wang H, Sui D H, et al. Woodpecker: Hallucination correction for multimodal large language models. Science China Information Sciences, 2024, 67(12): Article No. 220105 doi: 10.1007/s11432-024-4251-x
[12]	Wei J, Wang X, Schuurmans D, Bosma M, Ichter B, Xia F, et al. Chain-of-thought prompting elicits reasoning in large language models. In: Proceedings of Advances in Neural Information Processing Systems (NeurIPS). New Orleans, LA, USA: Curran Associates, 2022. 24824−24837
[13]	Yao S Y, Yu D, Zhao J, Shafran I, Griffiths T, Cao Y, et al. Tree of thoughts: Deliberate problem solving with large language models. In: Proceedings of Advances in Neural Information Processing Systems (NeurIPS). New Orleans, LA, USA: Curran Associates, 2023. 11809−11822
[14]	Hao S B, Gu Y, Ma H D, Hong J, Wang Z, Wang D, et al. Reasoning with language model is planning with world model. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). Singapore: Association for Computational Linguistics, 2023. 8154−8173
[15]	Ma C, Zhao H T, Zhang J L, He J X, Kong L P. Non-myopic generation of language models for reasoning and planning. In: Proceedings of the International Conference on Learning Representations (ICLR). Singapore: OpenReview. net, 2025. 35031−35058.
[16]	Wang X Z, Wei J, Schuurmans D, Le Q, Chi E, Narang S, et al. Self-consistency improves chain of thought reasoning in language models. In: Proceedings of the International Conference on Learning Representations (ICLR). Kigali, Rwanda: OpenReview. net, 2023.
[17]	Kahneman, D. Thinking, Fast and Slow. New York: Farrar, Straus and Giroux, 2011.
[18]	卢经纬, 郭超, 戴星原, 缪青海, 王兴霞, 杨静, 等. 问答ChatGPT之后: 超大预训练模型的机遇和挑战. 自动化学报, 2023, 49(4): 705−717 Lu Jing-Wei, Guo Chao, Dai Xing-Yuan, Miao Qing-Hai, Wang Xing-Xia, Yang Jing, et al. The ChatGPT after: Opportunities and challenges of very large scale pre-trained models. Acta Automatica Sinica, 2023, 49(4): 705−717
[19]	朱嘉骏, 包美凯, 张凯, 刘烨, 刘淇. 基于多源知识注入的常识问答方法研究. 计算机工程与科学, 2025, 47(2): 349−360 doi: 10.3969/j.issn.1007-130X.2025.02.017 Zhu Jia-Jun, Bao Mei-Kai, Zhang Kai, Liu Ye, Liu Qi. A commonsense question answering method based on multi-source knowledge infusion. Computer Engineering and Science, 2025, 47(2): 349−360 doi: 10.3969/j.issn.1007-130X.2025.02.017
[20]	Shinn N, Cassano F, Gopinath A, Narasimhan K, Yao S Y. Reflexion: Language agents with verbal reinforcement learning. In: Proceedings of Advances in Neural Information Processing Systems (NeurIPS). New Orleans, LA, USA: Curran Associates, 2023. 8634−8652
[21]	Lightman H, Kosaraju V, Burda Y, Edwards H, Baker B, Lee T, et al. Let's verify step by step. In: Proceedings of the International Conference on Learning Representations (ICLR). Vienna, Austria: OpenReview. net, 2024. 39578−39601.
[22]	王全子昂, 王仁振, 孟德宇, 徐宗本. 面向大模型时代的持续学习方法论演变. 自动化学报, 2025, 51(8): 1−27 Wang Quan-Zi-Ang, Wang Ren-Zhen, Meng De-Yu, Xu Zong-Ben. The evolution of continual learning methodologies in the era of large models. Acta Automatica Sinica, 2025, 51(8): 1−27
[23]	Ha D, Schmidhuber J. Recurrent world models facilitate policy evolution. In: Proceedings of Advances in Neural Information Processing Systems (NeurIPS). Montréal, QC, Canada: Curran Associates, 2018. 2455−2467
[24]	Yao S Y, Zhao J, Yu D, Du N, Shafran I, Narasimhan K R, et al. ReAct: Synergizing reasoning and acting in language models. In: Proceedings of the International Conference on Learning Representations (ICLR). Kigali, Rwanda: OpenReview. net, 2023.
[25]	郑逸宁, 余镇, 李不凡, 杨捷, 殷林琪, 印张悦, 等. 大语言模型的工具使用综述. 自动化学报, 2025, 51(11): 2371−2386 doi: 10.16383/j.aas.c240793 Zheng Yi-Ning, Yu Zhen, Li Bu-Fan, Yang Jie, Yin Lin-Qi, Yin Zhang-Yue, et al. Survey of tool use in large language models. Acta Automatica Sinica, 2025, 51(11): 2371−2386 doi: 10.16383/j.aas.c240793
[26]	Hu Y S, Shi W J, Fu X Y, Roth D, Ostendorf M, Zettlemoyer L, et al. Visual sketchpad: Sketching as a visual chain of thought for multimodal language models. In: Proceedings of Advances in Neural Information Processing Systems (NeurIPS). Vancouver, BC, Canada: Curran Associates, 2024, 139348−139379
[27]	Qi J, Ding M, Wang W H, Bai Y S, Lv Q S, Hong W Y, et al. CogCoM: A visual language model with chain-of-manipulations reasoning. In: Proceedings of the International Conference on Learning Representations (ICLR). Vienna, Austria: OpenReview. net, 2024. 11090−11110.
[28]	张慧, 梁姝彤, 李明轩, 田永林, 葛经纬, 于慧, 等. 视觉–语言–动作模型综述: 从前史到前沿. 自动化学报, 2025, 51(9): 1922−1950 doi: 10.16383/j.aas.c250417 Zhang Hui, Liang Shu-Tong, Li Ming-Xuan, Tian Yong-Lin, Ge Jing-Wei, Yu Hui, et al. Vision-language-action models: From the early foundations to the state-of-the-art. Acta Automatica Sinica, 2025, 51(9): 1922−1950 doi: 10.16383/j.aas.c250417
[29]	Dai W L, Li J N, Li D X, Tiong A, Zhao J Q, Wang W S, et al. InstructBLIP: Towards general-purpose vision-language models with instruction tuning. In: Proceedings of Advances in Neural Information Processing Systems (NeurIPS). New Orleans, LA, USA: Curran Associates, 2023. 49250−49267.
[30]	Wang K, Pan J T, Shi W K, Lu Z M, Ren H X, Zhou A J, et al. Measuring multimodal mathematical reasoning with Math-Vision dataset. In: Proceedings of Advances in Neural Information Processing Systems (NeurIPS). Vancouver, BC, Canada: Curran Associates, 2024. 95095−95169
[31]	Zhang X Y, Dong Y X, Wu Y R, Huang J X, Jia C Y, Fernando B, et al. PhysReason: A comprehensive benchmark towards physics-based reasoning. In: Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). Vienna, Austria: Association for Computational Linguistics, 2025. 16593−16615.
[32]	Kembhavi A, Salvato M, Kolve E, Seo M, Hajishirzi H, Farhadi A. A diagram is worth a dozen images. In: Proceedings of the European Conference on Computer Vision (ECCV). Amsterdam, The Netherlands: Springer, 2016. 235−251
[33]	Lu P, Mishra S, Xia T L, Qiu L, Chang K W, Zhu S C, et al. Learn to explain: Multimodal reasoning via thought chains for science question answering. In: Proceedings of Advances in Neural Information Processing Systems (NeurIPS). New Orleans, LA, USA: Curran Associates, 2022. 2707−2721
[34]	Chen L, Li J S, Dong X Y, Zhang P, Zang Y H, Chen Z H, et al. Are we on the right way for evaluating large vision-language models? In: Proceedings of Advances in Neural Information Processing Systems (NeurIPS). Vancouver, BC, Canada: Curran Associates, 2024. 27056−27087.
[35]	Chen Q G, Qin L B, Zhang J, Chen Z, Xu X, Che W X. M³ CoT: A Novel Benchmark for Multi-Domain Multi-step Multi-modal Chain-of-Thought. In: Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). Bangkok, Thailand: Association for Computational Linguistics, 2024. 8199−8221
[36]	Li G, Xu J L, Zhao Y Z, Peng Y X. DyFo: A training-free dynamic focus visual search for enhancing LMMs in fine-grained visual understanding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Nashville, TN, USA: IEEE, 2025. 9098−9108
[37]	Gao J, Li Y Q, Cao Z Q, Li W J. Interleaved-modal chain-of-thought. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Nashville, TN, USA: IEEE, 2025. 19520−19529