finding

active

finding:90-45-accuracy-on-scienceqa-benchmark-with-multimodal-cot-large-738m-parameters

90.45% accuracy on ScienceQA benchmark with Multimodal-CoT Large (738M parameters)

State-of-the-art result on ScienceQA; represents +3.91% improvement over prior best published result of 86.54%.

Source paper

extracted_from

Multimodal Chain-of-Thought Reasoning in Language Models

(2023) · Zhuosheng Zhang · Aston Zhang · Mu Li · Hai Zhao +2

Neighborhood — ranked by edge-count

Communities (3)

community

Chain-of-Thought reasoning robustness & safety
members_of
CoT effects on generalization, multimodal QA accuracy, and AI safety alignment training.
Multimodal chain-of-thought reasoning benchmarks
members_of
ScienceQA and related vision-language tasks evaluated via explicit reasoning steps, spanning 738M-parameter models with 89-95% accuracy ranges.
Multimodal Chain-of-Thought Reasoning
members_of
Two-stage rationale-then-answer framework evaluated on ScienceQA benchmark, ~738M parameters.

Frameworks (1)

framework

Multimodal-CoT
supports
A two-stage framework that separates rationale generation and answer inference by incorporating vision and language modalities.

Related by similarity (8)

cosine ≥ 0.65 · no typed edge

Entities in the same semantic neighborhood but without a typed relation to this one — candidates for new edges or unrecognized duplicates.

Multimodal-CoT trained with InstructBLIP/ChatGPT-generated rationales achieves 87.76% accuracy on ScienceQA, comparable to human-annotated rationale performance of 90.45%finding0.826
Evidence that Multimodal-CoT can operate without human-annotated reasoning chains by using large models to generate pseudo-rationales.
Multimodal-CoT with vision features achieves higher validation accuracy at early training epochs (epoch 1-3) compared to one-stage and two-stage language-only baselines on ScienceQAfinding0.801
Evidence that multimodal information accelerates convergence speed during training.
VS-CoT achieves the highest Top@1 accuracy (0.348) and Pass@N accuracy (0.485) on SimpleQA among all methods testedfinding0.791
Shows VS not only maintains but can slightly improve factual accuracy compared to baseline methods
One-stage CoT (QCM→RA) shows 12.31% accuracy drop vs. no-CoT (QCM→A) on ScienceQA; two-stage framework (rationale generation + answer inference) achieves 85.31% accuracy with vision featuresfinding0.785
Empirical evidence that naive one-stage CoT fails in language-only setting; two-stage + vision achieves state-of-the-art.
VS-Multi achieves average accuracy of 37.5% on math benchmarks (avg of MATH500, OlympiadBench, Minerva) with Gemini-2.5-Flash as generator and Qwen3-4B as SFT model, vs. 40.7% baselinefinding0.754
Best VS result in synthetic data generation for math, demonstrating downstream improvement through diversity
VS-CoT negative synthetic data achieves 36.81% accuracy on GSM8K test set after offline RL, vs. 34.12% with positive-only trainingfinding0.753
Demonstrates VS-generated diverse negative examples improve downstream model performance in offline RL
VS-CoT with GPT-4.1 as generator achieves 45.9% accuracy on Qwen3-1.7B-Base fine-tuned on math benchmarks, the highest single resultfinding0.750
Best performing VS variant for math synthetic data generation with GPT-4.1
Model organism accuracy on BigCodeBench: 56.8% (pre-fine-tuned: 59.1%); steering toward deployment: 55.2%; steering toward evaluation: 43.1%finding0.736
Quantifies performance cost of fine-tuning and steering; deployment steering has minimal accuracy cost.