CharXiv-R Benchmark Leaderboard

CharXiv-R is the reasoning component of the CharXiv benchmark, focusing on complex reasoning questions that require synthesizing information across visual chart elements. It evaluates multimodal large language models on their ability to understand and reason about scientific charts from arXiv papers through various reasoning tasks.

Leaderboard

Top 36 models on CharXiv-R Benchmark Leaderboard (scores from public evaluations).

Rank	Model	Score	Lab
1	Claude Mythos Preview	93.2%	—
2	Claude Opus 4.7	91.0%	—
3	Kimi K2.6	86.7%	—
4	Muse Spark	86.4%	—
5	Gemini 3.5 Flash	84.2%	—
6	GPT-5.2	82.1%	—
7	GPT-5.5 Instant	81.6%	—
8	Qwen3.6 Plus	81.5%	—
9	Gemini 3 Pro	81.4%	—
10	GPT-5	81.1%	—
11	Gemini 3 Flash	80.3%	—
12	Qwen3.5-27B	79.5%	—
13	o3	78.6%	—
14	Qwen3.6-27B	78.4%	—
15	Qwen3.6-35B-A3B	78.0%	—
16	Qwen3.5-35B-A3B	77.5%	—
16	Kimi K2.5	77.5%	—
18	Claude Opus 4.6	77.4%	—
19	Qwen3.5-122B-A10B	77.2%	—
20	Gemini 3.1 Flash-Lite	73.2%	—
21	o4-mini	72.0%	—
22	Qwen3 VL 235B A22B Thinking	66.1%	—
23	Qwen3 VL 32B Thinking	65.2%	—
24	Qwen3 VL 32B Instruct	62.8%	—
25	Qwen3 VL 235B A22B Instruct	62.1%	—
26	GPT-4o	58.8%	—
27	GPT-4.1 mini	56.8%	—
28	GPT-4.1	56.7%	—
29	Qwen3 VL 30B A3B Thinking	56.6%	—
30	GPT-4.5	55.4%	—
31	Qwen3 VL 8B Thinking	53.0%	—
32	Qwen3 VL 4B Thinking	50.3%	—
33	Qwen3 VL 30B A3B Instruct	48.9%	—
34	Qwen3 VL 8B Instruct	46.4%	—
35	GPT-4.1 nano	40.5%	—
36	Qwen3 VL 4B Instruct	39.7%	—

Models tracked

Models with charxiv-r in their evaluation profile.

No models linked yet.

View task leaderboards →