SCoRE: Spatial Reasoning in Large Vision-Language Models via Egocentric-to-Allocentric Transformation

motivation

Same scene, different views, different answers

An LVLM can answer a spatial question correctly from one view of a scene and wrongly from another. Asked where the road is when sitting on the bench, Qwen2.5-VL answers front, left and behind for three views of the same park.

Existing training supervises only the final answer, so models learn view-specific shortcuts. People instead transform what they see (egocentric) into a frame tied to the scene (allocentric), where the answer no longer depends on the viewpoint. SCoRE trains an LVLM to do this transformation explicitly.

A park with a bench and a road seen from three cameras. Qwen answers front, left and behind for the three views; SCoRE answers front for all. Below: egocentric relations per view, then decentering, mental rotation and reference frame coordination leading to the answer front.

abstract

Reasoning in a shared, scene-centered frame

Spatial reasoning in large vision-language models involves inferring view-specific spatial relations and integrating them into a consistent scene-centered representation. However, existing approaches often learn view-dependent relations tied to individual camera frames, which limits their ability to form viewpoint-invariant spatial representations. Motivated by cognitive models of spatial perspective-taking, we propose SCoRE (Spatial Consistency Reasoning), a method that realizes multi-view spatial reasoning with egocentric-to-allocentric transformation, which first trains the model to infer egocentric object relations from single-view observations, and then applies reinforcement learning to encourage the model to estimate viewpoint orientation, transform egocentric relations into a shared scene-centered reference frame, and predict allocentric object relations. To enforce viewpoint-invariant reasoning, we optimize the model with a multi-view consistency reward that aligns predictions from different views in the common allocentric space. Experiments show that SCoRE improves spatial reasoning performance by +3.69%p on multi-view benchmarks and +4.72%p on general spatial reasoning benchmarks over the state-of-the-art methods, demonstrating the effectiveness of explicit reference-frame transformation for robust multi-view spatial understanding.

method

From what each camera sees to what holds for the scene

SCoRE writes its reasoning in four parts: the egocentric observation r^ego, the facing direction of the reference subject r^head, the transformation r^trans, and the allocentric conclusion r^allo.

QWhen I’m sitting on the bench, where is the road relative to me?

View 1 of the park, bench on the right and road on the left.

bench: right
road: left

Qwenfront ✓

SCoREfront ✓

View 2 of the park, bench on the right and road on the left.

bench: right
road: left

Qwenleft ✗

SCoREfront ✓

View 3 of the park, bench in front and road behind.

bench: front
road: behind

Qwenbehind ✗

SCoREfront ✓

01Decentering

In view 1, the bench faces left.

02Mental rotation

The road is on the camera’s left, which is in front of the bench.

03Reference-frame coordination

From the bench, the road is in front.

V1, V2, V3 → front R_cons: every view agrees with the answer

Example from Figure 1 of the paper.

Why train it, not prompt it

Prompting five LVLMs to reason from egocentric to allocentric view helps on the multi-view data, but the gains are uneven across models and sensitive to how the prompt is written. SCoRE therefore builds the transformation into training.

Accuracy of five LVLMs on multi-view data with and without egocentric-to-allocentric chain-of-thought prompting.

Stage 1: egocentric spatial grounding

Supervised fine-tuning on 4,240 single-view questions such as “From the camera’s perspective, where is the chair relative to the black thing?” The model answers with one of eight directions. No reasoning trace is needed at this stage.

Stage 2: perspective-taking reasoning

Transformation fine-tuning on 788 four-view samples teaches the trace format: for each view, where the target appears, which way the subject faces, and the transformed relation, then one final answer. Traces are generated automatically from SCOPE scene metadata and camera poses.

GRPO then trains on 5,516 samples with 2, 4, 6 or 8 views and G = 4 rollouts, without a reference trace.

Consistency reward

R = 0.3·R_acc + 0.6·R_cons + 0.1·R_fmt

From each rollout, SCoRE reads the conclusion after “Therefore” in every view block, giving the set of view-wise answers 𝒱.

R_cons = R_match − 0.3·R_uniq
R_match = share of views that match the answer
R_uniq = (distinct answers − 1) / |𝒱|

A rollout with no view conclusions gets −0.5. R_acc is +1 or −1 for the final answer, and R_fmt gives 0.25 for each required part of the trace. Consistency has the largest weight because agreement across views is the goal.

results

More accurate and more consistent across views

Qwen2.5-VL-7B trained with SCoRE, compared with GRPO-based spatial reasoning models: VILASR, SpatialReasoner, SpatialThinker and Spatial-SSRL. In-domain: MindCube, MMSI-Bench, ViewSpatial-Bench. Out-of-domain: What’sUp, SPAR-Bench, BLINK.

+3.69

multi-view accuracy over the best prior model, averaged per benchmark

+4.72

general spatial accuracy over the best prior model

34.0

global consistency on MindCube-Hard, up from 23.0

Multi-view spatial reasoning

	MindCube				MMSI-Bench
Model	Rotation	Among	Around	Avg.	Positional	Attribute	Motion	MSR	Avg.	Avg.	Δ
Qwen2.5-VL-7B	31.00	30.17	27.25	29.47	28.87	25.53	20.61	19.70	25.40	27.44	–
Baselines
VILASR-7B	33.00	28.83	25.00	28.94	33.36	28.60	18.62	24.75	28.80	28.87	+1.43
SpatialReasoner	32.50	38.67	29.25	34.50	20.87	27.82	19.33	24.75	22.20	28.35	+0.91
SpatialThinker	35.00	30.50	25.25	30.25	28.37	27.75	26.57	26.26	28.14	29.20	+1.76
Spatial-SSRL	39.50	29.00	25.75	31.42	28.22	21.74	25.30	27.78	26.43	28.93	+1.49
Ours
SCoRE (w/o RL)	34.00	39.33	42.50	38.61	28.62	23.91	19.35	24.75	25.80	32.21	+4.77
SCoRE (full)	34.50	40.83	42.50	39.28	29.06	27.65	26.64	39.39	30.50	34.89	+7.45

QA accuracy (%). Bold: best; underlined: second best. Δ: gain over Qwen2.5-VL-7B.

SCoRE has the best average on both benchmarks. Its gains are largest on tasks that need a consistent layout across views: Among and Around on MindCube and multi-step reasoning (MSR) on MMSI-Bench.

Egocentric vs. allocentric questions

On a 1,000-question subset of ViewSpatial-Bench, SCoRE reaches 42.65%, 4.60 points above Spatial-SSRL. The gain comes mostly from allocentric questions: 46.31%, 11.14 points above the base model, against 2.83 points on egocentric questions.

ViewSpatial-Bench accuracy for egocentric, allocentric and average settings; SCoRE is highest at 37.15, 46.31 and 42.65.

Beyond multi-view data

Training on multi-view consistency also helps single-image and general spatial tasks. SCoRE reaches 98.00% on What’sUp and raises the average over three out-of-domain benchmarks from 64.32% to 68.57%.

Model	What’sUp	SPAR	BLINK	Avg.	Δ
Qwen2.5-VL-7B	90.10	33.85	69.00	64.32	–
SpatialReasoner	84.20	33.21	49.34	55.58	−8.74
SpatialThinker	92.30	34.04	65.20	63.85	−0.47
SCoRE (w/o RL)	97.60	34.10	71.75	67.82	+3.50
SCoRE (full)	98.00	35.47	72.25	68.57	+4.25

MindCube categories

Across the 11 MindCube subcategories, the biggest gains are on agent–object relations (A-O: 44.73% against 23.65% for the base model) and on viewpoint sequences (50.17% against 24.42%). Both require relating objects to an agent and updating those relations as the view changes.

Accuracy of SCoRE, the base model and baselines across 11 MindCube subcategories grouped into object arrangement, perspective taking, relation pattern and viewpoint dynamics.

Relational consistency

We turn MindCube-Hard into relational questions and check whether all predicted pairwise relations form one coherent layout. SCoRE is best on every metric, and global consistency, the share of samples where every relation is right, rises by 11 points.

Model	Relation acc.	Pair cov.	Global cons.
Qwen2.5-VL-7B	48.50	45.93	23.00
SpatialReasoner	36.74	40.33	19.00
SpatialThinker	50.67	49.40	27.00
Spatial-SSRL	51.23	46.37	32.00
SCoRE	52.47	50.33	34.00

An office seen from three views, and the object layouts reconstructed by each model; SCoRE's layout of the blue bin, monitors and black sofa matches the ground truth, while baselines distort it.

Ablations

Training	MindCube	MMSI	Avg.
Single stage
Perception SFT only	34.03	22.90	28.47
RL only	30.25	26.23	28.24
Transformation SFT only	31.09	27.84	29.47
Remove one stage
w/o perception SFT	36.87	31.32	34.10
w/o transformation SFT	35.76	30.49	33.13
w/o RL	38.61	25.80	32.21
SCoRE (full)	39.28	30.50	34.89

No single stage comes close to the full pipeline. Removing RL costs the most, and removing transformation SFT costs more than removing perception SFT.

Reward	Acc.	Cons.	Fmt.	MindCube	MMSI
w/o consistency	✓		✓	37.67	25.90
w/o accuracy		✓	✓	38.83	26.90
Full	✓	✓	✓	39.28	30.50

Dropping the consistency reward hurts more than dropping the accuracy reward, especially on MMSI-Bench (−4.60 points). Correct answers alone do not teach the model to agree with itself across views.

Checking each reasoning step

Step	Accuracy (%)
Egocentric relation (r^ego)	84.5
Heading (r^head)	81.2
Transformation (r^trans)	98.9
Allocentric relation (r^allo)	68.1

Given its own egocentric relation and heading, the model almost always composes the right transformed relation (98.9%). The gains come from the transformation itself, not from shortcuts to the final answer.

Without viewpoint labels

Model	No V&O	With V&O	Noisy V&O
Qwen2.5-VL-7B	24.00	25.00	25.33
SpatialReasoner	32.33	29.33	30.67
SpatialThinker	26.33	26.00	28.00
VILASR-7B	22.00	22.00	20.00
Spatial-SSRL	29.00	29.67	30.00
SCoRE	39.67	35.33	34.33

On MindCube, SCoRE stays best whether viewpoint and orientation (V&O) labels are given, removed or corrupted. It does best with no labels at all, so it does not depend on them.

Case study

Asked where the toilet is while using the handwashing area, SCoRE takes the perspective of the person at the sink, links the towel across both images, and answers to my left. Qwen reads the local cues correctly but stays in the camera’s frame and answers to my right.

Two bathroom images with a towel, sink and toilet. SCoRE reasons from the person's facing direction and answers that the toilet is to the left; Qwen answers right.

cite

BibTeX

@inproceedings{kim2026score,
  title     = {Spatial Reasoning in Large Vision-Language Models via
               Egocentric-to-Allocentric Transformation},
  author    = {Kim, Yoonji and Kim, Jieun and Cho, Sung-Bae},
  booktitle = {Findings of the Association for Computational Linguistics:
               EMNLP 2026},
  year      = {2026}
}