Skip to content
See the World Through ScienceA project of ALLATRA
Source: PreprintarXiv3 sources

A Navigation Test That Asks Who Is Walking

By Wilkens EtienneWriterAI & Technology6 min read

Republish this story

Our work is licensed under Creative Commons BY-NC 4.0. You may republish this piece for free — with credit to ALLATRA Media and a link to the original, unedited beyond length trims, and not for commercial use.

Read the full license

A woman pushes a wheelchair along the edge of a street corner in Taipei because parked scooters fill the sidewalk, photographed Dec. 1, 2013.
Parked scooters leave no clear path on Yanshou Street in Taipei, so the wheelchairs travel by way of the roadway. Photographed Dec. 1, 2013 (illustrative)."Two Foreign Care-Givers Pushing wheelchairs in Yanshou Street Sidewalk 20131201" by 玄史生, via Wikimedia Commons, CC BY-SA 3.0, cropped

Put a low temporary barrier across a sidewalk and you have created three different problems. A robot dog walks around it. A blind pedestrian needs to be told about it, in words that don’t rely on sight. Someone in a wheelchair may have to turn back. The barrier is the same object each time. Whether the route can be used, depends on who is traveling it.

That is the premise of VLAN, or Vision-Language Accessible Navigation, built by Jie Hu, Jiaming Zhang and four colleagues at Hunan University and posted to arXiv on Nov. 13, 2026. It is a preprint in which the authors note on the posting that it has been accepted to Othe NeurIPS 2026 Evaluations and Datasets Track, and no proceedings listing has appeared yet. Its test set is rebuilt from walk-through video, and every route through it is labeled five separate times. Once for a humanoid robot, once for a quadruped, once for a drone, once for a wheelchair user and once for a blind or visually impaired pedestrian, BVI in the paper's shorthand.

The scenes are not clips. Each is reconstructed as a Gaussian splat, a photorealistic 3D copy, then imported into the Unity game engine, where things can be added: barricades, cones, a curb, a hanging branch, a traffic light switched to red. The editing is the method. One reconstructed street becomes several navigation problems, and the obstacles went in precisely so the same geometry would mean different things to different travelers. Each route then carries a verdict per user: Passable, Bypass, meaning a detour works, HighRisk, meaning physically possible but unsafe, or Blocked.

A score only counts when everything is right

The models are not graded on getting close. The drone and the quadruped get a few still frames of the route and a target, and must produce the rest as small steps; a run counts only if it reaches the goal area without a collision. The other three travelers get their guidance in words, and it counts only if every constraint that user requires is satisfied at once. Guidance for a blind pedestrian has to measure in steps rather than meters, give clock directions rather than left and right, state the risk out loud, and decline to send anyone down a blocked route. Miss one and the whole route scores zero, so every figure below is a fraction of routes passed, not a collision rate.

Jie Hu and colleagues ran seven open-weight multimodal models, none larger than 11 billion parameters: Qwen2.5-VL-3B-Instruct, Qwen2.5-VL-7B-Instruct, Qwen3-VL-8B-Instruct, InternVL3.5-8B, Idefics3-8B-Llama3, Molmo2-8B and Llama-3.2-11B-Vision-Instruct. On their balanced overall score, AccSR, which is built to punish an uneven profile, the best was InternVL3.5-8B at 0.426, and the weakest Qwen2.5-VL-3B-Instruct at 0.147. The spread inside a single model is sharper still. Llama-3.2-11B-Vision-Instruct wrote the best humanoid guidance of the seven, 0.558, and still finished near the bottom overall. On wheelchair guidance, InternVL3.5-8B led at 0.347, and Idefics3-8B-Llama3 met the full set of required constraints on 0.004 of routes.

A gently sloping concrete walkway runs down to a street corner, with a textured pad set into the pavement at the bottom, photographed Oct. 11, 2025.
One slope, one lowered edge and one textured pad, built for travelers whose needs do not agree. Photographed Oct. 11, 2025 (illustrative). "Long gradual sidewalk ramp to tactile paving and curb cut in Colchester VT" by Ezekielf, via Wikimedia Commons, CC0

The models saw the obstacle and gave the same advice

Because every traveler is shown the identical picture, the authors can measure directly whether a model's answer actually changes with the user. Take the routes a model labels correctly for the humanoid, who is assigned ordinary pedestrian height, and count how many of those it labels wrongly for the blind pedestrian. Where that cell is reported, the failure runs from 2.2% of routes for Qwen3-VL-8B-Instruct to 38.4% for Idefics3-8B-Llama3. The opposite error, right for the blind traveler and wrong for the sighted one, is near zero everywhere. The authors read the asymmetry as diagnostic, writing that "models err by failing to apply BVI-specific constraints rather than by inventing them where they do not apply."

Where in the chain that goes wrong is the next question, and the authors answer that it is mostly not the eyes. Sorting every route by whether the model named the obstacle correctly and whether its predicted motion succeeded, Qwen2.5-VL-3B-Instruct named the obstacle and still failed the motion on 23.3% of routes, the largest such share in the table. Those cells are outcome categories rather than a clean separation of cause, as the paper itself says.

The authors' own summary is that "current vision-language models still struggle to adapt path selection, obstacle relevance, and safety reasoning across different user profiles." That is a benchmark's authors reporting on the field their benchmark tests, from a roster of small open-weight models; they also claim a first for the benchmark. The claim needs narrowing. CapNav, presented at CVPR 2026 by Xia Su and colleagues, had already tested vision-language models in real indoor scenes with several human and robot agents, including a wheelchair user. It found that the models did worse as agents became less mobile. What’s new in VLAN is how it combines the pieces. Its scenes are mostly outdoors and can be edited at scale, and blind and wheelchair travelers each get their own labels and their own scoring.

The labels come from body measurements, not from the people

The labels deserve the closest look, because they’re what make this a benchmark about people. Each traveler was given fixed dimensions, for example a wheelchair’s width and turning radius, the height a robot dog can step over, and the drone’s flying height. If an obstacle got in the way of those dimensions, it counted as a problem for that traveler. The paper argues that you don’t need personal experience to know whether a wheelchair fits through a gap, because you can work it out from the measurements. The paper also admits what this approach misses, noting that “involving participants with lived experience would strengthen the boundary cases between HighRisk and Blocked” — in other words, people who use wheelchairs or are blind could help decide when a route is risky and when it is fully blocked. No participant with lived experience took part in the labeling, and no agreement figure between annotators is reported.

Two more features of the construction shape the numbers. The drone task is two-dimensional by definition, flown at a fixed altitude, so an overhead branch is handled by going around it or stopping; on 71.0% of the drone's test routes the obstacle is irrelevant at that altitude and the route is simply Passable. And every score in the paper comes from one deterministic run with no spread reported, so no gap between two models here should be read as a significant one. The models were tested on only 30 real-world routes, which the authors treat as an early sign that the rankings hold up outside the simulation, not as proof.

The evaluation code and the test data are public under an MIT license, which lets anyone use them freely. The training data and the code for fine-tuning models have not been released yet. And nothing in these results is a product anyone is using. The authors chose general-purpose models on purpose, as a reasonable starting point. They see navigation systems designed for one specific kind of traveler as the natural next step.

Sources

Spot an error?

Spot an error?

Report an error

Spotted a mistake on this page? Tell us what's wrong and our editors will take a look.

What kind of problem?

Only if you'd like us to be able to follow up. We won't use it for anything else.

We correct mistakes openly. Select any text to flag it. Fixes are logged under our Corrections Policy.

Report an error

Reporting on

A Navigation Test That Asks Who Is Walking

What kind of problem?

Only if you'd like us to be able to follow up. We won't use it for anything else.

We read every report. Corrections are logged publicly.