Skip to content
Escape
Meridian

The future is still open.

Duan, Peitong, Gatmiry, Khashayar, Nichols, Jeffrey, Bengio, Samy, Abbe, Emmanuel · Oct 29, 2026

arXiv cs.AI: Online Dense Reinforcement Learning for UI Navigation

Markdown · JSON · Original source

Peitong Duan and coauthors propose training UI navigation agents directly on live iOS applications. They argue this could address two shortcomings: frontier vision-language models have high , while smaller models trained on static trajectories can fail to recover from errors. Their framework separates planning from UI grounding and uses an -as-a-Judge for step-level feedback and a final task-completion reward. The authors combine these signals with Proximal Policy Optimization (PPO) and introduce GiGPPO, which they say improves critic stability by using reward signals from visually similar states. The authors report that their agents outperform supervised (SFT) and terminal-only baselines on UINavBench, iOSWorld, and AndroidWorld. They report nearly a sevenfold improvement over the SFT baseline in generalization to Android environments for their 8B agents, and say the agents outperform Gemini-3.5-Flash and tie with Gemini-3.1-Pro on real iOS applications. These are reported evaluation results; the abstract does not establish performance in everyday use.

Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.

arXiv cs.AIArticle · arxiv.orgOpen original source

Original source

cs.AI · Article

Source details

Original source date: Oct 29, 2026 (2026-10-29).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-13T07:17:53.149Z. First observed: 2026-10-30T06:45:38.292Z. Retrieved: 2026-10-31T10:33:02.566Z.

Referenced by

Recorded citations and public file attachments. A reference alone does not establish agreement or influence.

No public references recorded here yet.

Reference data (JSON)

Limits

The evidence is the abstract; it does not include scores, experimental details, or enough information to assess how the reported results vary across tasks. The abstract reports evaluations on benchmarks and real iOS applications but does not establish performance in everyday use. The abstract identifies high as a problem for frontier vision-language models but does not quantify those costs or compare them with the proposed agents.

Open question

How do the reported gains hold up across different tasks and live apps beyond the evaluations described in the abstract?

Source-stated

The authors propose an online reinforcement-learning framework that trains UI navigation agents directly on live iOS applications.

we propose a novel online reinforcement learning framework that trains UI navigation agents directly on live iOS applications.

Source-stated

The framework separates planning and UI grounding, and uses an LLM-as-a-Judge for step-level and terminal reward signals.

We rely on an LLM-as-a-Judge to provide comprehensive reward signals, combining dense, step-level feedback to evaluate intermediate progress with a terminal reward for overall task completion.

Source-stated

The authors introduce GiGPPO, which they say improves critic stability by using reward signals from visually similar states across concurrent trajectories.

we introduce GiGPPO, a novel neighbor-conditioned value estimation method that improves critic stability by leveraging reward signals from visually similar states across concurrent trajectories.

Source-stated

The authors report that their agents outperform SFT and terminal-only RL baselines on UINavBench, iOSWorld, and AndroidWorld.

Evaluations on rigorous benchmarks, including UINavBench, iOSWorld, and AndroidWorld, demonstrate that our online RL trained agents substantially outperform both SFT and terminal-only RL baselines

Source-stated

The authors report nearly a sevenfold improvement over the SFT baseline for 8B agents in out-of-distribution generalization to Android environments.

our 8B RL agents achieve nearly a sevenfold improvement over the SFT baseline in out-of-distribution generalization to Android environments

Source-stated

The authors say their agents outperform Gemini-3.5-Flash and tie with Gemini-3.1-Pro on real iOS applications.

while outperforming Gemini-3.5-Flash and tying with Gemini-3.1-Pro on real iOS applications.

Source and contributor context

Duan, Peitong, Gatmiry, Khashayar, Nichols, Jeffrey, Bengio, Samy, Abbe, Emmanuel · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.

Source media

No suitable media was found by the supported page extraction.

Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.

Reinforcement learning

Training by trial and error, in which a system is rewarded for the outcomes its designers want.

Model inference

Using a trained model to produce a prediction or response to new input.

Large language model

A language model with many learned numerical settings, used to process and generate language.

Fine-tuning

Further training of an existing model to adapt it to a particular task or use.

Distribution shift

A change between the data a model was trained on and the data it meets in use, which can lower its accuracy.

Benchmark

A defined set of tests used to compare systems. Results depend on the tasks and measures selected.

arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

Editorial provenance

Version 1. Publication ID: d32e94ab-c6f0-409c-8692-9956502681dc.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-13T07:17:53.149Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material_development.

Observation: 8cef9760f7d494143cf9db245ca39d0d4179db4f4271611824e51cd6f0f0844f. Source text hash: ca3aed14080904e2a6b2b5c0333de9cdfa714bb309618f7edcbc488b81db9ba7.

Revision and source-check history

Version 1 · 2026-11-13T07:17:53.149Z

arXiv cs.AI: Online Dense Reinforcement Learning for UI Navigation

Peitong Duan and coauthors propose training UI navigation agents directly on live iOS applications. They argue this could address two shortcomings: frontier vision-language models have high , while smaller models trained on static trajectories can fail to recover from errors. Their framework separates planning from UI grounding and uses an -as-a-Judge for step-level feedback and a final task-completion reward. The authors combine these signals with Proximal Policy Optimization (PPO) and introduce GiGPPO, which they say improves critic stability by using reward signals from visually similar states. The authors report that their agents outperform supervised (SFT) and terminal-only baselines on UINavBench, iOSWorld, and AndroidWorld. They report nearly a sevenfold improvement over the SFT baseline in generalization to Android environments for their 8B agents, and say the agents outperform Gemini-3.5-Flash and tie with Gemini-3.1-Pro on real iOS applications. These are reported evaluation results; the abstract does not establish performance in everyday use.

Original source

cs.AI · Article

Source details

Original source date: Oct 29, 2026 (2026-10-29).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-13T07:17:53.149Z. First observed: 2026-10-30T06:45:38.292Z. Retrieved: 2026-10-31T10:33:02.566Z.

Limits

The evidence is the abstract; it does not include scores, experimental details, or enough information to assess how the reported results vary across tasks. The abstract reports evaluations on benchmarks and real iOS applications but does not establish performance in everyday use. The abstract identifies high as a problem for frontier vision-language models but does not quantify those costs or compare them with the proposed agents.

Open question

How do the reported gains hold up across different tasks and live apps beyond the evaluations described in the abstract?

Source-stated

The authors propose an online reinforcement-learning framework that trains UI navigation agents directly on live iOS applications.

we propose a novel online reinforcement learning framework that trains UI navigation agents directly on live iOS applications.

Source-stated

The framework separates planning and UI grounding, and uses an LLM-as-a-Judge for step-level and terminal reward signals.

We rely on an LLM-as-a-Judge to provide comprehensive reward signals, combining dense, step-level feedback to evaluate intermediate progress with a terminal reward for overall task completion.

Source-stated

The authors introduce GiGPPO, which they say improves critic stability by using reward signals from visually similar states across concurrent trajectories.

we introduce GiGPPO, a novel neighbor-conditioned value estimation method that improves critic stability by leveraging reward signals from visually similar states across concurrent trajectories.

Source-stated

The authors report that their agents outperform SFT and terminal-only RL baselines on UINavBench, iOSWorld, and AndroidWorld.

Evaluations on rigorous benchmarks, including UINavBench, iOSWorld, and AndroidWorld, demonstrate that our online RL trained agents substantially outperform both SFT and terminal-only RL baselines

Source-stated

The authors report nearly a sevenfold improvement over the SFT baseline for 8B agents in out-of-distribution generalization to Android environments.

our 8B RL agents achieve nearly a sevenfold improvement over the SFT baseline in out-of-distribution generalization to Android environments

Source-stated

The authors say their agents outperform Gemini-3.5-Flash and tie with Gemini-3.1-Pro on real iOS applications.

while outperforming Gemini-3.5-Flash and tying with Gemini-3.1-Pro on real iOS applications.

Source and contributor context

Duan, Peitong, Gatmiry, Khashayar, Nichols, Jeffrey, Bengio, Samy, Abbe, Emmanuel · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.

Source media

No suitable media was found by the supported page extraction.

Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.

Reinforcement learning

Training by trial and error, in which a system is rewarded for the outcomes its designers want.

Model inference

Using a trained model to produce a prediction or response to new input.

Large language model

A language model with many learned numerical settings, used to process and generate language.

Fine-tuning

Further training of an existing model to adapt it to a particular task or use.

Distribution shift

A change between the data a model was trained on and the data it meets in use, which can lower its accuracy.

Benchmark

A defined set of tests used to compare systems. Results depend on the tasks and measures selected.

arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

Editorial provenance

Version 1. Publication ID: d32e94ab-c6f0-409c-8692-9956502681dc.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-13T07:17:53.149Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material_development.

Observation: 8cef9760f7d494143cf9db245ca39d0d4179db4f4271611824e51cd6f0f0844f. Source text hash: ca3aed14080904e2a6b2b5c0333de9cdfa714bb309618f7edcbc488b81db9ba7.

uncertain · 2026-11-13T07:17:53.149Z · anthropic/claude-opus-5.5

Publication: d32e94ab-c6f0-409c-8692-9956502681dc

attribution — pass: The draft correctly credits Peitong Duan and coauthors, and the listed authors match the page. Results and claims are consistently framed as what the authors propose, report or say, including the GiGPPO stability claim and the Gemini comparisons.

Authors:Peitong Duan, Khashayar Gatmiry, Jeffrey Nichols, Samy Bengio, Emmanuel Abbe

summary — uncertain: The draft is faithful, but a non-specialist may miss the central stakes. It never plainly says that UI navigation agents are AI systems that operate app screens to complete tasks. It also does not tie the 8B size, meaning a smaller model, back to the authors' cost motivation, so the Gemini comparison is hard to read as the point of the paper. Smallest repair: add a plain gloss of UI navigation agent, say the 8B agents are much smaller models, and mention the hundreds of concurrent virtual iOS devices. Terms like UI grounding, LLM-as-a-Judge, PPO and critic stability need term-library tooltips.

While frontier vision-language models demonstrate strong capabilities in user interface (UI) navigation, their high inference costs prohibit everyday use.

date — pass: The draft's originalPublished date, 2026-10-29, matches the arXiv v1 submission date. The later feed date is correctly not used.

[Submitted on 29 Oct 2026]

support — pass: Every claim matches the abstract. The limitations and the question correctly avoid treating benchmark results or the cost motivation as demonstrated everyday benefit. The draft adds no unsupported benefits or costs.

our 8B RL agents achieve nearly a sevenfold improvement over the SFT baseline in out-of-distribution generalization to Android environments

media — pass: The selected media is the verified direct arXiv abstract page for this paper, which is the most direct access point to the source.

Challenge or correct this reading

Send a correction or an opposing source with this reading’s URL. Disagreement and uncertainty can remain unresolved.