# arXiv cs.AI: EngineAgent for On-Device LLM Inference Engines

[Canonical reading](https://escapemeridian.com/readings/arxiv-cs-ai-engineagent-for-on-device-llm-inference-engines-10ed1483)

The authors present EngineAgent, a system for helping coding agents adapt on-device LLM inference engines to new models. They argue that this work is difficult because each model may require coordinated changes across conversion, graph construction, operators and hardware backends. Existing agents can miss cross-layer dependencies and struggle to diagnose host--device failures.

EngineAgent builds a hierarchy-aware engine index of layers and dependencies, then pre-validates backend-specific workflows using already supported models. For a new model, a builder coordinates changes while an auditor checks completeness and drives debugging through device-side validation. The approach separates reusable engine and hardware knowledge from model-specific implementation.

Across 12 tasks spanning three mobile LLMs and four engine--backend pairs on CPUs, GPUs and NPUs, the authors report 75% end-to-end success with either of two underlying LLMs, versus 33.3% for the strongest coding-agent baseline. Success required compilation, on-device execution and output-logit agreement with reference implementations. The authors also report reduced average task time and model-API cost compared with baselines using the same underlying LLMs.

Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.

## Original source

arXiv cs.AI · Article

Original source date: Nov 10, 2026 (2026-11-10).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-16T21:47:57.583Z. First observed: 2026-11-11T05:06:29.124Z. Retrieved: 2026-11-16T21:46:57.294Z.

- [EngineAgent: Agentizing On-Device LLM Inference Engines with Inference Stack Hierarchy Awareness and Hardware Abilities](https://arxiv.org/abs/2611.14356)

## Limits

The reported evaluation covers 12 tasks, three mobile LLMs and four engine–backend pairs; the abstract does not establish performance beyond these tested configurations. The abstract reports lower average task time and model-API cost but does not give the size of either reduction or task-level results.


## Open question

How would EngineAgent perform on additional models and engine–backend pairs beyond the 12 tasks described?


## Source-stated

The authors say adapting inference engines to new on-device LLMs is difficult because it can require coordinated changes across multiple system layers, and existing coding agents can miss dependencies and struggle with host–device debugging.

> Each model may require coordinated changes across conversion, graph construction, operators, and hardware backends. Existing coding agents often miss these cross-layer dependencies and struggle to diagnose failures across host--device execution paths.

- [Supporting source](https://arxiv.org/abs/2611.14356)

## Source-stated

EngineAgent builds an index of engine layers and dependencies, pre-validates backend workflows with supported models, and uses a builder and auditor to implement and check support for each new model.

> Before enablement, EngineAgent constructs a hierarchy-aware engine index that captures engine abstraction layers and cross-layer dependencies, and pre-validates backend-specific build, execution, and debugging workflows using already supported models.

- [Supporting source](https://arxiv.org/abs/2611.14356)

## Source-stated

Across the reported 12 tasks, the authors report 75% end-to-end success for EngineAgent with either of two underlying LLMs, compared with 33.3% for the strongest coding-agent baseline.

> Across 12 tasks spanning three mobile LLMs and four engine--backend pairs on CPUs, GPUs, and NPUs, EngineAgent achieves 75% end-to-end success with either of two underlying LLMs, versus 33.3% for the strongest coding-agent baseline.

- [Supporting source](https://arxiv.org/abs/2611.14356)

## Source-stated

The authors define success as compilation, on-device execution and output-logit agreement with reference implementations, and report reduced average task time and model-API cost versus baselines using the same underlying LLMs.

> Success requires compilation, on-device execution, and output-logit agreement with reference implementations. EngineAgent also reduces average per-task time and model-API cost relative to baselines using the same underlying LLMs.

- [Supporting source](https://arxiv.org/abs/2611.14356)

## Source and contributor context

Huang, Zhengxiang, Sun, Yujie, Chen, Shengheng, Wang, Zhaode, Niu, Chaoyue, Zhao, Zeyu, Lv, Chengfei, Wu, Fan, Chen, Guihai · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.


## Source media

No suitable media was found by the supported page extraction.


## Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.


## Large language model

A language model with many learned numerical settings, used to process and generate language.

- [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model)

## CPU

The central processing unit: a general-purpose processor that carries out a computer's instructions.

- [Definition reference](https://aws.amazon.com/what-is/cpu/)

## Graphics processing unit

A chip that performs many calculations at once; the main hardware used to train and run AI models.

- [Definition reference](https://en.wikipedia.org/wiki/Graphics_processing_unit)

## API

An application programming interface: a defined way for software to request data or actions from another system.

- [Definition reference](https://developer.mozilla.org/en-US/docs/Glossary/API)

## arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

- [Explore this context](/entities/arxiv)
- [Reference](https://arxiv.org/)
- [Reference](https://info.arxiv.org/help/api/index.html)

## TIME

TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.

- [Explore this context](/entities/time)
- [Reference](https://time.com/)

## Editorial provenance

Version 1. Publication ID: bfd597c1-de19-4133-8663-a03180270fbb.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-16T21:47:57.583Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material\_development.

Observation: 49766a42bd2dca9c2a2a748d951e3d1e587fa1d9ee582dea3f6b4de21114bee0. Source text hash: 5bfc159be93433710e95c727b95f4054101a6a4fb09e8fa18bad500c66ea0e93.


## Revision and source-check history

### Version 1 — 2026-11-16T21:47:57.583Z

The authors present EngineAgent, a system for helping coding agents adapt on-device LLM inference engines to new models. They argue that this work is difficult because each model may require coordinated changes across conversion, graph construction, operators and hardware backends. Existing agents can miss cross-layer dependencies and struggle to diagnose host--device failures.

EngineAgent builds a hierarchy-aware engine index of layers and dependencies, then pre-validates backend-specific workflows using already supported models. For a new model, a builder coordinates changes while an auditor checks completeness and drives debugging through device-side validation. The approach separates reusable engine and hardware knowledge from model-specific implementation.

Across 12 tasks spanning three mobile LLMs and four engine--backend pairs on CPUs, GPUs and NPUs, the authors report 75% end-to-end success with either of two underlying LLMs, versus 33.3% for the strongest coding-agent baseline. Success required compilation, on-device execution and output-logit agreement with reference implementations. The authors also report reduced average task time and model-API cost compared with baselines using the same underlying LLMs.

## Original source

arXiv cs.AI · Article

Original source date: Nov 10, 2026 (2026-11-10).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-16T21:47:57.583Z. First observed: 2026-11-11T05:06:29.124Z. Retrieved: 2026-11-16T21:46:57.294Z.

- [EngineAgent: Agentizing On-Device LLM Inference Engines with Inference Stack Hierarchy Awareness and Hardware Abilities](https://arxiv.org/abs/2611.14356)

## Limits

The reported evaluation covers 12 tasks, three mobile LLMs and four engine–backend pairs; the abstract does not establish performance beyond these tested configurations. The abstract reports lower average task time and model-API cost but does not give the size of either reduction or task-level results.


## Open question

How would EngineAgent perform on additional models and engine–backend pairs beyond the 12 tasks described?


## Source-stated

The authors say adapting inference engines to new on-device LLMs is difficult because it can require coordinated changes across multiple system layers, and existing coding agents can miss dependencies and struggle with host–device debugging.

> Each model may require coordinated changes across conversion, graph construction, operators, and hardware backends. Existing coding agents often miss these cross-layer dependencies and struggle to diagnose failures across host--device execution paths.

- [Supporting source](https://arxiv.org/abs/2611.14356)

## Source-stated

EngineAgent builds an index of engine layers and dependencies, pre-validates backend workflows with supported models, and uses a builder and auditor to implement and check support for each new model.

> Before enablement, EngineAgent constructs a hierarchy-aware engine index that captures engine abstraction layers and cross-layer dependencies, and pre-validates backend-specific build, execution, and debugging workflows using already supported models.

- [Supporting source](https://arxiv.org/abs/2611.14356)

## Source-stated

Across the reported 12 tasks, the authors report 75% end-to-end success for EngineAgent with either of two underlying LLMs, compared with 33.3% for the strongest coding-agent baseline.

> Across 12 tasks spanning three mobile LLMs and four engine--backend pairs on CPUs, GPUs, and NPUs, EngineAgent achieves 75% end-to-end success with either of two underlying LLMs, versus 33.3% for the strongest coding-agent baseline.

- [Supporting source](https://arxiv.org/abs/2611.14356)

## Source-stated

The authors define success as compilation, on-device execution and output-logit agreement with reference implementations, and report reduced average task time and model-API cost versus baselines using the same underlying LLMs.

> Success requires compilation, on-device execution, and output-logit agreement with reference implementations. EngineAgent also reduces average per-task time and model-API cost relative to baselines using the same underlying LLMs.

- [Supporting source](https://arxiv.org/abs/2611.14356)

## Source and contributor context

Huang, Zhengxiang, Sun, Yujie, Chen, Shengheng, Wang, Zhaode, Niu, Chaoyue, Zhao, Zeyu, Lv, Chengfei, Wu, Fan, Chen, Guihai · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.


## Source media

No suitable media was found by the supported page extraction.


## Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.


## Large language model

A language model with many learned numerical settings, used to process and generate language.

- [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model)

## CPU

The central processing unit: a general-purpose processor that carries out a computer's instructions.

- [Definition reference](https://aws.amazon.com/what-is/cpu/)

## Graphics processing unit

A chip that performs many calculations at once; the main hardware used to train and run AI models.

- [Definition reference](https://en.wikipedia.org/wiki/Graphics_processing_unit)

## API

An application programming interface: a defined way for software to request data or actions from another system.

- [Definition reference](https://developer.mozilla.org/en-US/docs/Glossary/API)

## arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

- [Explore this context](/entities/arxiv)
- [Reference](https://arxiv.org/)
- [Reference](https://info.arxiv.org/help/api/index.html)

## TIME

TIME is a news magazine publishing reporting on world affairs, business, technology and the workers behind digital services.

- [Explore this context](/entities/time)
- [Reference](https://time.com/)

## Editorial provenance

Version 1. Publication ID: bfd597c1-de19-4133-8663-a03180270fbb.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-16T21:47:57.583Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material\_development.

Observation: 49766a42bd2dca9c2a2a748d951e3d1e587fa1d9ee582dea3f6b4de21114bee0. Source text hash: 5bfc159be93433710e95c727b95f4054101a6a4fb09e8fa18bad500c66ea0e93.


### uncertain — 2026-11-16T21:47:57.583Z — anthropic/claude-opus-5.5

Publication: bfd597c1-de19-4133-8663-a03180270fbb

attribution — pass: The reading attributes the system, argument and results to the paper's authors throughout, and the arXiv publisher and title match the evidence. Results are framed as author-reported, not independently verified.

> We present EngineAgent, an agentic system for automating on-device LLM model enablement.

summary — uncertain: The opening calls EngineAgent a system for helping coding agents adapt engines. The source describes it as an agentic system that automates model enablement and compares it against coding-agent baselines. Smallest repair: call it an agentic system that automates adding new models to on-device inference engines. Central terms are also unexplained for a general reader, including on-device inference engine, operators, hardware backends, NPUs, host--device and output-logit agreement. The reading also never says plainly why faster model support on phones matters. Add these terms to the tooltip library or gloss them briefly.

> We present EngineAgent, an agentic system for automating on-device LLM model enablement.

date — pass: originalPublished 2026-11-10 matches the v1 submission date shown on the arXiv page. The 2026-11-11 feed date is correctly not used.

> \[Submitted on 10 Nov 2026\]

support — pass: Each claim is backed by an exact abstract excerpt. The figures (12 tasks, 75% vs 33.3%) and the success criteria are preserved. Limitations correctly note that the size of the time and cost reductions is not given. The question about generalization beyond the tested configurations follows from the evidence and is not leading.

> EngineAgent achieves 75% end-to-end success with either of two underlying LLMs, versus 33.3% for the strongest coding-agent baseline.

media — pass: The selected media is the verified direct arXiv abstract page for this paper, which is the most direct access to the source.

## Recorded references

Explicit citations in public readings to this reading. Shared topics are not citations. Other record types need an explicit reference adapter.

No public references recorded here yet.

[Reference data (JSON)](/readings/10ed1483dce84e8e64a6a4d2392017793086a6f664fdf446529494660e8d8569/references.json)

## Corrections and challenges

[Send a correction or an opposing source](https://escapemeridian.com/corrections/)

[JSON record](https://escapemeridian.com/readings/arxiv-cs-ai-engineagent-for-on-device-llm-inference-engines-10ed1483.json)
