# arXiv cs.AI: TileQuant assigns mixed precision to smaller weight tiles [Canonical reading](https://escapemeridian.com/readings/arxiv-cs-ai-tilequant-assigns-mixed-precision-to-smaller-weight-973621c8) Hanzhi Zhang and coauthors introduce TileQuant, a training-free framework for mixed-precision post-training quantization (PTQ) of large language models (LLMs). Instead of keeping entire columns at higher precision, TileQuant scores hardware-aligned 2D weight tiles, reflecting the authors’ argument that sensitivity can vary even within a column. Offline calibration can score tiles using input-activation-aware forward statistics or loss sensitivity; within a retention budget, the highest-scoring tiles stay at 16-bit and the rest use 8-bit. The resulting map guides reconstruction into compact weight buffers, which the kernel reuses for integer and 16-bit computation with local accumulation to reduce memory traffic. The authors report 2.66x the original model’s generation throughput and say quality is preserved across language modeling, reasoning, long-context retrieval, and question answering, with TileQuant surpassing quantized and original models on multiple tasks. The proposed approach links finer-grained precision allocation to hardware execution, rather than treating sensitivity and runtime efficiency as separate concerns. The abstract does not identify the evaluated models, hardware, or task-by-task results, so it does not show how broadly the reported throughput and quality findings apply. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: Nov 4, 2026 (2026-11-04). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-08T03:34:08.684Z. First observed: 2026-11-06T05:12:28.731Z. Retrieved: 2026-11-08T03:33:19.841Z. - [TileQuant: Tile as the Precision-Allocation Unit for LLM Mixed-Precision Post-Training Quantization](https://arxiv.org/abs/2611.07217) ## Limits The abstract does not identify the evaluated models or hardware, or give task-by-task throughput and quality results; the reported results’ breadth cannot be assessed from this evidence alone. ## Open question How do TileQuant’s reported throughput and quality results vary across the evaluated models, hardware, and tasks? ## Source-stated TileQuant assigns precision to hardware-aligned 2D weight tiles, rather than retaining whole columns at higher precision. > We therefore introduce TileQuant, a training-free framework that assigns precision to hardware-aligned 2D weight tiles. - [Supporting source](https://arxiv.org/abs/2611.07217) ## Source-stated TileQuant offers two offline calibration scoring approaches: input-activation-aware scoring from forward statistics and gradient-informed scoring from loss sensitivity. > Offline calibration provides input-activation-aware scoring from forward statistics for lower calibration cost and gradient-informed scoring from loss sensitivity for stronger quality preservation. - [Supporting source](https://arxiv.org/abs/2611.07217) ## Source-stated Within each projection’s retention budget, TileQuant keeps the highest-scoring tiles at 16-bit and uses 8-bit for the rest. > Within each projection's retention budget, the highest-scoring tiles remain at 16-bit and the rest use 8-bit. - [Supporting source](https://arxiv.org/abs/2611.07217) ## Source-stated The authors report that TileQuant reaches 2.66x the original model’s generation throughput. > Experiments show that TileQuant reaches 2.66x the original model's generation throughput. - [Supporting source](https://arxiv.org/abs/2611.07217) ## Source-stated The authors report preserved quality across four task areas and results exceeding quantized and original models on multiple tasks. > Across language modeling, reasoning, long-context retrieval, and question answering, it preserves quality and surpasses quantized and original models on multiple tasks. - [Supporting source](https://arxiv.org/abs/2611.07217) ## Source and contributor context Zhang, Hanzhi, Zhang, Qiao, Cao, Qinglei, Fan, Heng, Huang, Yan, Sha, Kewei, Feng, Yunhe · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Quantization Storing a model's numbers at lower precision so it uses less memory and runs faster, sometimes at a cost in accuracy. - [Definition reference](https://developers.google.com/machine-learning/glossary#quantization) ## Large language model A language model with many learned numerical settings, used to process and generate language. - [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 98bba058-ad34-4947-8ef0-bce598129cb9. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-08T03:34:08.684Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: d69f2590c037826d945ffc54f224512463438a0f7710908a86d919bb5de6d3ff. Source text hash: 8824fd9421990bf11f80467699fb24eb75b7e99a4b0572e56de86a4ecd41d48e. ## Revision and source-check history ### Version 1 — 2026-11-08T03:34:08.684Z Hanzhi Zhang and coauthors introduce TileQuant, a training-free framework for mixed-precision post-training quantization (PTQ) of large language models (LLMs). Instead of keeping entire columns at higher precision, TileQuant scores hardware-aligned 2D weight tiles, reflecting the authors’ argument that sensitivity can vary even within a column. Offline calibration can score tiles using input-activation-aware forward statistics or loss sensitivity; within a retention budget, the highest-scoring tiles stay at 16-bit and the rest use 8-bit. The resulting map guides reconstruction into compact weight buffers, which the kernel reuses for integer and 16-bit computation with local accumulation to reduce memory traffic. The authors report 2.66x the original model’s generation throughput and say quality is preserved across language modeling, reasoning, long-context retrieval, and question answering, with TileQuant surpassing quantized and original models on multiple tasks. The proposed approach links finer-grained precision allocation to hardware execution, rather than treating sensitivity and runtime efficiency as separate concerns. The abstract does not identify the evaluated models, hardware, or task-by-task results, so it does not show how broadly the reported throughput and quality findings apply. ## Original source arXiv cs.AI · Article Original source date: Nov 4, 2026 (2026-11-04). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-08T03:34:08.684Z. First observed: 2026-11-06T05:12:28.731Z. Retrieved: 2026-11-08T03:33:19.841Z. - [TileQuant: Tile as the Precision-Allocation Unit for LLM Mixed-Precision Post-Training Quantization](https://arxiv.org/abs/2611.07217) ## Limits The abstract does not identify the evaluated models or hardware, or give task-by-task throughput and quality results; the reported results’ breadth cannot be assessed from this evidence alone. ## Open question How do TileQuant’s reported throughput and quality results vary across the evaluated models, hardware, and tasks? ## Source-stated TileQuant assigns precision to hardware-aligned 2D weight tiles, rather than retaining whole columns at higher precision. > We therefore introduce TileQuant, a training-free framework that assigns precision to hardware-aligned 2D weight tiles. - [Supporting source](https://arxiv.org/abs/2611.07217) ## Source-stated TileQuant offers two offline calibration scoring approaches: input-activation-aware scoring from forward statistics and gradient-informed scoring from loss sensitivity. > Offline calibration provides input-activation-aware scoring from forward statistics for lower calibration cost and gradient-informed scoring from loss sensitivity for stronger quality preservation. - [Supporting source](https://arxiv.org/abs/2611.07217) ## Source-stated Within each projection’s retention budget, TileQuant keeps the highest-scoring tiles at 16-bit and uses 8-bit for the rest. > Within each projection's retention budget, the highest-scoring tiles remain at 16-bit and the rest use 8-bit. - [Supporting source](https://arxiv.org/abs/2611.07217) ## Source-stated The authors report that TileQuant reaches 2.66x the original model’s generation throughput. > Experiments show that TileQuant reaches 2.66x the original model's generation throughput. - [Supporting source](https://arxiv.org/abs/2611.07217) ## Source-stated The authors report preserved quality across four task areas and results exceeding quantized and original models on multiple tasks. > Across language modeling, reasoning, long-context retrieval, and question answering, it preserves quality and surpasses quantized and original models on multiple tasks. - [Supporting source](https://arxiv.org/abs/2611.07217) ## Source and contributor context Zhang, Hanzhi, Zhang, Qiao, Cao, Qinglei, Fan, Heng, Huang, Yan, Sha, Kewei, Feng, Yunhe · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Quantization Storing a model's numbers at lower precision so it uses less memory and runs faster, sometimes at a cost in accuracy. - [Definition reference](https://developers.google.com/machine-learning/glossary#quantization) ## Large language model A language model with many learned numerical settings, used to process and generate language. - [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: 98bba058-ad34-4947-8ef0-bce598129cb9. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-08T03:34:08.684Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: d69f2590c037826d945ffc54f224512463438a0f7710908a86d919bb5de6d3ff. Source text hash: 8824fd9421990bf11f80467699fb24eb75b7e99a4b0572e56de86a4ecd41d48e. ### uncertain — 2026-11-08T03:34:08.684Z — anthropic/claude-opus-5.5 Publication: 98bba058-ad34-4947-8ef0-bce598129cb9 attribution — pass: The reading correctly credits Hanzhi Zhang and coauthors and keeps the performance claims attributed ('The authors report'). The title label 'arXiv cs.AI' matches the feed publisher. The paper's primary category is cs.LG, cross-listed to cs.AI, which is consistent. > Authors:Hanzhi Zhang, Qiao Zhang, Qinglei Cao, Heng Fan, Yan Huang, Kewei Sha, Yunhe Feng summary — uncertain: The summary leaves out the source's stated reason the work matters: PTQ is meant to lower LLM deployment costs. It also drops the motivating contrast that runtime outlier detection adds decoding overhead. Without these, a general reader is not told why throughput or quantization matters. Central terms (quantization, 8-bit/16-bit precision, kernel, retention budget) are not explained in plain language. Smallest repair: add one sentence stating that the abstract says PTQ reduces deployment costs by storing model weights at lower numeric precision. Minor: 'native integer' is shortened to 'integer'. The scoring cost/quality tradeoff appears only in the claims. > Post-training quantization (PTQ) reduces large language model (LLM) deployment costs, but preserving quality requires accounting for uneven weight sensitivity. date — pass: The originalPublished date of 2026-11-04 matches the v1 submission date on arXiv. The later feed date of Nov 6 is correctly not used. > \[Submitted on 4 Nov 2026\] support — pass: Every claim matches the abstract verbatim and stays attributed. The 2.66x throughput figure and the quality results are presented as author reports, not as verified facts. The interpretive line linking precision allocation to hardware execution is supported by the authors' concluding sentence. The limitation and question about unidentified models, hardware and per-task results are appropriate gaps, not invented claims. > These results support selecting precision units that capture local sensitivity while enabling efficient hardware execution. media — pass: The selected media is the verified direct arXiv abstract page for this paper. It is the most direct access to the source. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/973621c8ffae13efc4f0bbde3ef3021cfcf1b21507355dab1fec2aef421f189f/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/arxiv-cs-ai-tilequant-assigns-mixed-precision-to-smaller-weight-973621c8.json)