arXiv cs.AI: TileQuant assigns mixed precision to smaller weight tiles

arXiv cs.AI: TileQuant assigns mixed precision to smaller weight tiles

Hanzhi Zhang and coauthors introduce TileQuant, a training-free framework for mixed-precision post-training quantization (PTQ) of large language models (LLMs). Instead of keeping entire columns at higher precision, TileQuant scores hardware-aligned 2D weight tiles, reflecting the authors’ argument that sensitivity can vary even within a column. Offline calibration can score tiles using input-activation-aware forward statistics or loss sensitivity; within a retention budget, the highest-scoring tiles stay at 16-bit and the rest use 8-bit. The resulting map guides reconstruction into compact weight buffers, which the kernel reuses for integer and 16-bit computation with local accumulation to reduce memory traffic. The authors report 2.66x the original model’s generation throughput and say quality is preserved across language modeling, reasoning, long-context retrieval, and question answering, with TileQuant surpassing quantized and original models on multiple tasks. The proposed approach links finer-grained precision allocation to hardware execution, rather than treating sensitivity and runtime efficiency as separate concerns. The abstract does not identify the evaluated models, hardware, or task-by-task results, so it does not show how broadly the reported throughput and quality findings apply.

Publication: 98bba058-ad34-4947-8ef0-bce598129cb9

attribution — pass: The reading correctly credits Hanzhi Zhang and coauthors and keeps the performance claims attributed ('The authors report'). The title label 'arXiv cs.AI' matches the feed publisher. The paper's primary category is cs.LG, cross-listed to cs.AI, which is consistent.

Authors:Hanzhi Zhang, Qiao Zhang, Qinglei Cao, Heng Fan, Yan Huang, Kewei Sha, Yunhe Feng

summary — uncertain: The summary leaves out the source's stated reason the work matters: PTQ is meant to lower LLM deployment costs. It also drops the motivating contrast that runtime outlier detection adds decoding overhead. Without these, a general reader is not told why throughput or quantization matters. Central terms (quantization, 8-bit/16-bit precision, kernel, retention budget) are not explained in plain language. Smallest repair: add one sentence stating that the abstract says PTQ reduces deployment costs by storing model weights at lower numeric precision. Minor: 'native integer' is shortened to 'integer'. The scoring cost/quality tradeoff appears only in the claims.

Post-training quantization (PTQ) reduces large language model (LLM) deployment costs, but preserving quality requires accounting for uneven weight sensitivity.

date — pass: The originalPublished date of 2026-11-04 matches the v1 submission date on arXiv. The later feed date of Nov 6 is correctly not used.

[Submitted on 4 Nov 2026]

support — pass: Every claim matches the abstract verbatim and stays attributed. The 2.66x throughput figure and the quality results are presented as author reports, not as verified facts. The interpretive line linking precision allocation to hardware execution is supported by the authors' concluding sentence. The limitation and question about unidentified models, hardware and per-task results are appropriate gaps, not invented claims.

These results support selecting precision units that capture local sensitivity while enabling efficient hardware execution.

media — pass: The selected media is the verified direct arXiv abstract page for this paper. It is the most direct access to the source.