{"id":"2fe47b044c74a8261ad7ede9ef250508540c0c6841bbd928fbe8a781a5deb83e","title":"Kwan Yan, Jialun Zhong and Yanzeng Li propose LEAP for efficient LLM scheduling","argument":"In a paper submitted to arXiv on 30 October 2026, Kwan Yan, Jialun Zhong and Yanzeng Li propose LEAP, a system for predicting output length so a server can schedule requests predicted to be shorter first. The authors argue that existing predictors can lose ranking quality when the domain or task changes, and that retraining and prediction overhead can erode scheduling gains.\n\nLEAP uses the serving model’s prefill hidden states—the intermediate representations produced while processing an input—and a linear readout to estimate output length. It adapts that readout as requests finish, using their observed lengths as feedback without gradient computation or replaying historical data. The authors say CPU-based prediction and updates run asynchronously, overlapping GPU prefill.\n\nIn the evaluated cross-domain transfers, the authors report that online adaptation improved Spearman correlation by 64.7% under a larger domain shift. In serving experiments, they report 15.9–40.8% lower average per-token latency than the evaluated length predictors, and 1.9% higher throughput than the strongest evaluated baseline with 30,569× fewer predictor parameters. These are reported experimental results, not evidence here of performance across other workloads or production settings.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The supplied evidence is the arXiv abstract, not the paper’s full evaluation details; the workloads, baselines and serving conditions behind the reported results are not specified here. The abstract reports results from evaluated transfers and serving experiments; it does not establish performance across other workloads or production settings.","speaker":{"name":"Yan, Kwan, Zhong, Jialun, Li, Yanzeng","roleAtSource":"Article author"},"source":{"title":"LEAP: Efficient LLM Inference Scheduling with Generalizable Length Prediction","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2610.41359","publishedAt":"2026-10-30","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors propose LEAP to improve length-aware scheduling through generalizable length prediction, online adaptation and asynchronous CPU execution.","excerpt":"To improve the generalization and efficiency of length-aware scheduling, we propose \\textbf{LEAP}, a length prediction system with fast online adaptation and asynchronous CPU execution.","url":"https://arxiv.org/abs/2610.41359","relationship":"Source-stated"},{"text":"LEAP uses the serving model’s prefill hidden states and a linear readout to predict output length.","excerpt":"LEAP reuses the serving model's prefill hidden states, which contain information about the output length, and extracts this signal through a linear readout.","url":"https://arxiv.org/abs/2610.41359","relationship":"Source-stated"},{"text":"LEAP adapts its readout at test time using completed requests as supervision, without gradient computation or historical-data replay.","excerpt":"it updates the readout at test time through recursive least squares, using completed requests as supervision without gradient computation or historical-data replay.","url":"https://arxiv.org/abs/2610.41359","relationship":"Source-stated"},{"text":"The authors report that online adaptation improves Spearman correlation by 64.7% under a larger domain shift in the evaluated cross-domain transfers.","excerpt":"online adaptation improves Spearman correlation by 64.7\\% under a larger domain shift.","url":"https://arxiv.org/abs/2610.41359","relationship":"Source-stated"},{"text":"In serving experiments, the authors report 15.9–40.8% lower average per-token latency than evaluated length predictors, and 1.9% higher throughput with 30,569× fewer predictor parameters than the strongest evaluated baseline.","excerpt":"reduces average per-token latency by 15.9--40.8\\% compared with the evaluated length predictors, while achieving 1.9\\% higher throughput with $30{,}569\\times$ fewer predictor parameters than the strongest evaluated baseline.","url":"https://arxiv.org/abs/2610.41359","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.41359","checkedAt":"2026-11-10T12:40:09.826Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How well would LEAP’s reported gains hold across a wider range of workloads and serving environments?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Kwan Yan, Jialun Zhong and Yanzeng Li propose LEAP for efficient LLM scheduling","summary":"In a paper submitted to arXiv on 30 October 2026, Kwan Yan, Jialun Zhong and Yanzeng Li propose LEAP, a system for predicting output length so a server can schedule requests predicted to be shorter first. The authors argue that existing predictors can lose ranking quality when the domain or task changes, and that retraining and prediction overhead can erode scheduling gains.\n\nLEAP uses the serving model’s prefill hidden states—the intermediate representations produced while processing an input—and a linear readout to estimate output length. It adapts that readout as requests finish, using their observed lengths as feedback without gradient computation or replaying historical data. The authors say CPU-based prediction and updates run asynchronously, overlapping GPU prefill.\n\nIn the evaluated cross-domain transfers, the authors report that online adaptation improved Spearman correlation by 64.7% under a larger domain shift. In serving experiments, they report 15.9–40.8% lower average per-token latency than the evaluated length predictors, and 1.9% higher throughput than the strongest evaluated baseline with 30,569× fewer predictor parameters. These are reported experimental results, not evidence here of performance across other workloads or production settings.","question":"How well would LEAP’s reported gains hold across a wider range of workloads and serving environments?","claims":[{"text":"The authors propose LEAP to improve length-aware scheduling through generalizable length prediction, online adaptation and asynchronous CPU execution.","excerpt":"To improve the generalization and efficiency of length-aware scheduling, we propose \\textbf{LEAP}, a length prediction system with fast online adaptation and asynchronous CPU execution.","support":"source-stated"},{"text":"LEAP uses the serving model’s prefill hidden states and a linear readout to predict output length.","excerpt":"LEAP reuses the serving model's prefill hidden states, which contain information about the output length, and extracts this signal through a linear readout.","support":"source-stated"},{"text":"LEAP adapts its readout at test time using completed requests as supervision, without gradient computation or historical-data replay.","excerpt":"it updates the readout at test time through recursive least squares, using completed requests as supervision without gradient computation or historical-data replay.","support":"source-stated"},{"text":"The authors report that online adaptation improves Spearman correlation by 64.7% under a larger domain shift in the evaluated cross-domain transfers.","excerpt":"online adaptation improves Spearman correlation by 64.7\\% under a larger domain shift.","support":"source-stated"},{"text":"In serving experiments, the authors report 15.9–40.8% lower average per-token latency than evaluated length predictors, and 1.9% higher throughput with 30,569× fewer predictor parameters than the strongest evaluated baseline.","excerpt":"reduces average per-token latency by 15.9--40.8\\% compared with the evaluated length predictors, while achieving 1.9\\% higher throughput with $30{,}569\\times$ fewer predictor parameters than the strongest evaluated baseline.","support":"source-stated"}],"limitations":["The supplied evidence is the arXiv abstract, not the paper’s full evaluation details; the workloads, baselines and serving conditions behind the reported results are not specified here.","The abstract reports results from evaluated transfers and serving experiments; it does not establish performance across other workloads or production settings."],"mediaId":"677135cbe3c879f73c264d8bdc67b87cf3b26c03a117c4e8776352b916f05c70","correctionReason":null,"id":"2fe47b044c74a8261ad7ede9ef250508540c0c6841bbd928fbe8a781a5deb83e","publicationId":"7f7c9c9e-5bd7-4a31-827a-7e577d2e6fa2","version":1,"sourceUrl":"https://arxiv.org/abs/2610.41359","sourceTitle":"LEAP: Efficient LLM Inference Scheduling with Generalizable Length Prediction","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-10-30","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-02T05:04:44.763Z","createdAt":"2026-11-10T12:09:33.991Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"677135cbe3c879f73c264d8bdc67b87cf3b26c03a117c4e8776352b916f05c70","kind":"article","url":"https://arxiv.org/abs/2610.41359","sourceUrl":"https://arxiv.org/abs/2610.41359","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-02T09:49:19.075Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.41359","checkedAt":"2026-11-10T12:40:09.826Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"677135cbe3c879f73c264d8bdc67b87cf3b26c03a117c4e8776352b916f05c70","confidence":0.86,"provider":"typesafe","model":"jev-1.13.0","inputHash":"23102e08ffdf175215eb9c160beba4f8de4857fa96222cb4364cc886b9b18682","decidedAt":"2026-11-02T09:49:21.991Z"},"provenance":{"textHash":"8e4a241bf8a51096417de319cd8bf9d83390c735df7f75b7c2b8707d505e2609","observationId":"c29df14ed5cdd67f00e77961a013fa14b0bfc4406c34f9382ad1d1d79e667b11","retrievedAt":"2026-11-02T09:49:18.812Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"e45cfba6-10ba-4bdf-8fce-4f3ef3660a92"},"reviewer":{"model":"gpt-6-luna","runId":"91722273-9a3f-40e3-8067-fa2188878552"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-10T12:09:33.991Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Yan, Kwan","url":null,"kind":"unknown"},{"name":"Zhong, Jialun","url":null,"kind":"unknown"},{"name":"Li, Yanzeng","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Yan, Kwan","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Yan, Kwan","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.41359","locator":"head/meta[name=author|citation_author]"},"assertionId":"0db52259b3f2d22b1dc46f33b68ec021c32cb0d2229803e2761c72410e3ea25a:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T12:40:09.826Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"0db52259b3f2d22b1dc46f33b68ec021c32cb0d2229803e2761c72410e3ea25a","sourceUrl":"https://arxiv.org/abs/2610.41359","contentSha256":"e285f85eb84080fd7a98bcd76e86e55ee9c21ba31d42e0a263cea6d4b1a7a39a","textHash":"8e4a241bf8a51096417de319cd8bf9d83390c735df7f75b7c2b8707d505e2609","observedAt":"2026-11-10T12:40:09.826Z"}},{"actor":{"name":"Zhong, Jialun","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zhong, Jialun","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.41359","locator":"head/meta[name=author|citation_author]"},"assertionId":"0db52259b3f2d22b1dc46f33b68ec021c32cb0d2229803e2761c72410e3ea25a:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T12:40:09.826Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"0db52259b3f2d22b1dc46f33b68ec021c32cb0d2229803e2761c72410e3ea25a","sourceUrl":"https://arxiv.org/abs/2610.41359","contentSha256":"e285f85eb84080fd7a98bcd76e86e55ee9c21ba31d42e0a263cea6d4b1a7a39a","textHash":"8e4a241bf8a51096417de319cd8bf9d83390c735df7f75b7c2b8707d505e2609","observedAt":"2026-11-10T12:40:09.826Z"}},{"actor":{"name":"Li, Yanzeng","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Li, Yanzeng","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.41359","locator":"head/meta[name=author|citation_author]"},"assertionId":"0db52259b3f2d22b1dc46f33b68ec021c32cb0d2229803e2761c72410e3ea25a:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T12:40:09.826Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"0db52259b3f2d22b1dc46f33b68ec021c32cb0d2229803e2761c72410e3ea25a","sourceUrl":"https://arxiv.org/abs/2610.41359","contentSha256":"e285f85eb84080fd7a98bcd76e86e55ee9c21ba31d42e0a263cea6d4b1a7a39a","textHash":"8e4a241bf8a51096417de319cd8bf9d83390c735df7f75b7c2b8707d505e2609","observedAt":"2026-11-10T12:40:09.826Z"}}],"participationCoverage":{"total":3,"returned":3,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2610.41359","snapshotId":"0db52259b3f2d22b1dc46f33b68ec021c32cb0d2229803e2761c72410e3ea25a","contentSha256":"e285f85eb84080fd7a98bcd76e86e55ee9c21ba31d42e0a263cea6d4b1a7a39a","textHash":"8e4a241bf8a51096417de319cd8bf9d83390c735df7f75b7c2b8707d505e2609","observedAt":"2026-11-10T12:40:09.826Z","basis":"publisher-page-metadata","sourceLinks":41,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/2fe47b044c74a8261ad7ede9ef250508540c0c6841bbd928fbe8a781a5deb83e/source"}},"canonicalUrl":"https://escapemeridian.com/readings/kwan-yan-jialun-zhong-and-yanzeng-li-propose-leap-for-efficient-2fe47b04","formats":{"html":"/readings/kwan-yan-jialun-zhong-and-yanzeng-li-propose-leap-for-efficient-2fe47b04","markdown":"/readings/kwan-yan-jialun-zhong-and-yanzeng-li-propose-leap-for-efficient-2fe47b04.md","json":"/readings/kwan-yan-jialun-zhong-and-yanzeng-li-propose-leap-for-efficient-2fe47b04.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-llm","name":"Large language model","explanation":"A language model with many learned numerical settings, used to process and generate language.","referenceUrl":"https://developers.google.com/machine-learning/glossary#large-language-model"},{"id":"term-cpu","name":"CPU","explanation":"The central processing unit: a general-purpose processor that carries out a computer's instructions.","referenceUrl":"https://aws.amazon.com/what-is/cpu/"},{"id":"term-gpu","name":"Graphics processing unit","explanation":"A chip that performs many calculations at once; the main hardware used to train and run AI models.","referenceUrl":"https://en.wikipedia.org/wiki/Graphics_processing_unit"},{"id":"term-distribution-shift","name":"Distribution shift","explanation":"A change between the data a model was trained on and the data it meets in use, which can lower its accuracy.","referenceUrl":"https://en.wikipedia.org/wiki/Domain_adaptation"},{"id":"term-latency","name":"Latency","explanation":"The delay between sending a request and receiving the result.","referenceUrl":"https://developers.google.com/machine-learning/glossary#latency"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"2fe47b044c74a8261ad7ede9ef250508540c0c6841bbd928fbe8a781a5deb83e","revision":"7f7c9c9e-5bd7-4a31-827a-7e577d2e6fa2","revisionKind":"publication-id","title":"Kwan Yan, Jialun Zhong and Yanzeng Li propose LEAP for efficient LLM scheduling","url":"https://escapemeridian.com/readings/kwan-yan-jialun-zhong-and-yanzeng-li-propose-leap-for-efficient-2fe47b04"},"source":{"type":"source","id":"https://arxiv.org/abs/2610.41359","revision":"8e4a241bf8a51096417de319cd8bf9d83390c735df7f75b7c2b8707d505e2609","revisionKind":"sha256-source-text","title":"LEAP: Efficient LLM Inference Scheduling with Generalizable Length Prediction","url":"https://arxiv.org/abs/2610.41359"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"2fe47b044c74a8261ad7ede9ef250508540c0c6841bbd928fbe8a781a5deb83e","title":"Kwan Yan, Jialun Zhong and Yanzeng Li propose LEAP for efficient LLM scheduling","argument":"In a paper submitted to arXiv on 30 October 2026, Kwan Yan, Jialun Zhong and Yanzeng Li propose LEAP, a system for predicting output length so a server can schedule requests predicted to be shorter first. The authors argue that existing predictors can lose ranking quality when the domain or task changes, and that retraining and prediction overhead can erode scheduling gains.\n\nLEAP uses the serving model’s prefill hidden states—the intermediate representations produced while processing an input—and a linear readout to estimate output length. It adapts that readout as requests finish, using their observed lengths as feedback without gradient computation or replaying historical data. The authors say CPU-based prediction and updates run asynchronously, overlapping GPU prefill.\n\nIn the evaluated cross-domain transfers, the authors report that online adaptation improved Spearman correlation by 64.7% under a larger domain shift. In serving experiments, they report 15.9–40.8% lower average per-token latency than the evaluated length predictors, and 1.9% higher throughput than the strongest evaluated baseline with 30,569× fewer predictor parameters. These are reported experimental results, not evidence here of performance across other workloads or production settings.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The supplied evidence is the arXiv abstract, not the paper’s full evaluation details; the workloads, baselines and serving conditions behind the reported results are not specified here. The abstract reports results from evaluated transfers and serving experiments; it does not establish performance across other workloads or production settings.","speaker":{"name":"Yan, Kwan, Zhong, Jialun, Li, Yanzeng","roleAtSource":"Article author"},"source":{"title":"LEAP: Efficient LLM Inference Scheduling with Generalizable Length Prediction","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2610.41359","publishedAt":"2026-10-30","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors propose LEAP to improve length-aware scheduling through generalizable length prediction, online adaptation and asynchronous CPU execution.","excerpt":"To improve the generalization and efficiency of length-aware scheduling, we propose \\textbf{LEAP}, a length prediction system with fast online adaptation and asynchronous CPU execution.","url":"https://arxiv.org/abs/2610.41359","relationship":"Source-stated"},{"text":"LEAP uses the serving model’s prefill hidden states and a linear readout to predict output length.","excerpt":"LEAP reuses the serving model's prefill hidden states, which contain information about the output length, and extracts this signal through a linear readout.","url":"https://arxiv.org/abs/2610.41359","relationship":"Source-stated"},{"text":"LEAP adapts its readout at test time using completed requests as supervision, without gradient computation or historical-data replay.","excerpt":"it updates the readout at test time through recursive least squares, using completed requests as supervision without gradient computation or historical-data replay.","url":"https://arxiv.org/abs/2610.41359","relationship":"Source-stated"},{"text":"The authors report that online adaptation improves Spearman correlation by 64.7% under a larger domain shift in the evaluated cross-domain transfers.","excerpt":"online adaptation improves Spearman correlation by 64.7\\% under a larger domain shift.","url":"https://arxiv.org/abs/2610.41359","relationship":"Source-stated"},{"text":"In serving experiments, the authors report 15.9–40.8% lower average per-token latency than evaluated length predictors, and 1.9% higher throughput with 30,569× fewer predictor parameters than the strongest evaluated baseline.","excerpt":"reduces average per-token latency by 15.9--40.8\\% compared with the evaluated length predictors, while achieving 1.9\\% higher throughput with $30{,}569\\times$ fewer predictor parameters than the strongest evaluated baseline.","url":"https://arxiv.org/abs/2610.41359","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.41359","checkedAt":"2026-11-10T12:40:09.826Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How well would LEAP’s reported gains hold across a wider range of workloads and serving environments?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"Kwan Yan, Jialun Zhong and Yanzeng Li propose LEAP for efficient LLM scheduling","summary":"In a paper submitted to arXiv on 30 October 2026, Kwan Yan, Jialun Zhong and Yanzeng Li propose LEAP, a system for predicting output length so a server can schedule requests predicted to be shorter first. The authors argue that existing predictors can lose ranking quality when the domain or task changes, and that retraining and prediction overhead can erode scheduling gains.\n\nLEAP uses the serving model’s prefill hidden states—the intermediate representations produced while processing an input—and a linear readout to estimate output length. It adapts that readout as requests finish, using their observed lengths as feedback without gradient computation or replaying historical data. The authors say CPU-based prediction and updates run asynchronously, overlapping GPU prefill.\n\nIn the evaluated cross-domain transfers, the authors report that online adaptation improved Spearman correlation by 64.7% under a larger domain shift. In serving experiments, they report 15.9–40.8% lower average per-token latency than the evaluated length predictors, and 1.9% higher throughput than the strongest evaluated baseline with 30,569× fewer predictor parameters. These are reported experimental results, not evidence here of performance across other workloads or production settings.","question":"How well would LEAP’s reported gains hold across a wider range of workloads and serving environments?","claims":[{"text":"The authors propose LEAP to improve length-aware scheduling through generalizable length prediction, online adaptation and asynchronous CPU execution.","excerpt":"To improve the generalization and efficiency of length-aware scheduling, we propose \\textbf{LEAP}, a length prediction system with fast online adaptation and asynchronous CPU execution.","support":"source-stated"},{"text":"LEAP uses the serving model’s prefill hidden states and a linear readout to predict output length.","excerpt":"LEAP reuses the serving model's prefill hidden states, which contain information about the output length, and extracts this signal through a linear readout.","support":"source-stated"},{"text":"LEAP adapts its readout at test time using completed requests as supervision, without gradient computation or historical-data replay.","excerpt":"it updates the readout at test time through recursive least squares, using completed requests as supervision without gradient computation or historical-data replay.","support":"source-stated"},{"text":"The authors report that online adaptation improves Spearman correlation by 64.7% under a larger domain shift in the evaluated cross-domain transfers.","excerpt":"online adaptation improves Spearman correlation by 64.7\\% under a larger domain shift.","support":"source-stated"},{"text":"In serving experiments, the authors report 15.9–40.8% lower average per-token latency than evaluated length predictors, and 1.9% higher throughput with 30,569× fewer predictor parameters than the strongest evaluated baseline.","excerpt":"reduces average per-token latency by 15.9--40.8\\% compared with the evaluated length predictors, while achieving 1.9\\% higher throughput with $30{,}569\\times$ fewer predictor parameters than the strongest evaluated baseline.","support":"source-stated"}],"limitations":["The supplied evidence is the arXiv abstract, not the paper’s full evaluation details; the workloads, baselines and serving conditions behind the reported results are not specified here.","The abstract reports results from evaluated transfers and serving experiments; it does not establish performance across other workloads or production settings."],"mediaId":"677135cbe3c879f73c264d8bdc67b87cf3b26c03a117c4e8776352b916f05c70","correctionReason":null,"id":"2fe47b044c74a8261ad7ede9ef250508540c0c6841bbd928fbe8a781a5deb83e","publicationId":"7f7c9c9e-5bd7-4a31-827a-7e577d2e6fa2","version":1,"sourceUrl":"https://arxiv.org/abs/2610.41359","sourceTitle":"LEAP: Efficient LLM Inference Scheduling with Generalizable Length Prediction","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-10-30","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-02T05:04:44.763Z","createdAt":"2026-11-10T12:09:33.991Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"677135cbe3c879f73c264d8bdc67b87cf3b26c03a117c4e8776352b916f05c70","kind":"article","url":"https://arxiv.org/abs/2610.41359","sourceUrl":"https://arxiv.org/abs/2610.41359","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-02T09:49:19.075Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2610.41359","checkedAt":"2026-11-10T12:40:09.826Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"677135cbe3c879f73c264d8bdc67b87cf3b26c03a117c4e8776352b916f05c70","confidence":0.86,"provider":"typesafe","model":"jev-1.13.0","inputHash":"23102e08ffdf175215eb9c160beba4f8de4857fa96222cb4364cc886b9b18682","decidedAt":"2026-11-02T09:49:21.991Z"},"provenance":{"textHash":"8e4a241bf8a51096417de319cd8bf9d83390c735df7f75b7c2b8707d505e2609","observationId":"c29df14ed5cdd67f00e77961a013fa14b0bfc4406c34f9382ad1d1d79e667b11","retrievedAt":"2026-11-02T09:49:18.812Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"e45cfba6-10ba-4bdf-8fce-4f3ef3660a92"},"reviewer":{"model":"gpt-6-luna","runId":"91722273-9a3f-40e3-8067-fa2188878552"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-10T12:09:33.991Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Yan, Kwan","url":null,"kind":"unknown"},{"name":"Zhong, Jialun","url":null,"kind":"unknown"},{"name":"Li, Yanzeng","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Yan, Kwan","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Yan, Kwan","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.41359","locator":"head/meta[name=author|citation_author]"},"assertionId":"0db52259b3f2d22b1dc46f33b68ec021c32cb0d2229803e2761c72410e3ea25a:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T12:40:09.826Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"0db52259b3f2d22b1dc46f33b68ec021c32cb0d2229803e2761c72410e3ea25a","sourceUrl":"https://arxiv.org/abs/2610.41359","contentSha256":"e285f85eb84080fd7a98bcd76e86e55ee9c21ba31d42e0a263cea6d4b1a7a39a","textHash":"8e4a241bf8a51096417de319cd8bf9d83390c735df7f75b7c2b8707d505e2609","observedAt":"2026-11-10T12:40:09.826Z"}},{"actor":{"name":"Zhong, Jialun","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zhong, Jialun","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.41359","locator":"head/meta[name=author|citation_author]"},"assertionId":"0db52259b3f2d22b1dc46f33b68ec021c32cb0d2229803e2761c72410e3ea25a:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T12:40:09.826Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"0db52259b3f2d22b1dc46f33b68ec021c32cb0d2229803e2761c72410e3ea25a","sourceUrl":"https://arxiv.org/abs/2610.41359","contentSha256":"e285f85eb84080fd7a98bcd76e86e55ee9c21ba31d42e0a263cea6d4b1a7a39a","textHash":"8e4a241bf8a51096417de319cd8bf9d83390c735df7f75b7c2b8707d505e2609","observedAt":"2026-11-10T12:40:09.826Z"}},{"actor":{"name":"Li, Yanzeng","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Li, Yanzeng","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2610.41359","locator":"head/meta[name=author|citation_author]"},"assertionId":"0db52259b3f2d22b1dc46f33b68ec021c32cb0d2229803e2761c72410e3ea25a:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T12:40:09.826Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"0db52259b3f2d22b1dc46f33b68ec021c32cb0d2229803e2761c72410e3ea25a","sourceUrl":"https://arxiv.org/abs/2610.41359","contentSha256":"e285f85eb84080fd7a98bcd76e86e55ee9c21ba31d42e0a263cea6d4b1a7a39a","textHash":"8e4a241bf8a51096417de319cd8bf9d83390c735df7f75b7c2b8707d505e2609","observedAt":"2026-11-10T12:40:09.826Z"}}],"participationCoverage":{"total":3,"returned":3,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2610.41359","snapshotId":"0db52259b3f2d22b1dc46f33b68ec021c32cb0d2229803e2761c72410e3ea25a","contentSha256":"e285f85eb84080fd7a98bcd76e86e55ee9c21ba31d42e0a263cea6d4b1a7a39a","textHash":"8e4a241bf8a51096417de319cd8bf9d83390c735df7f75b7c2b8707d505e2609","observedAt":"2026-11-10T12:40:09.826Z","basis":"publisher-page-metadata","sourceLinks":41,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/2fe47b044c74a8261ad7ede9ef250508540c0c6841bbd928fbe8a781a5deb83e/source"}}}],"audits":[{"publicationId":"7f7c9c9e-5bd7-4a31-827a-7e577d2e6fa2","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The draft correctly attributes LEAP and its reported results to authors Kwan Yan, Jialun Zhong and Yanzeng Li, and frames the performance figures as the authors' reports rather than established facts.","excerpt":"Authors:Kwan Yan, Jialun Zhong, Yanzeng Li"},{"dimension":"summary","status":"uncertain","explanation":"The reading is mostly faithful but has three readability gaps. (1) It never says why shorter-first matters. The source's premise is that this ordering reduces request latency; add that clause to the first sentence. (2) Spearman correlation is central to a headline result but unexplained; add a term tooltip saying it measures how well predicted lengths rank requests. (3) LLM and recursive least squares lack tooltips. The draft also drops that existing predictors are large, which gives the 30,569x parameter figure its context.","excerpt":"Length-aware scheduling reduces request latency in LLM serving by prioritizing shorter requests."},{"dimension":"date","status":"pass","explanation":"originalPublished is 2026-10-30. This matches the arXiv v1 submission date, and the draft's statement that the paper was submitted on 30 October 2026 is correct. The 2 November feed date was correctly not used.","excerpt":"[v1]\nFri, 30 Oct 2026 07:23:27 UTC (300 KB)"},{"dimension":"support","status":"uncertain","explanation":"Most claims are directly supported by the abstract. However, the passage saying CPU-based prediction and updates run asynchronously while overlapping GPU prefill slightly overstates the source. The abstract says prediction and updates run asynchronously on CPU, but only prediction and initial batch ordering overlap the remaining GPU prefill. Smallest repair: say that prediction and initial batch ordering overlap the remaining GPU prefill. The limitations and the question are appropriately conditional.","excerpt":"Prediction and updates run asynchronously on CPU, while prediction and initial batch ordering overlap the remaining GPU prefill."},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct link to the original arXiv abstract page. It is the most direct way to access the source.","excerpt":null}],"createdAt":"2026-11-10T12:09:33.991Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}