{"id":"3937ae8c58dd4a3a1b30a192b7818e037a3f9c3ca2b2a49edd6371bc89dbb24d","title":"arXiv cs.AI: Intent-Aware Caching for Efficient LLM Serving","argument":"The authors propose Intent-Aware Caching (IAC), an online eviction policy for LLM serving systems that use prefix caching. Prefix caching reuses key-value states from earlier conversation prefixes, avoiding repeated prefill computation when a later request shares the same prefix. The authors argue that user intent can help predict whether a conversation will continue and how soon, which affects whether cached states will be reused. IAC classifies requests by intent, learns lightweight reuse statistics from recent serving logs, and uses them in an oracle-inspired eviction strategy. On the WildChat and ShareChat real workloads, the authors report a 34.6% cache hit rate, compared with 18.8% for a previous learning-based approach and 12.2% for least recently used (LRU). They also report an average serving latency reduction of up to 46.1%. These results suggest that using intent to guide cache eviction may improve reuse and reduce latency in the evaluated workloads; the abstract does not establish how the results carry over to other workloads.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The supplied evidence is the arXiv abstract; it does not provide the evaluation methods or results broken down by WildChat and ShareChat. The reported results concern the named real workloads; the abstract does not establish how they generalize to other workloads.","speaker":{"name":"Arif, Kazi Hasan Ibn, Yoon, JinYi, Nikolopoulos, Dimitrios S., Vandierendonck, Hans, John, Deepu, Ji, Bo","roleAtSource":"Article author"},"source":{"title":"Intent-Aware Caching for Efficient LLM Serving","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.09715","publishedAt":"2026-11-06","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors propose IAC, an online eviction policy that classifies requests by intent and uses reuse statistics learned from recent serving logs to guide cache eviction.","excerpt":"we propose Intent-Aware Caching (IAC), an online eviction policy that classifies requests by intent, learns lightweight reuse statistics from recent serving logs and leverages these statistics to guide an oracle-inspired eviction strategy.","url":"https://arxiv.org/abs/2611.09715","relationship":"Source-stated"},{"text":"The authors report that IAC reached a 34.6% cache hit rate on the WildChat and ShareChat real workloads.","excerpt":"Extensive evaluation on real workloads (WildChat and ShareChat) shows that IAC reaches a 34.6% cache hit rate.","url":"https://arxiv.org/abs/2611.09715","relationship":"Source-stated"},{"text":"The authors report that IAC's cache hit rate exceeded the reported results for a previous learning-based approach and least recently used (LRU).","excerpt":"This substantially outperforms both the previous learning-based approach (18.8%) and traditional methods such as least recently used (LRU, 12.2%)","url":"https://arxiv.org/abs/2611.09715","relationship":"Source-stated"},{"text":"The authors report an average serving latency reduction of up to 46.1%.","excerpt":"while reducing average serving latency by up to 46.1%.","url":"https://arxiv.org/abs/2611.09715","relationship":"Source-stated"},{"text":"The authors identify user intent as a useful signal for predicting conversation continuation and the time until the next turn.","excerpt":"user intent is a useful signal for this behavior, as different intents exhibit different continuation probabilities and inter-turn delays.","url":"https://arxiv.org/abs/2611.09715","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.09715","checkedAt":"2026-11-10T00:34:08.687Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How do IAC's cache hit rate and latency results vary between the WildChat and ShareChat workloads?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: Intent-Aware Caching for Efficient LLM Serving","summary":"The authors propose Intent-Aware Caching (IAC), an online eviction policy for LLM serving systems that use prefix caching. Prefix caching reuses key-value states from earlier conversation prefixes, avoiding repeated prefill computation when a later request shares the same prefix. The authors argue that user intent can help predict whether a conversation will continue and how soon, which affects whether cached states will be reused. IAC classifies requests by intent, learns lightweight reuse statistics from recent serving logs, and uses them in an oracle-inspired eviction strategy. On the WildChat and ShareChat real workloads, the authors report a 34.6% cache hit rate, compared with 18.8% for a previous learning-based approach and 12.2% for least recently used (LRU). They also report an average serving latency reduction of up to 46.1%. These results suggest that using intent to guide cache eviction may improve reuse and reduce latency in the evaluated workloads; the abstract does not establish how the results carry over to other workloads.","question":"How do IAC's cache hit rate and latency results vary between the WildChat and ShareChat workloads?","claims":[{"text":"The authors propose IAC, an online eviction policy that classifies requests by intent and uses reuse statistics learned from recent serving logs to guide cache eviction.","excerpt":"we propose Intent-Aware Caching (IAC), an online eviction policy that classifies requests by intent, learns lightweight reuse statistics from recent serving logs and leverages these statistics to guide an oracle-inspired eviction strategy.","support":"source-stated"},{"text":"The authors report that IAC reached a 34.6% cache hit rate on the WildChat and ShareChat real workloads.","excerpt":"Extensive evaluation on real workloads (WildChat and ShareChat) shows that IAC reaches a 34.6% cache hit rate.","support":"source-stated"},{"text":"The authors report that IAC's cache hit rate exceeded the reported results for a previous learning-based approach and least recently used (LRU).","excerpt":"This substantially outperforms both the previous learning-based approach (18.8%) and traditional methods such as least recently used (LRU, 12.2%)","support":"source-stated"},{"text":"The authors report an average serving latency reduction of up to 46.1%.","excerpt":"while reducing average serving latency by up to 46.1%.","support":"source-stated"},{"text":"The authors identify user intent as a useful signal for predicting conversation continuation and the time until the next turn.","excerpt":"user intent is a useful signal for this behavior, as different intents exhibit different continuation probabilities and inter-turn delays.","support":"source-stated"}],"limitations":["The supplied evidence is the arXiv abstract; it does not provide the evaluation methods or results broken down by WildChat and ShareChat.","The reported results concern the named real workloads; the abstract does not establish how they generalize to other workloads."],"mediaId":"00bd663767bd693f7f015402d22bd06cd35c1b37e1c4e91b685cc796e520d0ba","correctionReason":null,"id":"3937ae8c58dd4a3a1b30a192b7818e037a3f9c3ca2b2a49edd6371bc89dbb24d","publicationId":"f53f4d69-efaf-4af6-897d-ddfad317306d","version":1,"sourceUrl":"https://arxiv.org/abs/2611.09715","sourceTitle":"Intent-Aware Caching for Efficient LLM Serving","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-06","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-09T05:06:57.790Z","createdAt":"2026-11-10T00:02:28.833Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"00bd663767bd693f7f015402d22bd06cd35c1b37e1c4e91b685cc796e520d0ba","kind":"article","url":"https://arxiv.org/abs/2611.09715","sourceUrl":"https://arxiv.org/abs/2611.09715","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-09T23:45:52.698Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.09715","checkedAt":"2026-11-10T00:34:08.687Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"00bd663767bd693f7f015402d22bd06cd35c1b37e1c4e91b685cc796e520d0ba","confidence":0.87,"provider":"typesafe","model":"jev-1.13.0","inputHash":"33d9279baef14c3518bc1954bce70f969e651114ee308280d3ed95320e088d9d","decidedAt":"2026-11-09T23:45:53.653Z"},"provenance":{"textHash":"184eaf3ddbeed6e2dc4a8ad8bbfe2c6ed9ebc2c65974b0e223f8d646a1e76398","observationId":"bd9857963e147207f797006cced630b6797c5083376649eb893b8f3794d9c6e5","retrievedAt":"2026-11-09T23:45:52.592Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"cb2bfd7f-f75e-4edf-8d97-dead47635aa1"},"reviewer":{"model":"gpt-6-luna","runId":"e28f73ee-e73b-4d3d-8ba8-80ac3916cd9c"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-10T00:02:28.833Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Arif, Kazi Hasan Ibn","url":null,"kind":"unknown"},{"name":"Yoon, JinYi","url":null,"kind":"unknown"},{"name":"Nikolopoulos, Dimitrios S.","url":null,"kind":"unknown"},{"name":"Vandierendonck, Hans","url":null,"kind":"unknown"},{"name":"John, Deepu","url":null,"kind":"unknown"},{"name":"Ji, Bo","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Arif, Kazi Hasan Ibn","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Arif, Kazi Hasan Ibn","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.09715","locator":"head/meta[name=author|citation_author]"},"assertionId":"6c39d49d5fc0cafaa4aaf9d2cd04be046c81cd3dc6451d8dbb181318d7a3eea5:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T00:34:08.687Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"6c39d49d5fc0cafaa4aaf9d2cd04be046c81cd3dc6451d8dbb181318d7a3eea5","sourceUrl":"https://arxiv.org/abs/2611.09715","contentSha256":"b122a7857280ab39df796ca0b93b60940ea0a9db2d98d6bed327f33b644d8d74","textHash":"184eaf3ddbeed6e2dc4a8ad8bbfe2c6ed9ebc2c65974b0e223f8d646a1e76398","observedAt":"2026-11-10T00:34:08.687Z"}},{"actor":{"name":"Yoon, JinYi","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Yoon, JinYi","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.09715","locator":"head/meta[name=author|citation_author]"},"assertionId":"6c39d49d5fc0cafaa4aaf9d2cd04be046c81cd3dc6451d8dbb181318d7a3eea5:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T00:34:08.687Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"6c39d49d5fc0cafaa4aaf9d2cd04be046c81cd3dc6451d8dbb181318d7a3eea5","sourceUrl":"https://arxiv.org/abs/2611.09715","contentSha256":"b122a7857280ab39df796ca0b93b60940ea0a9db2d98d6bed327f33b644d8d74","textHash":"184eaf3ddbeed6e2dc4a8ad8bbfe2c6ed9ebc2c65974b0e223f8d646a1e76398","observedAt":"2026-11-10T00:34:08.687Z"}},{"actor":{"name":"Nikolopoulos, Dimitrios S.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Nikolopoulos, Dimitrios S.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.09715","locator":"head/meta[name=author|citation_author]"},"assertionId":"6c39d49d5fc0cafaa4aaf9d2cd04be046c81cd3dc6451d8dbb181318d7a3eea5:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T00:34:08.687Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"6c39d49d5fc0cafaa4aaf9d2cd04be046c81cd3dc6451d8dbb181318d7a3eea5","sourceUrl":"https://arxiv.org/abs/2611.09715","contentSha256":"b122a7857280ab39df796ca0b93b60940ea0a9db2d98d6bed327f33b644d8d74","textHash":"184eaf3ddbeed6e2dc4a8ad8bbfe2c6ed9ebc2c65974b0e223f8d646a1e76398","observedAt":"2026-11-10T00:34:08.687Z"}},{"actor":{"name":"Vandierendonck, Hans","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Vandierendonck, Hans","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.09715","locator":"head/meta[name=author|citation_author]"},"assertionId":"6c39d49d5fc0cafaa4aaf9d2cd04be046c81cd3dc6451d8dbb181318d7a3eea5:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T00:34:08.687Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"6c39d49d5fc0cafaa4aaf9d2cd04be046c81cd3dc6451d8dbb181318d7a3eea5","sourceUrl":"https://arxiv.org/abs/2611.09715","contentSha256":"b122a7857280ab39df796ca0b93b60940ea0a9db2d98d6bed327f33b644d8d74","textHash":"184eaf3ddbeed6e2dc4a8ad8bbfe2c6ed9ebc2c65974b0e223f8d646a1e76398","observedAt":"2026-11-10T00:34:08.687Z"}},{"actor":{"name":"John, Deepu","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"John, Deepu","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.09715","locator":"head/meta[name=author|citation_author]"},"assertionId":"6c39d49d5fc0cafaa4aaf9d2cd04be046c81cd3dc6451d8dbb181318d7a3eea5:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T00:34:08.687Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"6c39d49d5fc0cafaa4aaf9d2cd04be046c81cd3dc6451d8dbb181318d7a3eea5","sourceUrl":"https://arxiv.org/abs/2611.09715","contentSha256":"b122a7857280ab39df796ca0b93b60940ea0a9db2d98d6bed327f33b644d8d74","textHash":"184eaf3ddbeed6e2dc4a8ad8bbfe2c6ed9ebc2c65974b0e223f8d646a1e76398","observedAt":"2026-11-10T00:34:08.687Z"}},{"actor":{"name":"Ji, Bo","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Ji, Bo","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.09715","locator":"head/meta[name=author|citation_author]"},"assertionId":"6c39d49d5fc0cafaa4aaf9d2cd04be046c81cd3dc6451d8dbb181318d7a3eea5:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T00:34:08.687Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"6c39d49d5fc0cafaa4aaf9d2cd04be046c81cd3dc6451d8dbb181318d7a3eea5","sourceUrl":"https://arxiv.org/abs/2611.09715","contentSha256":"b122a7857280ab39df796ca0b93b60940ea0a9db2d98d6bed327f33b644d8d74","textHash":"184eaf3ddbeed6e2dc4a8ad8bbfe2c6ed9ebc2c65974b0e223f8d646a1e76398","observedAt":"2026-11-10T00:34:08.687Z"}}],"participationCoverage":{"total":6,"returned":6,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.09715","snapshotId":"6c39d49d5fc0cafaa4aaf9d2cd04be046c81cd3dc6451d8dbb181318d7a3eea5","contentSha256":"b122a7857280ab39df796ca0b93b60940ea0a9db2d98d6bed327f33b644d8d74","textHash":"184eaf3ddbeed6e2dc4a8ad8bbfe2c6ed9ebc2c65974b0e223f8d646a1e76398","observedAt":"2026-11-10T00:34:08.687Z","basis":"publisher-page-metadata","sourceLinks":45,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/3937ae8c58dd4a3a1b30a192b7818e037a3f9c3ca2b2a49edd6371bc89dbb24d/source"}},"canonicalUrl":"https://escapemeridian.com/readings/arxiv-cs-ai-intent-aware-caching-for-efficient-llm-serving-3937ae8c","formats":{"html":"/readings/arxiv-cs-ai-intent-aware-caching-for-efficient-llm-serving-3937ae8c","markdown":"/readings/arxiv-cs-ai-intent-aware-caching-for-efficient-llm-serving-3937ae8c.md","json":"/readings/arxiv-cs-ai-intent-aware-caching-for-efficient-llm-serving-3937ae8c.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-llm","name":"Large language model","explanation":"A language model with many learned numerical settings, used to process and generate language.","referenceUrl":"https://developers.google.com/machine-learning/glossary#large-language-model"},{"id":"term-latency","name":"Latency","explanation":"The delay between sending a request and receiving the result.","referenceUrl":"https://developers.google.com/machine-learning/glossary#latency"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"3937ae8c58dd4a3a1b30a192b7818e037a3f9c3ca2b2a49edd6371bc89dbb24d","revision":"f53f4d69-efaf-4af6-897d-ddfad317306d","revisionKind":"publication-id","title":"arXiv cs.AI: Intent-Aware Caching for Efficient LLM Serving","url":"https://escapemeridian.com/readings/arxiv-cs-ai-intent-aware-caching-for-efficient-llm-serving-3937ae8c"},"source":{"type":"source","id":"https://arxiv.org/abs/2611.09715","revision":"184eaf3ddbeed6e2dc4a8ad8bbfe2c6ed9ebc2c65974b0e223f8d646a1e76398","revisionKind":"sha256-source-text","title":"Intent-Aware Caching for Efficient LLM Serving","url":"https://arxiv.org/abs/2611.09715"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"3937ae8c58dd4a3a1b30a192b7818e037a3f9c3ca2b2a49edd6371bc89dbb24d","title":"arXiv cs.AI: Intent-Aware Caching for Efficient LLM Serving","argument":"The authors propose Intent-Aware Caching (IAC), an online eviction policy for LLM serving systems that use prefix caching. Prefix caching reuses key-value states from earlier conversation prefixes, avoiding repeated prefill computation when a later request shares the same prefix. The authors argue that user intent can help predict whether a conversation will continue and how soon, which affects whether cached states will be reused. IAC classifies requests by intent, learns lightweight reuse statistics from recent serving logs, and uses them in an oracle-inspired eviction strategy. On the WildChat and ShareChat real workloads, the authors report a 34.6% cache hit rate, compared with 18.8% for a previous learning-based approach and 12.2% for least recently used (LRU). They also report an average serving latency reduction of up to 46.1%. These results suggest that using intent to guide cache eviction may improve reuse and reduce latency in the evaluated workloads; the abstract does not establish how the results carry over to other workloads.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The supplied evidence is the arXiv abstract; it does not provide the evaluation methods or results broken down by WildChat and ShareChat. The reported results concern the named real workloads; the abstract does not establish how they generalize to other workloads.","speaker":{"name":"Arif, Kazi Hasan Ibn, Yoon, JinYi, Nikolopoulos, Dimitrios S., Vandierendonck, Hans, John, Deepu, Ji, Bo","roleAtSource":"Article author"},"source":{"title":"Intent-Aware Caching for Efficient LLM Serving","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.09715","publishedAt":"2026-11-06","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors propose IAC, an online eviction policy that classifies requests by intent and uses reuse statistics learned from recent serving logs to guide cache eviction.","excerpt":"we propose Intent-Aware Caching (IAC), an online eviction policy that classifies requests by intent, learns lightweight reuse statistics from recent serving logs and leverages these statistics to guide an oracle-inspired eviction strategy.","url":"https://arxiv.org/abs/2611.09715","relationship":"Source-stated"},{"text":"The authors report that IAC reached a 34.6% cache hit rate on the WildChat and ShareChat real workloads.","excerpt":"Extensive evaluation on real workloads (WildChat and ShareChat) shows that IAC reaches a 34.6% cache hit rate.","url":"https://arxiv.org/abs/2611.09715","relationship":"Source-stated"},{"text":"The authors report that IAC's cache hit rate exceeded the reported results for a previous learning-based approach and least recently used (LRU).","excerpt":"This substantially outperforms both the previous learning-based approach (18.8%) and traditional methods such as least recently used (LRU, 12.2%)","url":"https://arxiv.org/abs/2611.09715","relationship":"Source-stated"},{"text":"The authors report an average serving latency reduction of up to 46.1%.","excerpt":"while reducing average serving latency by up to 46.1%.","url":"https://arxiv.org/abs/2611.09715","relationship":"Source-stated"},{"text":"The authors identify user intent as a useful signal for predicting conversation continuation and the time until the next turn.","excerpt":"user intent is a useful signal for this behavior, as different intents exhibit different continuation probabilities and inter-turn delays.","url":"https://arxiv.org/abs/2611.09715","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.09715","checkedAt":"2026-11-10T00:34:08.687Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How do IAC's cache hit rate and latency results vary between the WildChat and ShareChat workloads?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: Intent-Aware Caching for Efficient LLM Serving","summary":"The authors propose Intent-Aware Caching (IAC), an online eviction policy for LLM serving systems that use prefix caching. Prefix caching reuses key-value states from earlier conversation prefixes, avoiding repeated prefill computation when a later request shares the same prefix. The authors argue that user intent can help predict whether a conversation will continue and how soon, which affects whether cached states will be reused. IAC classifies requests by intent, learns lightweight reuse statistics from recent serving logs, and uses them in an oracle-inspired eviction strategy. On the WildChat and ShareChat real workloads, the authors report a 34.6% cache hit rate, compared with 18.8% for a previous learning-based approach and 12.2% for least recently used (LRU). They also report an average serving latency reduction of up to 46.1%. These results suggest that using intent to guide cache eviction may improve reuse and reduce latency in the evaluated workloads; the abstract does not establish how the results carry over to other workloads.","question":"How do IAC's cache hit rate and latency results vary between the WildChat and ShareChat workloads?","claims":[{"text":"The authors propose IAC, an online eviction policy that classifies requests by intent and uses reuse statistics learned from recent serving logs to guide cache eviction.","excerpt":"we propose Intent-Aware Caching (IAC), an online eviction policy that classifies requests by intent, learns lightweight reuse statistics from recent serving logs and leverages these statistics to guide an oracle-inspired eviction strategy.","support":"source-stated"},{"text":"The authors report that IAC reached a 34.6% cache hit rate on the WildChat and ShareChat real workloads.","excerpt":"Extensive evaluation on real workloads (WildChat and ShareChat) shows that IAC reaches a 34.6% cache hit rate.","support":"source-stated"},{"text":"The authors report that IAC's cache hit rate exceeded the reported results for a previous learning-based approach and least recently used (LRU).","excerpt":"This substantially outperforms both the previous learning-based approach (18.8%) and traditional methods such as least recently used (LRU, 12.2%)","support":"source-stated"},{"text":"The authors report an average serving latency reduction of up to 46.1%.","excerpt":"while reducing average serving latency by up to 46.1%.","support":"source-stated"},{"text":"The authors identify user intent as a useful signal for predicting conversation continuation and the time until the next turn.","excerpt":"user intent is a useful signal for this behavior, as different intents exhibit different continuation probabilities and inter-turn delays.","support":"source-stated"}],"limitations":["The supplied evidence is the arXiv abstract; it does not provide the evaluation methods or results broken down by WildChat and ShareChat.","The reported results concern the named real workloads; the abstract does not establish how they generalize to other workloads."],"mediaId":"00bd663767bd693f7f015402d22bd06cd35c1b37e1c4e91b685cc796e520d0ba","correctionReason":null,"id":"3937ae8c58dd4a3a1b30a192b7818e037a3f9c3ca2b2a49edd6371bc89dbb24d","publicationId":"f53f4d69-efaf-4af6-897d-ddfad317306d","version":1,"sourceUrl":"https://arxiv.org/abs/2611.09715","sourceTitle":"Intent-Aware Caching for Efficient LLM Serving","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-06","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-09T05:06:57.790Z","createdAt":"2026-11-10T00:02:28.833Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"00bd663767bd693f7f015402d22bd06cd35c1b37e1c4e91b685cc796e520d0ba","kind":"article","url":"https://arxiv.org/abs/2611.09715","sourceUrl":"https://arxiv.org/abs/2611.09715","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-09T23:45:52.698Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.09715","checkedAt":"2026-11-10T00:34:08.687Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"00bd663767bd693f7f015402d22bd06cd35c1b37e1c4e91b685cc796e520d0ba","confidence":0.87,"provider":"typesafe","model":"jev-1.13.0","inputHash":"33d9279baef14c3518bc1954bce70f969e651114ee308280d3ed95320e088d9d","decidedAt":"2026-11-09T23:45:53.653Z"},"provenance":{"textHash":"184eaf3ddbeed6e2dc4a8ad8bbfe2c6ed9ebc2c65974b0e223f8d646a1e76398","observationId":"bd9857963e147207f797006cced630b6797c5083376649eb893b8f3794d9c6e5","retrievedAt":"2026-11-09T23:45:52.592Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"cb2bfd7f-f75e-4edf-8d97-dead47635aa1"},"reviewer":{"model":"gpt-6-luna","runId":"e28f73ee-e73b-4d3d-8ba8-80ac3916cd9c"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-10T00:02:28.833Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Arif, Kazi Hasan Ibn","url":null,"kind":"unknown"},{"name":"Yoon, JinYi","url":null,"kind":"unknown"},{"name":"Nikolopoulos, Dimitrios S.","url":null,"kind":"unknown"},{"name":"Vandierendonck, Hans","url":null,"kind":"unknown"},{"name":"John, Deepu","url":null,"kind":"unknown"},{"name":"Ji, Bo","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Arif, Kazi Hasan Ibn","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Arif, Kazi Hasan Ibn","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.09715","locator":"head/meta[name=author|citation_author]"},"assertionId":"6c39d49d5fc0cafaa4aaf9d2cd04be046c81cd3dc6451d8dbb181318d7a3eea5:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T00:34:08.687Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"6c39d49d5fc0cafaa4aaf9d2cd04be046c81cd3dc6451d8dbb181318d7a3eea5","sourceUrl":"https://arxiv.org/abs/2611.09715","contentSha256":"b122a7857280ab39df796ca0b93b60940ea0a9db2d98d6bed327f33b644d8d74","textHash":"184eaf3ddbeed6e2dc4a8ad8bbfe2c6ed9ebc2c65974b0e223f8d646a1e76398","observedAt":"2026-11-10T00:34:08.687Z"}},{"actor":{"name":"Yoon, JinYi","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Yoon, JinYi","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.09715","locator":"head/meta[name=author|citation_author]"},"assertionId":"6c39d49d5fc0cafaa4aaf9d2cd04be046c81cd3dc6451d8dbb181318d7a3eea5:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T00:34:08.687Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"6c39d49d5fc0cafaa4aaf9d2cd04be046c81cd3dc6451d8dbb181318d7a3eea5","sourceUrl":"https://arxiv.org/abs/2611.09715","contentSha256":"b122a7857280ab39df796ca0b93b60940ea0a9db2d98d6bed327f33b644d8d74","textHash":"184eaf3ddbeed6e2dc4a8ad8bbfe2c6ed9ebc2c65974b0e223f8d646a1e76398","observedAt":"2026-11-10T00:34:08.687Z"}},{"actor":{"name":"Nikolopoulos, Dimitrios S.","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Nikolopoulos, Dimitrios S.","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.09715","locator":"head/meta[name=author|citation_author]"},"assertionId":"6c39d49d5fc0cafaa4aaf9d2cd04be046c81cd3dc6451d8dbb181318d7a3eea5:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T00:34:08.687Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"6c39d49d5fc0cafaa4aaf9d2cd04be046c81cd3dc6451d8dbb181318d7a3eea5","sourceUrl":"https://arxiv.org/abs/2611.09715","contentSha256":"b122a7857280ab39df796ca0b93b60940ea0a9db2d98d6bed327f33b644d8d74","textHash":"184eaf3ddbeed6e2dc4a8ad8bbfe2c6ed9ebc2c65974b0e223f8d646a1e76398","observedAt":"2026-11-10T00:34:08.687Z"}},{"actor":{"name":"Vandierendonck, Hans","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Vandierendonck, Hans","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.09715","locator":"head/meta[name=author|citation_author]"},"assertionId":"6c39d49d5fc0cafaa4aaf9d2cd04be046c81cd3dc6451d8dbb181318d7a3eea5:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T00:34:08.687Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"6c39d49d5fc0cafaa4aaf9d2cd04be046c81cd3dc6451d8dbb181318d7a3eea5","sourceUrl":"https://arxiv.org/abs/2611.09715","contentSha256":"b122a7857280ab39df796ca0b93b60940ea0a9db2d98d6bed327f33b644d8d74","textHash":"184eaf3ddbeed6e2dc4a8ad8bbfe2c6ed9ebc2c65974b0e223f8d646a1e76398","observedAt":"2026-11-10T00:34:08.687Z"}},{"actor":{"name":"John, Deepu","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"John, Deepu","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.09715","locator":"head/meta[name=author|citation_author]"},"assertionId":"6c39d49d5fc0cafaa4aaf9d2cd04be046c81cd3dc6451d8dbb181318d7a3eea5:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T00:34:08.687Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"6c39d49d5fc0cafaa4aaf9d2cd04be046c81cd3dc6451d8dbb181318d7a3eea5","sourceUrl":"https://arxiv.org/abs/2611.09715","contentSha256":"b122a7857280ab39df796ca0b93b60940ea0a9db2d98d6bed327f33b644d8d74","textHash":"184eaf3ddbeed6e2dc4a8ad8bbfe2c6ed9ebc2c65974b0e223f8d646a1e76398","observedAt":"2026-11-10T00:34:08.687Z"}},{"actor":{"name":"Ji, Bo","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Ji, Bo","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.09715","locator":"head/meta[name=author|citation_author]"},"assertionId":"6c39d49d5fc0cafaa4aaf9d2cd04be046c81cd3dc6451d8dbb181318d7a3eea5:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-10T00:34:08.687Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"6c39d49d5fc0cafaa4aaf9d2cd04be046c81cd3dc6451d8dbb181318d7a3eea5","sourceUrl":"https://arxiv.org/abs/2611.09715","contentSha256":"b122a7857280ab39df796ca0b93b60940ea0a9db2d98d6bed327f33b644d8d74","textHash":"184eaf3ddbeed6e2dc4a8ad8bbfe2c6ed9ebc2c65974b0e223f8d646a1e76398","observedAt":"2026-11-10T00:34:08.687Z"}}],"participationCoverage":{"total":6,"returned":6,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.09715","snapshotId":"6c39d49d5fc0cafaa4aaf9d2cd04be046c81cd3dc6451d8dbb181318d7a3eea5","contentSha256":"b122a7857280ab39df796ca0b93b60940ea0a9db2d98d6bed327f33b644d8d74","textHash":"184eaf3ddbeed6e2dc4a8ad8bbfe2c6ed9ebc2c65974b0e223f8d646a1e76398","observedAt":"2026-11-10T00:34:08.687Z","basis":"publisher-page-metadata","sourceLinks":45,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/3937ae8c58dd4a3a1b30a192b7818e037a3f9c3ca2b2a49edd6371bc89dbb24d/source"}}}],"audits":[{"publicationId":"f53f4d69-efaf-4af6-897d-ddfad317306d","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading attributes the proposal, the intent finding and all results to 'the authors' of the arXiv paper, matching the abstract's first-person 'we propose' and 'Our analysis'. The numbers stay reported claims, not established facts. The title prefix 'arXiv cs.AI' correctly names the venue.","excerpt":"Based on this key insight, we propose Intent-Aware Caching (IAC), an online eviction policy"},{"dimension":"summary","status":"uncertain","explanation":"The summary is faithful and keeps the source's force. It explains prefix caching, but central terms stay opaque to a reader outside AI. 'Eviction policy' is the core idea: deciding which cached conversation data to discard when space runs out. 'Cache hit rate', 'key-value states', 'prefill' and 'oracle-inspired' are also unexplained, so readers cannot easily see what IAC changes or why the gain from 12.2% to 34.6% matters. Smallest repair: add tooltip entries for these terms, or one plain clause on eviction. The question about per-workload variation is legitimate.","excerpt":"an online eviction policy that classifies requests by intent"},{"dimension":"date","status":"pass","explanation":"The originalPublished value, 2026-11-06, matches the arXiv v1 submission date. The later feed date of 2026-11-09 is correctly not used.","excerpt":"[Submitted on 6 Nov 2026]"},{"dimension":"support","status":"pass","explanation":"Every claim is backed by a verbatim excerpt, and the figures 34.6%, 18.8%, 12.2% and 'up to 46.1%' are exact. The closing interpretation is hedged and limited to the evaluated workloads. The limitations correctly note that the abstract gives no per-dataset breakdown and does not establish generalization.","excerpt":"IAC reaches a 34.6% cache hit rate. This substantially outperforms both the previous learning-based approach (18.8%) and traditional methods such as least recently used (LRU, 12.2%)"},{"dimension":"media","status":"pass","explanation":"The selected media ID is the verified direct link to the arXiv abstract page, which is the original source. No better source media was found.","excerpt":null}],"createdAt":"2026-11-10T00:02:28.833Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}