{"id":"113fff2c94eab6f91ee6d7e9cdc3881947957d3ff0ad1992f6d3e0b6afa458b0","title":"CascadeRec authors report adaptive LLM compute allocation in production","argument":"The authors present CascadeRec, a two-tier LLM reranking system deployed for content recommendations at a financial planning and analysis SaaS provider. A lightweight Tier-1 model handles every request; a router sends selected requests to a more expensive Tier-2 procedure when extra inference-time compute is likely to help. The authors say the system’s segment-gated approach reflects an important complication: applying the expensive tier uniformly can lower quality for users with sparse interaction histories, and escalating cold users can make the loss larger. In public-benchmark tests, uniform LLM reranking lowered NDCG@10 for sparse-history users by up to 30%; its only statistically significant gain was on the segment with the longest histories. Which segments benefit from escalation depends on the domain, the authors report. In a production A/B experiment lasting over a month, CascadeRec reduced inference cost by about 10% compared with uniform use of the expensive tier, with minimal latency impact and no quality regression observed to date. The authors also describe inter-tenant load imbalance and handling LLM output-parsing failures. Tenant data cannot be released, so the public-benchmark results provide a separate view of the pipeline, not a direct disclosure of its production data.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"Tenant data cannot be released; the abstract reports public-benchmark evaluation alongside the production experiment. The production result is described as an A/B experiment that had run for over a month; the authors report no quality regression observed to date, not a guarantee of future quality.","speaker":{"name":"Zhao, Kai, Zhang, Guilin, Cheng, Xiang, Cui, Xiquan, Heng, Henry, Chu, Xu, Blanken, Alet","roleAtSource":"Article author"},"source":{"title":"CascadeRec: Deployed Adaptive LLM Compute Allocation for Multi-Tenant Content Recommendation","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.08566","publishedAt":"2026-11-05","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors describe CascadeRec as a deployed two-tier LLM reranking system that routes selected requests to a more expensive procedure.","excerpt":"A lightweight Tier-1 model reranks every request. A router escalates only the requests where extra inference-time compute is likely to help","url":"https://arxiv.org/abs/2611.08566","relationship":"Source-stated"},{"text":"The authors report that uniform LLM reranking can lower NDCG@10 for sparse-history users, and that escalating cold users can worsen the loss.","excerpt":"uniform LLM reranking lowers NDCG@10 for sparse-history users by up to 30%, and its only statistically significant gain appears on the segment with the longest histories. Escalating cold users can make the loss larger instead of repairing it.","url":"https://arxiv.org/abs/2611.08566","relationship":"Source-stated"},{"text":"The authors report about a 10% inference-cost reduction in the production A/B experiment compared with uniform use of the expensive tier, without observed quality regression to date.","excerpt":"Compared with uniform use of the expensive tier, CascadeRec has reduced inference cost by about 10%, with minimal latency impact and no quality regression observed to date.","url":"https://arxiv.org/abs/2611.08566","relationship":"Source-stated"},{"text":"The authors say which segments benefit from escalation depends on the domain, motivating their segment-gated production rollout.","excerpt":"Which segments benefit from escalation depends on the domain, and this motivates the segment-gated rollout we use in production.","url":"https://arxiv.org/abs/2611.08566","relationship":"Source-stated"},{"text":"The authors say tenant data cannot be released and report evaluating the pipeline on public benchmarks instead.","excerpt":"Tenant data cannot be released, so we also evaluate the pipeline on public benchmarks (MovieLens-1M and Amazon-Electronics).","url":"https://arxiv.org/abs/2611.08566","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.08566","checkedAt":"2026-11-13T01:12:08.137Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How does the segment-level benefit of escalation vary across domains beyond the public benchmarks described?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"CascadeRec authors report adaptive LLM compute allocation in production","summary":"The authors present CascadeRec, a two-tier LLM reranking system deployed for content recommendations at a financial planning and analysis SaaS provider. A lightweight Tier-1 model handles every request; a router sends selected requests to a more expensive Tier-2 procedure when extra inference-time compute is likely to help. The authors say the system’s segment-gated approach reflects an important complication: applying the expensive tier uniformly can lower quality for users with sparse interaction histories, and escalating cold users can make the loss larger. In public-benchmark tests, uniform LLM reranking lowered NDCG@10 for sparse-history users by up to 30%; its only statistically significant gain was on the segment with the longest histories. Which segments benefit from escalation depends on the domain, the authors report. In a production A/B experiment lasting over a month, CascadeRec reduced inference cost by about 10% compared with uniform use of the expensive tier, with minimal latency impact and no quality regression observed to date. The authors also describe inter-tenant load imbalance and handling LLM output-parsing failures. Tenant data cannot be released, so the public-benchmark results provide a separate view of the pipeline, not a direct disclosure of its production data.","question":"How does the segment-level benefit of escalation vary across domains beyond the public benchmarks described?","claims":[{"text":"The authors describe CascadeRec as a deployed two-tier LLM reranking system that routes selected requests to a more expensive procedure.","excerpt":"A lightweight Tier-1 model reranks every request. A router escalates only the requests where extra inference-time compute is likely to help","support":"source-stated"},{"text":"The authors report that uniform LLM reranking can lower NDCG@10 for sparse-history users, and that escalating cold users can worsen the loss.","excerpt":"uniform LLM reranking lowers NDCG@10 for sparse-history users by up to 30%, and its only statistically significant gain appears on the segment with the longest histories. Escalating cold users can make the loss larger instead of repairing it.","support":"source-stated"},{"text":"The authors report about a 10% inference-cost reduction in the production A/B experiment compared with uniform use of the expensive tier, without observed quality regression to date.","excerpt":"Compared with uniform use of the expensive tier, CascadeRec has reduced inference cost by about 10%, with minimal latency impact and no quality regression observed to date.","support":"source-stated"},{"text":"The authors say which segments benefit from escalation depends on the domain, motivating their segment-gated production rollout.","excerpt":"Which segments benefit from escalation depends on the domain, and this motivates the segment-gated rollout we use in production.","support":"source-stated"},{"text":"The authors say tenant data cannot be released and report evaluating the pipeline on public benchmarks instead.","excerpt":"Tenant data cannot be released, so we also evaluate the pipeline on public benchmarks (MovieLens-1M and Amazon-Electronics).","support":"source-stated"}],"limitations":["Tenant data cannot be released; the abstract reports public-benchmark evaluation alongside the production experiment.","The production result is described as an A/B experiment that had run for over a month; the authors report no quality regression observed to date, not a guarantee of future quality."],"mediaId":"a96850b6d2c6219bd4bd0d0ef453ced1d4c83efff9b2471af75d1d48c945e4a1","correctionReason":null,"id":"113fff2c94eab6f91ee6d7e9cdc3881947957d3ff0ad1992f6d3e0b6afa458b0","publicationId":"80e71ea4-e0d1-414c-895e-04ffef8f9553","version":1,"sourceUrl":"https://arxiv.org/abs/2611.08566","sourceTitle":"CascadeRec: Deployed Adaptive LLM Compute Allocation for Multi-Tenant Content Recommendation","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-05","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-09T05:07:26.635Z","createdAt":"2026-11-12T23:04:52.315Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"a96850b6d2c6219bd4bd0d0ef453ced1d4c83efff9b2471af75d1d48c945e4a1","kind":"article","url":"https://arxiv.org/abs/2611.08566","sourceUrl":"https://arxiv.org/abs/2611.08566","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-12T23:03:53.318Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.08566","checkedAt":"2026-11-13T01:12:08.137Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v6","mediaId":"a96850b6d2c6219bd4bd0d0ef453ced1d4c83efff9b2471af75d1d48c945e4a1","confidence":0.9400000000000001,"provider":"typesafe","model":"jev-1.13.0","inputHash":"1d528f9f99b340c0add38e2eb8cd755fc7f52659241e28d503115701d9d532ac","decidedAt":"2026-11-12T23:03:56.122Z"},"provenance":{"textHash":"48ba715962562aebde1f4fb30bd7c9b660bd9a3f7d2e4e4f46d84a1ec4e8c312","observationId":"e8d70c01854d61004ed6f47ee7227f391b36a8834cb82cedeef5b5dc4c36dd34","retrievedAt":"2026-11-12T23:03:52.995Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"05a05bb0-724f-4c61-8e63-33a6f1f1e4d7"},"reviewer":{"model":"gpt-6-luna","runId":"1e48efe4-5eed-46ae-8cf9-7e245ce3060a"}},"audit":{"status":"passed","reviewedAt":"2026-11-12T23:04:52.315Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Zhao, Kai","url":null,"kind":"unknown"},{"name":"Zhang, Guilin","url":null,"kind":"unknown"},{"name":"Cheng, Xiang","url":null,"kind":"unknown"},{"name":"Cui, Xiquan","url":null,"kind":"unknown"},{"name":"Heng, Henry","url":null,"kind":"unknown"},{"name":"Chu, Xu","url":null,"kind":"unknown"},{"name":"Blanken, Alet","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Zhao, Kai","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zhao, Kai","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08566","locator":"head/meta[name=author|citation_author]"},"assertionId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-13T01:12:08.137Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a","sourceUrl":"https://arxiv.org/abs/2611.08566","contentSha256":"167ccd5e482acb6c48c74cd73888cf4dd17f3557f6f490c94984b973b98c352f","textHash":"48ba715962562aebde1f4fb30bd7c9b660bd9a3f7d2e4e4f46d84a1ec4e8c312","observedAt":"2026-11-13T01:12:08.137Z"}},{"actor":{"name":"Zhang, Guilin","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zhang, Guilin","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08566","locator":"head/meta[name=author|citation_author]"},"assertionId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-13T01:12:08.137Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a","sourceUrl":"https://arxiv.org/abs/2611.08566","contentSha256":"167ccd5e482acb6c48c74cd73888cf4dd17f3557f6f490c94984b973b98c352f","textHash":"48ba715962562aebde1f4fb30bd7c9b660bd9a3f7d2e4e4f46d84a1ec4e8c312","observedAt":"2026-11-13T01:12:08.137Z"}},{"actor":{"name":"Cheng, Xiang","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Cheng, Xiang","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08566","locator":"head/meta[name=author|citation_author]"},"assertionId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-13T01:12:08.137Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a","sourceUrl":"https://arxiv.org/abs/2611.08566","contentSha256":"167ccd5e482acb6c48c74cd73888cf4dd17f3557f6f490c94984b973b98c352f","textHash":"48ba715962562aebde1f4fb30bd7c9b660bd9a3f7d2e4e4f46d84a1ec4e8c312","observedAt":"2026-11-13T01:12:08.137Z"}},{"actor":{"name":"Cui, Xiquan","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Cui, Xiquan","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08566","locator":"head/meta[name=author|citation_author]"},"assertionId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-13T01:12:08.137Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a","sourceUrl":"https://arxiv.org/abs/2611.08566","contentSha256":"167ccd5e482acb6c48c74cd73888cf4dd17f3557f6f490c94984b973b98c352f","textHash":"48ba715962562aebde1f4fb30bd7c9b660bd9a3f7d2e4e4f46d84a1ec4e8c312","observedAt":"2026-11-13T01:12:08.137Z"}},{"actor":{"name":"Heng, Henry","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Heng, Henry","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08566","locator":"head/meta[name=author|citation_author]"},"assertionId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-13T01:12:08.137Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a","sourceUrl":"https://arxiv.org/abs/2611.08566","contentSha256":"167ccd5e482acb6c48c74cd73888cf4dd17f3557f6f490c94984b973b98c352f","textHash":"48ba715962562aebde1f4fb30bd7c9b660bd9a3f7d2e4e4f46d84a1ec4e8c312","observedAt":"2026-11-13T01:12:08.137Z"}},{"actor":{"name":"Chu, Xu","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Chu, Xu","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08566","locator":"head/meta[name=author|citation_author]"},"assertionId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-13T01:12:08.137Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a","sourceUrl":"https://arxiv.org/abs/2611.08566","contentSha256":"167ccd5e482acb6c48c74cd73888cf4dd17f3557f6f490c94984b973b98c352f","textHash":"48ba715962562aebde1f4fb30bd7c9b660bd9a3f7d2e4e4f46d84a1ec4e8c312","observedAt":"2026-11-13T01:12:08.137Z"}},{"actor":{"name":"Blanken, Alet","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Blanken, Alet","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08566","locator":"head/meta[name=author|citation_author]"},"assertionId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a:participation:6","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-13T01:12:08.137Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a","sourceUrl":"https://arxiv.org/abs/2611.08566","contentSha256":"167ccd5e482acb6c48c74cd73888cf4dd17f3557f6f490c94984b973b98c352f","textHash":"48ba715962562aebde1f4fb30bd7c9b660bd9a3f7d2e4e4f46d84a1ec4e8c312","observedAt":"2026-11-13T01:12:08.137Z"}}],"participationCoverage":{"total":7,"returned":7,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.08566","snapshotId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a","contentSha256":"167ccd5e482acb6c48c74cd73888cf4dd17f3557f6f490c94984b973b98c352f","textHash":"48ba715962562aebde1f4fb30bd7c9b660bd9a3f7d2e4e4f46d84a1ec4e8c312","observedAt":"2026-11-13T01:12:08.137Z","basis":"publisher-page-metadata","sourceLinks":47,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/113fff2c94eab6f91ee6d7e9cdc3881947957d3ff0ad1992f6d3e0b6afa458b0/source"}},"canonicalUrl":"https://escapemeridian.com/readings/cascaderec-authors-report-adaptive-llm-compute-allocation-in-113fff2c","formats":{"html":"/readings/cascaderec-authors-report-adaptive-llm-compute-allocation-in-113fff2c","markdown":"/readings/cascaderec-authors-report-adaptive-llm-compute-allocation-in-113fff2c.md","json":"/readings/cascaderec-authors-report-adaptive-llm-compute-allocation-in-113fff2c.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-llm","name":"Large language model","explanation":"A language model with many learned numerical settings, used to process and generate language.","referenceUrl":"https://developers.google.com/machine-learning/glossary#large-language-model"},{"id":"term-test-time-compute","name":"Test-time compute","explanation":"Extra computation a model spends while answering, for example by reasoning longer, rather than during training.","referenceUrl":"https://arxiv.org/abs/2408.03314"},{"id":"term-benchmark","name":"Benchmark","explanation":"A defined set of tests used to compare systems. Results depend on the tasks and measures selected.","referenceUrl":"https://arxiv.org/abs/2211.09110"},{"id":"term-statistical-significance","name":"Statistical significance","explanation":"A test of whether a result is unlikely to be due to chance alone; it does not say how large or important an effect is.","referenceUrl":"https://en.wikipedia.org/wiki/Statistical_significance"},{"id":"term-latency","name":"Latency","explanation":"The delay between sending a request and receiving the result.","referenceUrl":"https://developers.google.com/machine-learning/glossary#latency"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"113fff2c94eab6f91ee6d7e9cdc3881947957d3ff0ad1992f6d3e0b6afa458b0","revision":"80e71ea4-e0d1-414c-895e-04ffef8f9553","revisionKind":"publication-id","title":"CascadeRec authors report adaptive LLM compute allocation in production","url":"https://escapemeridian.com/readings/cascaderec-authors-report-adaptive-llm-compute-allocation-in-113fff2c"},"source":{"type":"source","id":"https://arxiv.org/abs/2611.08566","revision":"48ba715962562aebde1f4fb30bd7c9b660bd9a3f7d2e4e4f46d84a1ec4e8c312","revisionKind":"sha256-source-text","title":"CascadeRec: Deployed Adaptive LLM Compute Allocation for Multi-Tenant Content Recommendation","url":"https://arxiv.org/abs/2611.08566"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"113fff2c94eab6f91ee6d7e9cdc3881947957d3ff0ad1992f6d3e0b6afa458b0","title":"CascadeRec authors report adaptive LLM compute allocation in production","argument":"The authors present CascadeRec, a two-tier LLM reranking system deployed for content recommendations at a financial planning and analysis SaaS provider. A lightweight Tier-1 model handles every request; a router sends selected requests to a more expensive Tier-2 procedure when extra inference-time compute is likely to help. The authors say the system’s segment-gated approach reflects an important complication: applying the expensive tier uniformly can lower quality for users with sparse interaction histories, and escalating cold users can make the loss larger. In public-benchmark tests, uniform LLM reranking lowered NDCG@10 for sparse-history users by up to 30%; its only statistically significant gain was on the segment with the longest histories. Which segments benefit from escalation depends on the domain, the authors report. In a production A/B experiment lasting over a month, CascadeRec reduced inference cost by about 10% compared with uniform use of the expensive tier, with minimal latency impact and no quality regression observed to date. The authors also describe inter-tenant load imbalance and handling LLM output-parsing failures. Tenant data cannot be released, so the public-benchmark results provide a separate view of the pipeline, not a direct disclosure of its production data.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"Tenant data cannot be released; the abstract reports public-benchmark evaluation alongside the production experiment. The production result is described as an A/B experiment that had run for over a month; the authors report no quality regression observed to date, not a guarantee of future quality.","speaker":{"name":"Zhao, Kai, Zhang, Guilin, Cheng, Xiang, Cui, Xiquan, Heng, Henry, Chu, Xu, Blanken, Alet","roleAtSource":"Article author"},"source":{"title":"CascadeRec: Deployed Adaptive LLM Compute Allocation for Multi-Tenant Content Recommendation","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.08566","publishedAt":"2026-11-05","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors describe CascadeRec as a deployed two-tier LLM reranking system that routes selected requests to a more expensive procedure.","excerpt":"A lightweight Tier-1 model reranks every request. A router escalates only the requests where extra inference-time compute is likely to help","url":"https://arxiv.org/abs/2611.08566","relationship":"Source-stated"},{"text":"The authors report that uniform LLM reranking can lower NDCG@10 for sparse-history users, and that escalating cold users can worsen the loss.","excerpt":"uniform LLM reranking lowers NDCG@10 for sparse-history users by up to 30%, and its only statistically significant gain appears on the segment with the longest histories. Escalating cold users can make the loss larger instead of repairing it.","url":"https://arxiv.org/abs/2611.08566","relationship":"Source-stated"},{"text":"The authors report about a 10% inference-cost reduction in the production A/B experiment compared with uniform use of the expensive tier, without observed quality regression to date.","excerpt":"Compared with uniform use of the expensive tier, CascadeRec has reduced inference cost by about 10%, with minimal latency impact and no quality regression observed to date.","url":"https://arxiv.org/abs/2611.08566","relationship":"Source-stated"},{"text":"The authors say which segments benefit from escalation depends on the domain, motivating their segment-gated production rollout.","excerpt":"Which segments benefit from escalation depends on the domain, and this motivates the segment-gated rollout we use in production.","url":"https://arxiv.org/abs/2611.08566","relationship":"Source-stated"},{"text":"The authors say tenant data cannot be released and report evaluating the pipeline on public benchmarks instead.","excerpt":"Tenant data cannot be released, so we also evaluate the pipeline on public benchmarks (MovieLens-1M and Amazon-Electronics).","url":"https://arxiv.org/abs/2611.08566","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.08566","checkedAt":"2026-11-13T01:12:08.137Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How does the segment-level benefit of escalation vary across domains beyond the public benchmarks described?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"CascadeRec authors report adaptive LLM compute allocation in production","summary":"The authors present CascadeRec, a two-tier LLM reranking system deployed for content recommendations at a financial planning and analysis SaaS provider. A lightweight Tier-1 model handles every request; a router sends selected requests to a more expensive Tier-2 procedure when extra inference-time compute is likely to help. The authors say the system’s segment-gated approach reflects an important complication: applying the expensive tier uniformly can lower quality for users with sparse interaction histories, and escalating cold users can make the loss larger. In public-benchmark tests, uniform LLM reranking lowered NDCG@10 for sparse-history users by up to 30%; its only statistically significant gain was on the segment with the longest histories. Which segments benefit from escalation depends on the domain, the authors report. In a production A/B experiment lasting over a month, CascadeRec reduced inference cost by about 10% compared with uniform use of the expensive tier, with minimal latency impact and no quality regression observed to date. The authors also describe inter-tenant load imbalance and handling LLM output-parsing failures. Tenant data cannot be released, so the public-benchmark results provide a separate view of the pipeline, not a direct disclosure of its production data.","question":"How does the segment-level benefit of escalation vary across domains beyond the public benchmarks described?","claims":[{"text":"The authors describe CascadeRec as a deployed two-tier LLM reranking system that routes selected requests to a more expensive procedure.","excerpt":"A lightweight Tier-1 model reranks every request. A router escalates only the requests where extra inference-time compute is likely to help","support":"source-stated"},{"text":"The authors report that uniform LLM reranking can lower NDCG@10 for sparse-history users, and that escalating cold users can worsen the loss.","excerpt":"uniform LLM reranking lowers NDCG@10 for sparse-history users by up to 30%, and its only statistically significant gain appears on the segment with the longest histories. Escalating cold users can make the loss larger instead of repairing it.","support":"source-stated"},{"text":"The authors report about a 10% inference-cost reduction in the production A/B experiment compared with uniform use of the expensive tier, without observed quality regression to date.","excerpt":"Compared with uniform use of the expensive tier, CascadeRec has reduced inference cost by about 10%, with minimal latency impact and no quality regression observed to date.","support":"source-stated"},{"text":"The authors say which segments benefit from escalation depends on the domain, motivating their segment-gated production rollout.","excerpt":"Which segments benefit from escalation depends on the domain, and this motivates the segment-gated rollout we use in production.","support":"source-stated"},{"text":"The authors say tenant data cannot be released and report evaluating the pipeline on public benchmarks instead.","excerpt":"Tenant data cannot be released, so we also evaluate the pipeline on public benchmarks (MovieLens-1M and Amazon-Electronics).","support":"source-stated"}],"limitations":["Tenant data cannot be released; the abstract reports public-benchmark evaluation alongside the production experiment.","The production result is described as an A/B experiment that had run for over a month; the authors report no quality regression observed to date, not a guarantee of future quality."],"mediaId":"a96850b6d2c6219bd4bd0d0ef453ced1d4c83efff9b2471af75d1d48c945e4a1","correctionReason":null,"id":"113fff2c94eab6f91ee6d7e9cdc3881947957d3ff0ad1992f6d3e0b6afa458b0","publicationId":"80e71ea4-e0d1-414c-895e-04ffef8f9553","version":1,"sourceUrl":"https://arxiv.org/abs/2611.08566","sourceTitle":"CascadeRec: Deployed Adaptive LLM Compute Allocation for Multi-Tenant Content Recommendation","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-05","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-09T05:07:26.635Z","createdAt":"2026-11-12T23:04:52.315Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"a96850b6d2c6219bd4bd0d0ef453ced1d4c83efff9b2471af75d1d48c945e4a1","kind":"article","url":"https://arxiv.org/abs/2611.08566","sourceUrl":"https://arxiv.org/abs/2611.08566","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-12T23:03:53.318Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.08566","checkedAt":"2026-11-13T01:12:08.137Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v6","mediaId":"a96850b6d2c6219bd4bd0d0ef453ced1d4c83efff9b2471af75d1d48c945e4a1","confidence":0.9400000000000001,"provider":"typesafe","model":"jev-1.13.0","inputHash":"1d528f9f99b340c0add38e2eb8cd755fc7f52659241e28d503115701d9d532ac","decidedAt":"2026-11-12T23:03:56.122Z"},"provenance":{"textHash":"48ba715962562aebde1f4fb30bd7c9b660bd9a3f7d2e4e4f46d84a1ec4e8c312","observationId":"e8d70c01854d61004ed6f47ee7227f391b36a8834cb82cedeef5b5dc4c36dd34","retrievedAt":"2026-11-12T23:03:52.995Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"05a05bb0-724f-4c61-8e63-33a6f1f1e4d7"},"reviewer":{"model":"gpt-6-luna","runId":"1e48efe4-5eed-46ae-8cf9-7e245ce3060a"}},"audit":{"status":"passed","reviewedAt":"2026-11-12T23:04:52.315Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Zhao, Kai","url":null,"kind":"unknown"},{"name":"Zhang, Guilin","url":null,"kind":"unknown"},{"name":"Cheng, Xiang","url":null,"kind":"unknown"},{"name":"Cui, Xiquan","url":null,"kind":"unknown"},{"name":"Heng, Henry","url":null,"kind":"unknown"},{"name":"Chu, Xu","url":null,"kind":"unknown"},{"name":"Blanken, Alet","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Zhao, Kai","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zhao, Kai","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08566","locator":"head/meta[name=author|citation_author]"},"assertionId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-13T01:12:08.137Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a","sourceUrl":"https://arxiv.org/abs/2611.08566","contentSha256":"167ccd5e482acb6c48c74cd73888cf4dd17f3557f6f490c94984b973b98c352f","textHash":"48ba715962562aebde1f4fb30bd7c9b660bd9a3f7d2e4e4f46d84a1ec4e8c312","observedAt":"2026-11-13T01:12:08.137Z"}},{"actor":{"name":"Zhang, Guilin","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Zhang, Guilin","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08566","locator":"head/meta[name=author|citation_author]"},"assertionId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-13T01:12:08.137Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a","sourceUrl":"https://arxiv.org/abs/2611.08566","contentSha256":"167ccd5e482acb6c48c74cd73888cf4dd17f3557f6f490c94984b973b98c352f","textHash":"48ba715962562aebde1f4fb30bd7c9b660bd9a3f7d2e4e4f46d84a1ec4e8c312","observedAt":"2026-11-13T01:12:08.137Z"}},{"actor":{"name":"Cheng, Xiang","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Cheng, Xiang","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08566","locator":"head/meta[name=author|citation_author]"},"assertionId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-13T01:12:08.137Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a","sourceUrl":"https://arxiv.org/abs/2611.08566","contentSha256":"167ccd5e482acb6c48c74cd73888cf4dd17f3557f6f490c94984b973b98c352f","textHash":"48ba715962562aebde1f4fb30bd7c9b660bd9a3f7d2e4e4f46d84a1ec4e8c312","observedAt":"2026-11-13T01:12:08.137Z"}},{"actor":{"name":"Cui, Xiquan","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Cui, Xiquan","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08566","locator":"head/meta[name=author|citation_author]"},"assertionId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-13T01:12:08.137Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a","sourceUrl":"https://arxiv.org/abs/2611.08566","contentSha256":"167ccd5e482acb6c48c74cd73888cf4dd17f3557f6f490c94984b973b98c352f","textHash":"48ba715962562aebde1f4fb30bd7c9b660bd9a3f7d2e4e4f46d84a1ec4e8c312","observedAt":"2026-11-13T01:12:08.137Z"}},{"actor":{"name":"Heng, Henry","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Heng, Henry","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08566","locator":"head/meta[name=author|citation_author]"},"assertionId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-13T01:12:08.137Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a","sourceUrl":"https://arxiv.org/abs/2611.08566","contentSha256":"167ccd5e482acb6c48c74cd73888cf4dd17f3557f6f490c94984b973b98c352f","textHash":"48ba715962562aebde1f4fb30bd7c9b660bd9a3f7d2e4e4f46d84a1ec4e8c312","observedAt":"2026-11-13T01:12:08.137Z"}},{"actor":{"name":"Chu, Xu","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Chu, Xu","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08566","locator":"head/meta[name=author|citation_author]"},"assertionId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a:participation:5","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-13T01:12:08.137Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a","sourceUrl":"https://arxiv.org/abs/2611.08566","contentSha256":"167ccd5e482acb6c48c74cd73888cf4dd17f3557f6f490c94984b973b98c352f","textHash":"48ba715962562aebde1f4fb30bd7c9b660bd9a3f7d2e4e4f46d84a1ec4e8c312","observedAt":"2026-11-13T01:12:08.137Z"}},{"actor":{"name":"Blanken, Alet","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Blanken, Alet","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.08566","locator":"head/meta[name=author|citation_author]"},"assertionId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a:participation:6","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-13T01:12:08.137Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a","sourceUrl":"https://arxiv.org/abs/2611.08566","contentSha256":"167ccd5e482acb6c48c74cd73888cf4dd17f3557f6f490c94984b973b98c352f","textHash":"48ba715962562aebde1f4fb30bd7c9b660bd9a3f7d2e4e4f46d84a1ec4e8c312","observedAt":"2026-11-13T01:12:08.137Z"}}],"participationCoverage":{"total":7,"returned":7,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.08566","snapshotId":"7c8c04db75c6c3973529061778f700848194fd66e9784b2d2d8f8ce77bca7d7a","contentSha256":"167ccd5e482acb6c48c74cd73888cf4dd17f3557f6f490c94984b973b98c352f","textHash":"48ba715962562aebde1f4fb30bd7c9b660bd9a3f7d2e4e4f46d84a1ec4e8c312","observedAt":"2026-11-13T01:12:08.137Z","basis":"publisher-page-metadata","sourceLinks":47,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/113fff2c94eab6f91ee6d7e9cdc3881947957d3ff0ad1992f6d3e0b6afa458b0/source"}}}],"audits":[{"publicationId":"80e71ea4-e0d1-414c-895e-04ffef8f9553","verdict":"passed","findings":[{"dimension":"attribution","status":"pass","explanation":"The draft consistently attributes CascadeRec and its results to the paper's authors (\"The authors present\", \"the authors report\"). The production deployment and the 10% cost figure stay framed as author-reported, not independently verified.","excerpt":"We present CascadeRec, a two-tier LLM reranking cascade deployed in production at a large financial planning and analysis (FP&A) SaaS provider."},{"dimension":"summary","status":"pass","explanation":"The draft leads with the actor and the system, then explains the two tiers and the router in plain terms. It conveys the counterintuitive finding that spending more compute can hurt quality for users with sparse histories, and it gives the cost result with its scope. Technical terms such as NDCG@10 and reranking are kept exactly, consistent with the tooltip approach. The title and question are faithful, and the question follows from the paper's statement about domain dependence.","excerpt":"Running an expensive model on every request is costly, and for users with sparse interaction histories it often lowers quality."},{"dimension":"date","status":"pass","explanation":"The originalPublished date of 2026-11-05 matches the arXiv v1 submission date. It is not the later feed date of 2026-11-09.","excerpt":"[Submitted on 5 Nov 2026]"},{"dimension":"support","status":"pass","explanation":"Each claim's excerpt matches the source verbatim. The draft preserves the key conditions: about 10%, minimal latency impact, no regression observed \"to date\", over a month, and the public benchmarks as a separate view. It invents no benefits or stakeholders. Claim 5's word \"instead\" is slightly looser than the source's \"also evaluate\", but the summary and limitations correctly frame the benchmarks as alongside the production results, so this is not material.","excerpt":"Compared with uniform use of the expensive tier, CascadeRec has reduced inference cost by about 10%, with minimal latency impact and no quality regression observed to date."},{"dimension":"media","status":"pass","explanation":"The selected media is the verified direct arXiv abstract page for this paper, which matches the source URL. It is the most direct way for readers to reach the original paper.","excerpt":null}],"createdAt":"2026-11-12T23:04:52.315Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}