Can preparing a reference library once make repeated questions cheaper?
The Cartridges researchers train a compact model memory for a fixed corpus, then reuse it across questions. They report lower serving memory and higher throughput than putting the full corpus in context on tested long-context tasks.