{"id":"229dc3d04a7f06ecbd69874357d0c5d296677c978dec33376e0a01dd553cabfb","title":"arXiv cs.AI: A Night at the LLM Zoo—Generating pretrained LLM weights from data mixtures","argument":"The authors of “A Night at the LLM Zoo” argue that a curated population of large language models can help extend Weight Space Learning—the study and generation of models through their weights—to LLMs. They trained populations of GPT-2 models at three parameter sizes, varying the proportions of web text, books, mathematics, coding and French in the training data. The authors report that variation among these models is structured by those data mixtures. They then conditioned a Flow Matching model on mixture composition to generate complete pretrained LLM weights in Gram-PCA compressed weight space. On held-out mixtures, the generator showed zero-shot generalization; the authors also report that generated models performed similarly to models in their zoo, with minimal changes in downstream test-set perplexity and general capability benchmark scores. The work offers a way to study generating models for data mixtures not used to train the generator, but its evidence comes from the described GPT-2 populations and evaluations. The abstract does not give the size or range of the held-out mixtures, or the detailed results behind its performance comparisons. The authors say they release source code and artifacts to support further work in Weight Space Learning and Generation.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The abstract does not specify the number or range of held-out mixtures, or provide detailed results for the reported performance comparisons. The reported study uses GPT-2 model populations at the parameter sizes and with the data-mixture categories described in the abstract; the abstract does not establish how well the findings extend beyond them.","speaker":{"name":"Biggs, Scott, Soro, Bedionita, Gupta, Saumya, Andreis, Bruno, Hwang, Sung Ju","roleAtSource":"Article author"},"source":{"title":"A Night at the LLM Zoo: Data Conditioned Flow Matching for Generating Pretrained LLM Weights","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.07103","publishedAt":"2026-11-04","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors created GPT-2 model populations with varied proportions of five kinds of training data, at three parameter sizes.","excerpt":"systematically varying the data mixture composition between web text, books, mathematics, coding, and French across populations of 51M, 124M, and 355M parameter GPT-2 Models","url":"https://arxiv.org/abs/2611.07103","relationship":"Source-stated"},{"text":"The authors report that model variation in their zoo is structured by training mixture composition.","excerpt":"the zoo contains diverse models whose variation is structured by training mixture composition","url":"https://arxiv.org/abs/2611.07103","relationship":"Source-stated"},{"text":"The authors conditioned a Flow Matching model on training mixture composition to generate pretrained LLM weights in a compressed weight space.","excerpt":"We embed the training mixture composition to condition a Flow Matching model of complete pretrained LLM weights in Gram-PCA compressed weight space.","url":"https://arxiv.org/abs/2611.07103","relationship":"Source-stated"},{"text":"The authors report zero-shot generalization to held-out mixtures and similar performance to models in their zoo, with minimal changes on the stated evaluations.","excerpt":"By demonstrating zero-shot generalization to held out mixtures, measuring Continuous Ranked Probability Score skill, as well as finding minimal changes in downstream test-set perplexity and general capability benchmark scores","url":"https://arxiv.org/abs/2611.07103","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.07103","checkedAt":"2026-11-09T04:58:06.085Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How broadly does the reported zero-shot generalization hold across held-out data mixtures beyond those evaluated in this study?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: A Night at the LLM Zoo—Generating pretrained LLM weights from data mixtures","summary":"The authors of “A Night at the LLM Zoo” argue that a curated population of large language models can help extend Weight Space Learning—the study and generation of models through their weights—to LLMs. They trained populations of GPT-2 models at three parameter sizes, varying the proportions of web text, books, mathematics, coding and French in the training data. The authors report that variation among these models is structured by those data mixtures. They then conditioned a Flow Matching model on mixture composition to generate complete pretrained LLM weights in Gram-PCA compressed weight space. On held-out mixtures, the generator showed zero-shot generalization; the authors also report that generated models performed similarly to models in their zoo, with minimal changes in downstream test-set perplexity and general capability benchmark scores. The work offers a way to study generating models for data mixtures not used to train the generator, but its evidence comes from the described GPT-2 populations and evaluations. The abstract does not give the size or range of the held-out mixtures, or the detailed results behind its performance comparisons. The authors say they release source code and artifacts to support further work in Weight Space Learning and Generation.","question":"How broadly does the reported zero-shot generalization hold across held-out data mixtures beyond those evaluated in this study?","claims":[{"text":"The authors created GPT-2 model populations with varied proportions of five kinds of training data, at three parameter sizes.","excerpt":"systematically varying the data mixture composition between web text, books, mathematics, coding, and French across populations of 51M, 124M, and 355M parameter GPT-2 Models","support":"source-stated"},{"text":"The authors report that model variation in their zoo is structured by training mixture composition.","excerpt":"the zoo contains diverse models whose variation is structured by training mixture composition","support":"source-stated"},{"text":"The authors conditioned a Flow Matching model on training mixture composition to generate pretrained LLM weights in a compressed weight space.","excerpt":"We embed the training mixture composition to condition a Flow Matching model of complete pretrained LLM weights in Gram-PCA compressed weight space.","support":"source-stated"},{"text":"The authors report zero-shot generalization to held-out mixtures and similar performance to models in their zoo, with minimal changes on the stated evaluations.","excerpt":"By demonstrating zero-shot generalization to held out mixtures, measuring Continuous Ranked Probability Score skill, as well as finding minimal changes in downstream test-set perplexity and general capability benchmark scores","support":"source-stated"}],"limitations":["The abstract does not specify the number or range of held-out mixtures, or provide detailed results for the reported performance comparisons.","The reported study uses GPT-2 model populations at the parameter sizes and with the data-mixture categories described in the abstract; the abstract does not establish how well the findings extend beyond them."],"mediaId":"85c5e44c8ed79ed4164f56b38510662cbff48651002f6ae9205507bf1e52842b","correctionReason":null,"id":"229dc3d04a7f06ecbd69874357d0c5d296677c978dec33376e0a01dd553cabfb","publicationId":"7793042a-593e-4c12-8c90-ad00f8b9be73","version":1,"sourceUrl":"https://arxiv.org/abs/2611.07103","sourceTitle":"A Night at the LLM Zoo: Data Conditioned Flow Matching for Generating Pretrained LLM Weights","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-04","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-06T05:07:13.370Z","createdAt":"2026-11-09T03:54:04.011Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"85c5e44c8ed79ed4164f56b38510662cbff48651002f6ae9205507bf1e52842b","kind":"article","url":"https://arxiv.org/abs/2611.07103","sourceUrl":"https://arxiv.org/abs/2611.07103","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-09T03:53:11.765Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.07103","checkedAt":"2026-11-09T04:58:06.085Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"85c5e44c8ed79ed4164f56b38510662cbff48651002f6ae9205507bf1e52842b","confidence":0.81,"provider":"typesafe","model":"jev-1.13.0","inputHash":"9c55c28e04c4a47138c6f02c567a50b0d507bb9b7bd4f3561fa93bfcab3d8e2d","decidedAt":"2026-11-09T03:53:13.290Z"},"provenance":{"textHash":"41ec07656ad44df7bcbdd10cdc4bac6f699b5c0e9fde1ff9d07aa526a3a56d88","observationId":"71f154218c80a43177a09f603a97d518517b32b259f19e9b8bd772b58272dd29","retrievedAt":"2026-11-09T03:53:11.499Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"bfdfecaa-2c22-41f0-840a-1726490482bf"},"reviewer":{"model":"gpt-6-luna","runId":"8255ad47-1b34-4c65-85fa-27e0f5d6e423"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-09T03:54:04.011Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Biggs, Scott","url":null,"kind":"unknown"},{"name":"Soro, Bedionita","url":null,"kind":"unknown"},{"name":"Gupta, Saumya","url":null,"kind":"unknown"},{"name":"Andreis, Bruno","url":null,"kind":"unknown"},{"name":"Hwang, Sung Ju","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Biggs, Scott","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Biggs, Scott","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.07103","locator":"head/meta[name=author|citation_author]"},"assertionId":"8178441aaf66702a668a7f2b82a92a7b2d4b86f9f424f2bf0334290570759106:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T04:58:06.085Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"8178441aaf66702a668a7f2b82a92a7b2d4b86f9f424f2bf0334290570759106","sourceUrl":"https://arxiv.org/abs/2611.07103","contentSha256":"c675811ff82f2e5ca63919db667b2efb2e6896bfcb863c39a9e63735043340f0","textHash":"41ec07656ad44df7bcbdd10cdc4bac6f699b5c0e9fde1ff9d07aa526a3a56d88","observedAt":"2026-11-09T04:58:06.085Z"}},{"actor":{"name":"Soro, Bedionita","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Soro, Bedionita","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.07103","locator":"head/meta[name=author|citation_author]"},"assertionId":"8178441aaf66702a668a7f2b82a92a7b2d4b86f9f424f2bf0334290570759106:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T04:58:06.085Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"8178441aaf66702a668a7f2b82a92a7b2d4b86f9f424f2bf0334290570759106","sourceUrl":"https://arxiv.org/abs/2611.07103","contentSha256":"c675811ff82f2e5ca63919db667b2efb2e6896bfcb863c39a9e63735043340f0","textHash":"41ec07656ad44df7bcbdd10cdc4bac6f699b5c0e9fde1ff9d07aa526a3a56d88","observedAt":"2026-11-09T04:58:06.085Z"}},{"actor":{"name":"Gupta, Saumya","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Gupta, Saumya","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.07103","locator":"head/meta[name=author|citation_author]"},"assertionId":"8178441aaf66702a668a7f2b82a92a7b2d4b86f9f424f2bf0334290570759106:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T04:58:06.085Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"8178441aaf66702a668a7f2b82a92a7b2d4b86f9f424f2bf0334290570759106","sourceUrl":"https://arxiv.org/abs/2611.07103","contentSha256":"c675811ff82f2e5ca63919db667b2efb2e6896bfcb863c39a9e63735043340f0","textHash":"41ec07656ad44df7bcbdd10cdc4bac6f699b5c0e9fde1ff9d07aa526a3a56d88","observedAt":"2026-11-09T04:58:06.085Z"}},{"actor":{"name":"Andreis, Bruno","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Andreis, Bruno","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.07103","locator":"head/meta[name=author|citation_author]"},"assertionId":"8178441aaf66702a668a7f2b82a92a7b2d4b86f9f424f2bf0334290570759106:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T04:58:06.085Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"8178441aaf66702a668a7f2b82a92a7b2d4b86f9f424f2bf0334290570759106","sourceUrl":"https://arxiv.org/abs/2611.07103","contentSha256":"c675811ff82f2e5ca63919db667b2efb2e6896bfcb863c39a9e63735043340f0","textHash":"41ec07656ad44df7bcbdd10cdc4bac6f699b5c0e9fde1ff9d07aa526a3a56d88","observedAt":"2026-11-09T04:58:06.085Z"}},{"actor":{"name":"Hwang, Sung Ju","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Hwang, Sung Ju","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.07103","locator":"head/meta[name=author|citation_author]"},"assertionId":"8178441aaf66702a668a7f2b82a92a7b2d4b86f9f424f2bf0334290570759106:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T04:58:06.085Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"8178441aaf66702a668a7f2b82a92a7b2d4b86f9f424f2bf0334290570759106","sourceUrl":"https://arxiv.org/abs/2611.07103","contentSha256":"c675811ff82f2e5ca63919db667b2efb2e6896bfcb863c39a9e63735043340f0","textHash":"41ec07656ad44df7bcbdd10cdc4bac6f699b5c0e9fde1ff9d07aa526a3a56d88","observedAt":"2026-11-09T04:58:06.085Z"}}],"participationCoverage":{"total":5,"returned":5,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.07103","snapshotId":"8178441aaf66702a668a7f2b82a92a7b2d4b86f9f424f2bf0334290570759106","contentSha256":"c675811ff82f2e5ca63919db667b2efb2e6896bfcb863c39a9e63735043340f0","textHash":"41ec07656ad44df7bcbdd10cdc4bac6f699b5c0e9fde1ff9d07aa526a3a56d88","observedAt":"2026-11-09T04:58:06.085Z","basis":"publisher-page-metadata","sourceLinks":45,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/229dc3d04a7f06ecbd69874357d0c5d296677c978dec33376e0a01dd553cabfb/source"}},"canonicalUrl":"https://escapemeridian.com/readings/arxiv-cs-ai-a-night-at-the-llm-zoo-generating-pretrained-llm-229dc3d0","formats":{"html":"/readings/arxiv-cs-ai-a-night-at-the-llm-zoo-generating-pretrained-llm-229dc3d0","markdown":"/readings/arxiv-cs-ai-a-night-at-the-llm-zoo-generating-pretrained-llm-229dc3d0.md","json":"/readings/arxiv-cs-ai-a-night-at-the-llm-zoo-generating-pretrained-llm-229dc3d0.json"},"attribution":"Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.","maintainedGuideLinks":[],"readerContext":{"version":"reader-context-v1","attribution":"Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.","terms":[{"id":"term-llm","name":"Large language model","explanation":"A language model with many learned numerical settings, used to process and generate language.","referenceUrl":"https://developers.google.com/machine-learning/glossary#large-language-model"},{"id":"term-pre-training","name":"Pre-training","explanation":"A model's initial training on a large dataset, before further adaptation for particular uses.","referenceUrl":"https://developers.google.com/machine-learning/glossary#pre-training"},{"id":"term-zero-shot","name":"Zero-shot","explanation":"Doing a task without worked examples (zero-shot) or with only a few examples given in the prompt (few-shot).","referenceUrl":"https://developers.google.com/machine-learning/glossary#zero-shot-prompting"},{"id":"term-held-out-set","name":"Held-out set","explanation":"Data kept aside during training and used only to test how well a model works on examples it has not seen.","referenceUrl":"https://developers.google.com/machine-learning/glossary#holdout-data"},{"id":"term-benchmark","name":"Benchmark","explanation":"A defined set of tests used to compare systems. Results depend on the tasks and measures selected.","referenceUrl":"https://arxiv.org/abs/2211.09110"}],"entities":[{"id":"arxiv","name":"arXiv","kind":"organization","context":"arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.","referenceUrls":["https://arxiv.org/","https://info.arxiv.org/help/api/index.html"],"path":"/entities/arxiv","json":"/entities/arxiv.json","markdown":"/entities/arxiv.md"}]},"researchQuestions":[],"withdrawn":false,"references":{"schemaVersion":1,"target":{"type":"reading","id":"229dc3d04a7f06ecbd69874357d0c5d296677c978dec33376e0a01dd553cabfb","revision":"7793042a-593e-4c12-8c90-ad00f8b9be73","revisionKind":"publication-id","title":"arXiv cs.AI: A Night at the LLM Zoo—Generating pretrained LLM weights from data mixtures","url":"https://escapemeridian.com/readings/arxiv-cs-ai-a-night-at-the-llm-zoo-generating-pretrained-llm-229dc3d0"},"source":{"type":"source","id":"https://arxiv.org/abs/2611.07103","revision":"41ec07656ad44df7bcbdd10cdc4bac6f699b5c0e9fde1ff9d07aa526a3a56d88","revisionKind":"sha256-source-text","title":"A Night at the LLM Zoo: Data Conditioned Flow Matching for Generating Pretrained LLM Weights","url":"https://arxiv.org/abs/2611.07103"},"status":"available","items":[],"next":null,"scope":"Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter."},"revisions":[{"id":"229dc3d04a7f06ecbd69874357d0c5d296677c978dec33376e0a01dd553cabfb","title":"arXiv cs.AI: A Night at the LLM Zoo—Generating pretrained LLM weights from data mixtures","argument":"The authors of “A Night at the LLM Zoo” argue that a curated population of large language models can help extend Weight Space Learning—the study and generation of models through their weights—to LLMs. They trained populations of GPT-2 models at three parameter sizes, varying the proportions of web text, books, mathematics, coding and French in the training data. The authors report that variation among these models is structured by those data mixtures. They then conditioned a Flow Matching model on mixture composition to generate complete pretrained LLM weights in Gram-PCA compressed weight space. On held-out mixtures, the generator showed zero-shot generalization; the authors also report that generated models performed similarly to models in their zoo, with minimal changes in downstream test-set perplexity and general capability benchmark scores. The work offers a way to study generating models for data mixtures not used to train the generator, but its evidence comes from the described GPT-2 populations and evaluations. The abstract does not give the size or range of the held-out mixtures, or the detailed results behind its performance comparisons. The authors say they release source code and artifacts to support further work in Weight Space Learning and Generation.","topicIds":["intelligence-and-discovery"],"domain":"science","claimType":"attributed reading","limits":"The abstract does not specify the number or range of held-out mixtures, or provide detailed results for the reported performance comparisons. The reported study uses GPT-2 model populations at the parameter sizes and with the data-mixture categories described in the abstract; the abstract does not establish how well the findings extend beyond them.","speaker":{"name":"Biggs, Scott, Soro, Bedionita, Gupta, Saumya, Andreis, Bruno, Hwang, Sung Ju","roleAtSource":"Article author"},"source":{"title":"A Night at the LLM Zoo: Data Conditioned Flow Matching for Generating Pretrained LLM Weights","publisher":"arXiv cs.AI","format":"Article","url":"https://arxiv.org/abs/2611.07103","publishedAt":"2026-11-04","dateBasis":"Date basis not provided."},"evidence":[{"text":"The authors created GPT-2 model populations with varied proportions of five kinds of training data, at three parameter sizes.","excerpt":"systematically varying the data mixture composition between web text, books, mathematics, coding, and French across populations of 51M, 124M, and 355M parameter GPT-2 Models","url":"https://arxiv.org/abs/2611.07103","relationship":"Source-stated"},{"text":"The authors report that model variation in their zoo is structured by training mixture composition.","excerpt":"the zoo contains diverse models whose variation is structured by training mixture composition","url":"https://arxiv.org/abs/2611.07103","relationship":"Source-stated"},{"text":"The authors conditioned a Flow Matching model on training mixture composition to generate pretrained LLM weights in a compressed weight space.","excerpt":"We embed the training mixture composition to condition a Flow Matching model of complete pretrained LLM weights in Gram-PCA compressed weight space.","url":"https://arxiv.org/abs/2611.07103","relationship":"Source-stated"},{"text":"The authors report zero-shot generalization to held-out mixtures and similar performance to models in their zoo, with minimal changes on the stated evaluations.","excerpt":"By demonstrating zero-shot generalization to held out mixtures, measuring Continuous Ranked Probability Score skill, as well as finding minimal changes in downstream test-set perplexity and general capability benchmark scores","url":"https://arxiv.org/abs/2611.07103","relationship":"Source-stated"}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.07103","checkedAt":"2026-11-09T04:58:06.085Z","items":[],"note":"No suitable media was found by the supported page extraction."},"assumptions":[],"openQuestion":{"attribution":"Open question","text":"How broadly does the reported zero-shot generalization hold across held-out data mixtures beyond those evaluated in this study?"},"whatWouldChangeView":{"attribution":"Change condition not recorded","text":""},"editorialComparisons":[],"live":{"title":"arXiv cs.AI: A Night at the LLM Zoo—Generating pretrained LLM weights from data mixtures","summary":"The authors of “A Night at the LLM Zoo” argue that a curated population of large language models can help extend Weight Space Learning—the study and generation of models through their weights—to LLMs. They trained populations of GPT-2 models at three parameter sizes, varying the proportions of web text, books, mathematics, coding and French in the training data. The authors report that variation among these models is structured by those data mixtures. They then conditioned a Flow Matching model on mixture composition to generate complete pretrained LLM weights in Gram-PCA compressed weight space. On held-out mixtures, the generator showed zero-shot generalization; the authors also report that generated models performed similarly to models in their zoo, with minimal changes in downstream test-set perplexity and general capability benchmark scores. The work offers a way to study generating models for data mixtures not used to train the generator, but its evidence comes from the described GPT-2 populations and evaluations. The abstract does not give the size or range of the held-out mixtures, or the detailed results behind its performance comparisons. The authors say they release source code and artifacts to support further work in Weight Space Learning and Generation.","question":"How broadly does the reported zero-shot generalization hold across held-out data mixtures beyond those evaluated in this study?","claims":[{"text":"The authors created GPT-2 model populations with varied proportions of five kinds of training data, at three parameter sizes.","excerpt":"systematically varying the data mixture composition between web text, books, mathematics, coding, and French across populations of 51M, 124M, and 355M parameter GPT-2 Models","support":"source-stated"},{"text":"The authors report that model variation in their zoo is structured by training mixture composition.","excerpt":"the zoo contains diverse models whose variation is structured by training mixture composition","support":"source-stated"},{"text":"The authors conditioned a Flow Matching model on training mixture composition to generate pretrained LLM weights in a compressed weight space.","excerpt":"We embed the training mixture composition to condition a Flow Matching model of complete pretrained LLM weights in Gram-PCA compressed weight space.","support":"source-stated"},{"text":"The authors report zero-shot generalization to held-out mixtures and similar performance to models in their zoo, with minimal changes on the stated evaluations.","excerpt":"By demonstrating zero-shot generalization to held out mixtures, measuring Continuous Ranked Probability Score skill, as well as finding minimal changes in downstream test-set perplexity and general capability benchmark scores","support":"source-stated"}],"limitations":["The abstract does not specify the number or range of held-out mixtures, or provide detailed results for the reported performance comparisons.","The reported study uses GPT-2 model populations at the parameter sizes and with the data-mixture categories described in the abstract; the abstract does not establish how well the findings extend beyond them."],"mediaId":"85c5e44c8ed79ed4164f56b38510662cbff48651002f6ae9205507bf1e52842b","correctionReason":null,"id":"229dc3d04a7f06ecbd69874357d0c5d296677c978dec33376e0a01dd553cabfb","publicationId":"7793042a-593e-4c12-8c90-ad00f8b9be73","version":1,"sourceUrl":"https://arxiv.org/abs/2611.07103","sourceTitle":"A Night at the LLM Zoo: Data Conditioned Flow Matching for Generating Pretrained LLM Weights","publisher":"arXiv cs.AI","publishedAt":null,"originalPublished":{"value":"2026-11-04","precision":"day","basis":"publisher_citation_date"},"firstObservedAt":"2026-11-06T05:07:13.370Z","createdAt":"2026-11-09T03:54:04.011Z","historical":true,"entityIds":["arxiv"],"topicIds":["intelligence-and-discovery"],"media":[{"id":"85c5e44c8ed79ed4164f56b38510662cbff48651002f6ae9205507bf1e52842b","kind":"article","url":"https://arxiv.org/abs/2611.07103","sourceUrl":"https://arxiv.org/abs/2611.07103","label":"Read original","provenance":"source_link","verification":{"status":"verified","checkedAt":"2026-11-09T03:53:11.765Z","method":"acquired_source","contentType":"text/html"},"recommendationReason":"Recommended by Jev for understanding this source; checked by Luna."}],"sourceMedia":{"status":"not_found","sourceUrl":"https://arxiv.org/abs/2611.07103","checkedAt":"2026-11-09T04:58:06.085Z","items":[],"note":"No suitable media was found by the supported page extraction."},"editorialDecision":{"outcome":"accept","reasonCodes":["material_development"],"rubricVersion":"stream-admission-v5","mediaId":"85c5e44c8ed79ed4164f56b38510662cbff48651002f6ae9205507bf1e52842b","confidence":0.81,"provider":"typesafe","model":"jev-1.13.0","inputHash":"9c55c28e04c4a47138c6f02c567a50b0d507bb9b7bd4f3561fa93bfcab3d8e2d","decidedAt":"2026-11-09T03:53:13.290Z"},"provenance":{"textHash":"41ec07656ad44df7bcbdd10cdc4bac6f699b5c0e9fde1ff9d07aa526a3a56d88","observationId":"71f154218c80a43177a09f603a97d518517b32b259f19e9b8bd772b58272dd29","retrievedAt":"2026-11-09T03:53:11.499Z","kind":"publisher-page","preparer":{"model":"gpt-6-luna","runId":"bfdfecaa-2c22-41f0-840a-1726490482bf"},"reviewer":{"model":"gpt-6-luna","runId":"8255ad47-1b34-4c65-85fa-27e0f5d6e423"}},"audit":{"status":"uncertain","reviewedAt":"2026-11-09T03:54:04.011Z","model":"anthropic/claude-opus-5.5"},"sourceAttribution":{"authors":[{"name":"Biggs, Scott","url":null,"kind":"unknown"},{"name":"Soro, Bedionita","url":null,"kind":"unknown"},{"name":"Gupta, Saumya","url":null,"kind":"unknown"},{"name":"Andreis, Bruno","url":null,"kind":"unknown"},{"name":"Hwang, Sung Ju","url":null,"kind":"unknown"}],"authorRole":"article-author","hosts":[],"status":"stated","authorRoster":{"truncated":false,"omitted":0,"assertionGroupsOmitted":0},"participation":[{"actor":{"name":"Biggs, Scott","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Biggs, Scott","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.07103","locator":"head/meta[name=author|citation_author]"},"assertionId":"8178441aaf66702a668a7f2b82a92a7b2d4b86f9f424f2bf0334290570759106:participation:0","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T04:58:06.085Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"8178441aaf66702a668a7f2b82a92a7b2d4b86f9f424f2bf0334290570759106","sourceUrl":"https://arxiv.org/abs/2611.07103","contentSha256":"c675811ff82f2e5ca63919db667b2efb2e6896bfcb863c39a9e63735043340f0","textHash":"41ec07656ad44df7bcbdd10cdc4bac6f699b5c0e9fde1ff9d07aa526a3a56d88","observedAt":"2026-11-09T04:58:06.085Z"}},{"actor":{"name":"Soro, Bedionita","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Soro, Bedionita","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.07103","locator":"head/meta[name=author|citation_author]"},"assertionId":"8178441aaf66702a668a7f2b82a92a7b2d4b86f9f424f2bf0334290570759106:participation:1","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T04:58:06.085Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"8178441aaf66702a668a7f2b82a92a7b2d4b86f9f424f2bf0334290570759106","sourceUrl":"https://arxiv.org/abs/2611.07103","contentSha256":"c675811ff82f2e5ca63919db667b2efb2e6896bfcb863c39a9e63735043340f0","textHash":"41ec07656ad44df7bcbdd10cdc4bac6f699b5c0e9fde1ff9d07aa526a3a56d88","observedAt":"2026-11-09T04:58:06.085Z"}},{"actor":{"name":"Gupta, Saumya","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Gupta, Saumya","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.07103","locator":"head/meta[name=author|citation_author]"},"assertionId":"8178441aaf66702a668a7f2b82a92a7b2d4b86f9f424f2bf0334290570759106:participation:2","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T04:58:06.085Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"8178441aaf66702a668a7f2b82a92a7b2d4b86f9f424f2bf0334290570759106","sourceUrl":"https://arxiv.org/abs/2611.07103","contentSha256":"c675811ff82f2e5ca63919db667b2efb2e6896bfcb863c39a9e63735043340f0","textHash":"41ec07656ad44df7bcbdd10cdc4bac6f699b5c0e9fde1ff9d07aa526a3a56d88","observedAt":"2026-11-09T04:58:06.085Z"}},{"actor":{"name":"Andreis, Bruno","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Andreis, Bruno","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.07103","locator":"head/meta[name=author|citation_author]"},"assertionId":"8178441aaf66702a668a7f2b82a92a7b2d4b86f9f424f2bf0334290570759106:participation:3","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T04:58:06.085Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"8178441aaf66702a668a7f2b82a92a7b2d4b86f9f424f2bf0334290570759106","sourceUrl":"https://arxiv.org/abs/2611.07103","contentSha256":"c675811ff82f2e5ca63919db667b2efb2e6896bfcb863c39a9e63735043340f0","textHash":"41ec07656ad44df7bcbdd10cdc4bac6f699b5c0e9fde1ff9d07aa526a3a56d88","observedAt":"2026-11-09T04:58:06.085Z"}},{"actor":{"name":"Hwang, Sung Ju","url":null,"kind":"unknown","actorId":null},"role":"author","scope":{"kind":"work-credit","locator":null},"literalCredit":"Hwang, Sung Ju","status":"stated","provenance":{"basis":"head-meta","sourceUrl":"https://arxiv.org/abs/2611.07103","locator":"head/meta[name=author|citation_author]"},"assertionId":"8178441aaf66702a668a7f2b82a92a7b2d4b86f9f424f2bf0334290570759106:participation:4","identity":{"version":"entity-identity-v1","status":"unresolved","actorId":null,"canonicalActorId":null,"candidateActorIds":[],"assertions":[],"sourceObservedAt":"2026-11-09T04:58:06.085Z","knowledgeAsOf":null},"sourceVersion":{"snapshotId":"8178441aaf66702a668a7f2b82a92a7b2d4b86f9f424f2bf0334290570759106","sourceUrl":"https://arxiv.org/abs/2611.07103","contentSha256":"c675811ff82f2e5ca63919db667b2efb2e6896bfcb863c39a9e63735043340f0","textHash":"41ec07656ad44df7bcbdd10cdc4bac6f699b5c0e9fde1ff9d07aa526a3a56d88","observedAt":"2026-11-09T04:58:06.085Z"}}],"participationCoverage":{"total":5,"returned":5,"complete":true,"fullAssertions":"source-metadata-endpoint"},"distribution":null,"publisher":null,"sourceUrl":"https://arxiv.org/abs/2611.07103","snapshotId":"8178441aaf66702a668a7f2b82a92a7b2d4b86f9f424f2bf0334290570759106","contentSha256":"c675811ff82f2e5ca63919db667b2efb2e6896bfcb863c39a9e63735043340f0","textHash":"41ec07656ad44df7bcbdd10cdc4bac6f699b5c0e9fde1ff9d07aa526a3a56d88","observedAt":"2026-11-09T04:58:06.085Z","basis":"publisher-page-metadata","sourceLinks":45,"linksTruncated":false,"metadataUrl":"/api/pipeline/stream/229dc3d04a7f06ecbd69874357d0c5d296677c978dec33376e0a01dd553cabfb/source"}}}],"audits":[{"publicationId":"7793042a-593e-4c12-8c90-ad00f8b9be73","verdict":"uncertain","findings":[{"dimension":"attribution","status":"pass","explanation":"The reading credits the arXiv preprint's named authors and publisher, and mostly frames results as what the authors report. One sentence states 'the generator showed zero-shot generalization' without attribution; the summary finding addresses this.","excerpt":"Authors:Scott Biggs, Bedionita Soro, Saumya Gupta, Bruno Andreis, Sung Ju Hwang"},{"dimension":"summary","status":"uncertain","explanation":"The draft is faithful but hard for a non-specialist to follow. It never says what model 'weights' are (the learned numerical parameters). It leaves Flow Matching, Gram-PCA, zero-shot and perplexity unexplained. It also drops the abstract's central 'ex nihilo' idea: models are generated without being trained. Smallest repairs: define weights in plain words, add term tooltips, state that the generated models are produced without training, and attribute the zero-shot sentence to the authors ('the authors report').","excerpt":"we show that our generator generalizes on the simplex of data mixtures to produce models that perform similarly to those in the zoo ex nihilo"},{"dimension":"date","status":"pass","explanation":"originalPublished 2026-11-04 matches the v1 submission date on arXiv. It is not the later feed date.","excerpt":"[Submitted on 4 Nov 2026]"},{"dimension":"support","status":"pass","explanation":"Each claim and excerpt matches the abstract. The limitations correctly note what the abstract does not specify. The question is open rather than leading, and no unsupported benefits or costs are implied.","excerpt":"finding minimal changes in downstream test-set perplexity and general capability benchmark scores"},{"dimension":"media","status":"pass","explanation":"The selected media is the verified link to the arXiv abstract page itself, which is the most direct way to reach the source.","excerpt":null}],"createdAt":"2026-11-09T03:54:04.011Z","model":"anthropic/claude-opus-5.5"}],"correctionUrl":"https://escapemeridian.com/corrections/"}