# ArXiv authors Sharma, Saini and Behl argue data splits can decide encrypted-traffic scores [Canonical reading](https://escapemeridian.com/readings/arxiv-authors-sharma-saini-and-behl-argue-data-splits-can-1b98261a) Mayan Sharma, Kriti Saini and Devansh Behl argue that encrypted traffic classification scores can depend sharply on how data is divided for evaluation. On one public corpus, the same model and code scored macro-F1 0.992 with a commonly used split and 0.567 when flows from one packet capture stayed together. The authors say the problem is not random splitting alone: on a month of backbone QUIC traffic, server-disjoint splitting reduced macro-F1, while three other approaches were statistically indistinguishable from session-disjoint splitting. Which split matters, they report, depends on how a corpus was collected, and may not be apparent from its labels. Their proposed diagnostic uses single-identifier probes before model training; a decision tree using only capture identifiers scored 0.983 with a random split and 0.000 with session-disjoint splitting. The paper also reports that five pre-registered architectural claims failed, including a finding that removing every metric objective improved rejection of unknown applications. Across corpora, five labelled flows per class recovered 91% of the deficit from failed zero-shot transfer. The authors release splits, results, code and negative findings. These results concern the reported evaluations; the abstract does not establish how well the diagnostic works on other corpora. Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims. ## Original source arXiv cs.AI · Article Original source date: Nov 4, 2026 (2026-11-04). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-09T11:09:12.360Z. First observed: 2026-11-05T05:06:32.020Z. Retrieved: 2026-11-09T11:08:22.678Z. - [The Split Protocol Decides the Result: Re-Evaluating Encrypted Traffic Classification](https://arxiv.org/abs/2611.06290) ## Limits The evidence is the arXiv abstract, which reports selected results but does not provide the paper's full methods or detailed corpus descriptions. The abstract reports results on one public corpus and a month of backbone QUIC traffic; it does not establish how well the diagnostic generalizes to other corpora. ## Open question How reliably do single-identifier probes identify the relevant split axis across corpora collected in different ways? ## Source-stated On one public corpus, the same model and code received substantially different macro-F1 scores under two split protocols. > the same model and code score macro-F1 0.992 under the evaluation protocol standard in the literature and 0.567 under a protocol that keeps flows from one packet capture on one side of the split - [Supporting source](https://arxiv.org/abs/2611.06290) ## Source-stated The authors report that, on a month of backbone QUIC traffic, server-disjoint splitting reduced macro-F1, while random, temporal and client-disjoint splitting were statistically indistinguishable from session-disjoint splitting. > random, temporal and client-disjoint splitting are statistically indistinguishable from session-disjoint, while server-disjoint splitting costs 0.192 macro-F1 - [Supporting source](https://arxiv.org/abs/2611.06290) ## Source-stated The authors propose single-identifier probes to identify a relevant split axis before training; a decision tree using only the capture identifier scored differently under random and session-disjoint splits. > A decision tree on the capture identifier alone reaches 0.983 under a random split and exactly 0.000 under session-disjoint splitting. - [Supporting source](https://arxiv.org/abs/2611.06290) ## Source-stated The paper reports that five pre-registered architectural claims failed, and that removing every metric objective improved rejection of unknown applications. > Five of our own pre-registered architectural claims fail: distance-based rejection of unknown applications beats softmax thresholding, but ablating every metric objective makes rejection better - [Supporting source](https://arxiv.org/abs/2611.06290) ## Source-stated Across corpora, five labelled flows per class recovered 91% of the deficit left by failed zero-shot transfer. > across corpora, five labelled flows per class recover 91% of the deficit left by failed zero-shot transfer. - [Supporting source](https://arxiv.org/abs/2611.06290) ## Source and contributor context Sharma, Mayan, Saini, Kriti, Behl, Devansh · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Zero-shot Doing a task without worked examples (zero-shot) or with only a few examples given in the prompt (few-shot). - [Definition reference](https://developers.google.com/machine-learning/glossary#zero-shot-prompting) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: deda022a-06bf-4ae6-807a-8090e4f8251a. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-09T11:09:12.360Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 9d45473c269a8909e76876a149e40cfbf3c3f6b747a42f5d9072e9414f9f241d. Source text hash: 7e48fce442aaef58c366cb143b8ad20419386b3d29cca643d4cd5098421d4f68. ## Revision and source-check history ### Version 1 — 2026-11-09T11:09:12.360Z Mayan Sharma, Kriti Saini and Devansh Behl argue that encrypted traffic classification scores can depend sharply on how data is divided for evaluation. On one public corpus, the same model and code scored macro-F1 0.992 with a commonly used split and 0.567 when flows from one packet capture stayed together. The authors say the problem is not random splitting alone: on a month of backbone QUIC traffic, server-disjoint splitting reduced macro-F1, while three other approaches were statistically indistinguishable from session-disjoint splitting. Which split matters, they report, depends on how a corpus was collected, and may not be apparent from its labels. Their proposed diagnostic uses single-identifier probes before model training; a decision tree using only capture identifiers scored 0.983 with a random split and 0.000 with session-disjoint splitting. The paper also reports that five pre-registered architectural claims failed, including a finding that removing every metric objective improved rejection of unknown applications. Across corpora, five labelled flows per class recovered 91% of the deficit from failed zero-shot transfer. The authors release splits, results, code and negative findings. These results concern the reported evaluations; the abstract does not establish how well the diagnostic works on other corpora. ## Original source arXiv cs.AI · Article Original source date: Nov 4, 2026 (2026-11-04). Date basis: Date basis not provided. Added to Escape Meridian: 2026-11-09T11:09:12.360Z. First observed: 2026-11-05T05:06:32.020Z. Retrieved: 2026-11-09T11:08:22.678Z. - [The Split Protocol Decides the Result: Re-Evaluating Encrypted Traffic Classification](https://arxiv.org/abs/2611.06290) ## Limits The evidence is the arXiv abstract, which reports selected results but does not provide the paper's full methods or detailed corpus descriptions. The abstract reports results on one public corpus and a month of backbone QUIC traffic; it does not establish how well the diagnostic generalizes to other corpora. ## Open question How reliably do single-identifier probes identify the relevant split axis across corpora collected in different ways? ## Source-stated On one public corpus, the same model and code received substantially different macro-F1 scores under two split protocols. > the same model and code score macro-F1 0.992 under the evaluation protocol standard in the literature and 0.567 under a protocol that keeps flows from one packet capture on one side of the split - [Supporting source](https://arxiv.org/abs/2611.06290) ## Source-stated The authors report that, on a month of backbone QUIC traffic, server-disjoint splitting reduced macro-F1, while random, temporal and client-disjoint splitting were statistically indistinguishable from session-disjoint splitting. > random, temporal and client-disjoint splitting are statistically indistinguishable from session-disjoint, while server-disjoint splitting costs 0.192 macro-F1 - [Supporting source](https://arxiv.org/abs/2611.06290) ## Source-stated The authors propose single-identifier probes to identify a relevant split axis before training; a decision tree using only the capture identifier scored differently under random and session-disjoint splits. > A decision tree on the capture identifier alone reaches 0.983 under a random split and exactly 0.000 under session-disjoint splitting. - [Supporting source](https://arxiv.org/abs/2611.06290) ## Source-stated The paper reports that five pre-registered architectural claims failed, and that removing every metric objective improved rejection of unknown applications. > Five of our own pre-registered architectural claims fail: distance-based rejection of unknown applications beats softmax thresholding, but ablating every metric objective makes rejection better - [Supporting source](https://arxiv.org/abs/2611.06290) ## Source-stated Across corpora, five labelled flows per class recovered 91% of the deficit left by failed zero-shot transfer. > across corpora, five labelled flows per class recover 91% of the deficit left by failed zero-shot transfer. - [Supporting source](https://arxiv.org/abs/2611.06290) ## Source and contributor context Sharma, Mayan, Saini, Kriti, Behl, Devansh · Article author These are credits stated by the source; they do not establish a profile match or authorship of every passage. ## Source media No suitable media was found by the supported page extraction. ## Terms and references Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement. ## Zero-shot Doing a task without worked examples (zero-shot) or with only a few examples given in the prompt (few-shot). - [Definition reference](https://developers.google.com/machine-learning/glossary#zero-shot-prompting) ## arXiv arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed. - [Explore this context](/entities/arxiv) - [Reference](https://arxiv.org/) - [Reference](https://info.arxiv.org/help/api/index.html) ## Editorial provenance Version 1. Publication ID: deda022a-06bf-4ae6-807a-8090e4f8251a. Prepared by gpt-6-luna; grounding review by gpt-6-luna. Source check: uncertain by anthropic/claude-opus-5.5 on 2026-11-09T11:09:12.360Z. This checks fidelity to the source, not whether its claims are independently true. Editorial selection: accept; reasons: material\_development. Observation: 9d45473c269a8909e76876a149e40cfbf3c3f6b747a42f5d9072e9414f9f241d. Source text hash: 7e48fce442aaef58c366cb143b8ad20419386b3d29cca643d4cd5098421d4f68. ### uncertain — 2026-11-09T11:09:12.360Z — anthropic/claude-opus-5.5 Publication: deda022a-06bf-4ae6-807a-8090e4f8251a attribution — pass: The reading correctly credits Mayan Sharma, Kriti Saini and Devansh Behl as the arXiv authors and keeps the findings framed as their reports. > Authors:Mayan Sharma, Kriti Saini, Devansh Behl summary — uncertain: The summary is faithful but hard for non-specialists to follow. Central terms are unexplained: encrypted traffic classification, macro-F1, packet capture, backbone QUIC, session/server-disjoint splits, single-identifier probes, metric objective and zero-shot transfer; tooltips are needed. It omits the stake that motivates the paper, that such classifiers are routinely reported at 95-99% accuracy. It also never says plainly that the split matters because related data can leak between training and test sets. Smallest repair: restore the 95-99% framing and add one plain sentence on why keeping a capture on one side of the split changes the score. > Encrypted traffic classifiers are routinely reported at 95-99% accuracy. date — pass: The originalPublished date of 2026-11-04 matches the v1 submission date. The feed date of Nov 5 is later and was correctly not used. > \[Submitted on 4 Nov 2026\] support — uncertain: The five claims match the abstract. The second limitation says the abstract reports results on one public corpus plus QUIC traffic. However, the abstract describes one of two public corpora and reports cross-corpus few-shot results, so this understates the evaluation scope. Smallest repair: state that the headline gap was observed on one of two public corpora. The closing question is a reasonable open issue and not a leading claim. > We show that on one of two public corpora, the same model and code score macro-F1 0.992 media — pass: The selected media is the verified arXiv abstract page itself, the direct original source. ## Recorded references Explicit citations in public readings and public file attachments to this reading. Shared topics are not citations. Other artifact types need an explicit reference adapter. No public references recorded here yet. [Reference data (JSON)](/readings/1b98261a4a52a714451cef121fb0f2b5e521ba0481518289491bc77292731754/references.json) ## Corrections and challenges [Send a correction or an opposing source](https://escapemeridian.com/corrections/) [JSON record](https://escapemeridian.com/readings/arxiv-authors-sharma-saini-and-behl-argue-data-splits-can-1b98261a.json)