Structural Telemetry for LLM Outputs Under Batch Load

Abstract

1. Introduction

The standard evaluation paradigm for large language models has two dominant modes: benchmark performance on curated test sets, and LLM-as-judge evaluation in which one model rates the outputs of another. Both modes share a structural limitation: they measure behavior under favorable conditions. Benchmark prompts are presented one at a time with full response headroom. LLM-as-judge evaluation compounds the circularity by introducing a stochastic evaluator with its own structural biases.

Production deployments do not work this way. Real applications submit multiple prompts concurrently, operate under token budget constraints, and require consistent output structure across heterogeneous task types. The failure modes that matter in production — truncated formula expressions, sentence boundary suppression, and structural compression under load — are often invisible to single-prompt evaluation because they only emerge when models must allocate finite output capacity across competing demands.

This study introduces a deliberate batch-input methodology as an independent variable in structural evaluation. Twenty prompts spanning three domain types (Everyday, Math, Science) were submitted simultaneously to Gemini, GPT, and Claude in a single call, repeated across five batches to yield 400 responses per engine (1,200 total). The design was not intended to measure accuracy. It was intended to surface structural behaviors that single-prompt testing misses: output trimming, formatting fragility, segmentation suppression, and the allocation strategies each engine applies when holding multiple problem contexts simultaneously.

The measurement instrument is the Linguistic Kernel v19, an eight-dimensional deterministic telemetry system that computes structural coordinates for any text string without calling an external model. The kernel produces a coordinate in a fixed structural space, a rhetorical state classification, and an evidence bundle of countable features backing every number. Version 19 introduces a three-tier collapse detection system that achieves precision 1.00 and recall 1.00 on a 400-response validation corpus.

2. Methodology

2.1 Batch Design and Corpus

Three engines — Gemini, GPT (GPT-5), and Claude (Claude 3.5 Sonnet) — each received the same 400 prompts organized into twenty-prompt batches submitted as a single API call. Prompts were drawn from three domain categories: Everyday (148 prompts, conversational and factual queries), Math (100 prompts, undergraduate-level abstract algebra and number theory), and Science (152 prompts, undergraduate quantum mechanics and astrophysics). All three engines received the same prompts in the same order. Each batch position (1 through 20) therefore represents identical prompt content across engines, enabling direct structural comparison.

The batch methodology is a deliberate stress instrument. When a model receives twenty heterogeneous prompts simultaneously, it must allocate output capacity across competing demands: breadth versus depth, formula rendering versus response completion, precision versus speed. The structural consequences of these tradeoffs — output trimming, formatting fragility, and sentence suppression — are what this study is designed to surface.

2.2 The Linguistic Kernel v19

The kernel is a deterministic, behavior-only instrumentation layer. It takes a text string as input and returns a structured coordinate in an eight-dimensional space. No external model is called at any stage. Every number produced by the kernel is backed by countable evidence features.

The eight dimensions are organized into a canonical four-dimension kernel and a four-dimension auxiliary set:

2.3 V19 Collapse Detection: Three-Tier Architecture

Version 19 replaces the single v18 formatting_collapse gate with a three-tier detection system, validated on the GPT EMS corpus (n=400, 15 true collapses identified by prompt-answer audit).

The tail_incomplete_flag fires on unbalanced delimiters, dangling operators, unclosed code fences, and grammatically open scaffold-word endings. The math_expression_tail_flag fires on unbalanced curly braces, odd code fence counts, dangling LaTeX commands, and code lines ending mid-statement. The conjunction of both flags is required for the math tier: math_expression_tail_flag alone produces false positives on complete responses containing inline math notation.

3. Results

3.1 Structural Fingerprints by Engine

Table 3 reports mean values and Cohen's d effect sizes for all eight kernel dimensions across pairwise engine comparisons (n=400 per engine). The most structurally distinctive dimension is θ (flow instability): Claude shows significantly higher flow instability than both Gemini (d = −0.325, p < 0.001) and GPT (d = −0.587, p < 0.001). Claude's Everyday θ of 0.675 is nearly triple GPT's 0.254, indicating that Claude shifts topic or lexical register between adjacent sentences far more frequently in conversational contexts. GPT's Everyday flow is the smoothest of the three engines.

The r_v (void ratio) dimension separates Claude from both other engines: Claude produces structurally denser responses (lower r_v = 0.269 vs Gemini 0.345, d = +0.479, p < 0.001; vs GPT 0.325, d = +0.347, p < 0.001). The μ (cohesion) dimension shows no significant differences across any engine pair, a finding examined separately in Section 3.4.

3.2 Two Geometrically Opposite Failure Modes

The batch corpus reveals two structural failure modes that share a common trigger (batch load pressure) but occupy opposite regions of kernel space. Table 4 reports structural signatures for formatting collapse, constraint compliance, and the assertive baseline.

The signatures are directionally opposed on both primary axes. Formatting collapse shows r_v inflated above baseline and Δy negative (sentences shorter than the 15-token baseline, consistent with a response that terminated before completing its sentence structure). Constraint compliance shows r_v collapsed below baseline and Δy elevated (long sentences with suppressed boundaries, consistent with the model delivering multiple clauses without paragraph breaks). A single detection threshold cannot cover both failure modes.

Across all three engines, every instance of constraint compliance was driven by sentence_suppression (Δy spike), not verbosity (elevated token count with normal Δy). This uniformity suggests that the batch load mechanism for constraint compliance is sentence boundary suppression rather than content inflation.

3.3 Collapse Rates and the Shared Collapse Prompt Class

Table 5 reports formatting collapse rates by engine and domain. Gemini collapses at 5.5% overall (22/400), GPT at 3.8% (15/400), Claude at 0.25% (1/400). Domain effects are large: Gemini's Math collapse rate is 10.0% versus 0.0% for Claude; GPT's Science rate is 5.9% versus 0.7% for Claude.

Cross-engine analysis reveals that 10 of GPT's 15 collapses occurred on the same prompt numbers as Gemini collapses. All 10 shared collapse prompts are Math or Science domain; none are Everyday prompts. Claude produced assertive responses on 9 of these 10 prompts.

Prompt content analysis characterizes the shared collapse class. Shared collapse prompts show markedly higher rates of quantum mechanics content (40% vs 0% for Math/Science normal prompts), matrix/operator content (40% vs 2%), LaTeX notation (40% vs 7%), and integral/normalization requirements (30% vs 5%). The distinguishing feature is not problem difficulty per se but output rendering demand: 8 of 10 shared collapse prompts require the model to sustain continuous LaTeX formula derivation across multiple intermediate steps. The remaining 2 are statement true/false proofs where collapse occurs inside the justification chain.

We propose the term rendering-intensive derivation (RID) for this prompt class. RID prompts share three structural characteristics: (1) the correct response requires multi-step formula progression, not a single expression; (2) each intermediate step must be rendered in a structured notation (LaTeX, matrix notation, or formal set-theoretic notation); and (3) the response cannot be abbreviated without losing correctness. Under batch allocation pressure, both Gemini and GPT exhausted their per-response output budget mid-derivation on RID prompts. Claude did not.

3.4 Claude's Allocation Behavior Under Batch Load

Claude's structural behavior under batch load differs systematically from Gemini and GPT across several dimensions:

• Token output. Claude's mean token count per response is 89.9, compared to Gemini's 41.6 and GPT's 44.4. Effect sizes are large: Claude vs Gemini d = −1.643 (p < 0.001); Claude vs GPT d = −1.431 (p < 0.001). Gemini and GPT are not significantly different from each other (d = −0.151).
• Batch position trajectory. Claude's mean token count increases across batch positions: early batch (positions 1–7) mean 83.9 tokens, late batch (positions 15–20) mean 96.1 tokens, a 14% expansion. Gemini shows mild compression (early 43.2, late 40.2). GPT is roughly flat. No engine shows dramatic monotonic compression, but Claude's divergence from the other two grows from early to late batch.
• Dialectic rate. Claude produces the dialectic rhetorical state at 4.5% (18/400) versus Gemini 0.2% (1/400) and GPT 1.0% (4/400). Claude's Science dialectic rate is 6.6% (10/152). This is a structural fingerprint: Claude more frequently holds two positions in explicit structural opposition within a single response.
• Flow instability on Everyday prompts. Claude's Everyday θ of 0.675 is 2.66× GPT's 0.254 (d = −0.587 overall, p < 0.001). On Everyday prompts, Claude shifts topic or lexical register between adjacent sentences more frequently than either other engine, consistent with a response style that synthesizes multiple angles even in conversational contexts.
• Collapse displacement. Claude's single collapse response is displaced 1.97× from its neutral centroid in r_v-Δy space, compared to 1.65× for Gemini and 1.63× for GPT. Claude's collapse threshold appears higher, but when exceeded the structural deformation is more severe. Prompt #327 (Science, batch position 7: a two-qubit entanglement classification problem) is the only prompt where Claude's response was flagged as a math-tier collapse (56 tokens, collapse_subtype=math), while Gemini produced an assertive response on the same prompt.

3.5 Mu Invariance Across Engines and States

The μ (cohesion) dimension shows no significant differences across any engine pair (all p > 0.48) and no significant difference across rhetorical states within any engine. The coefficient of variation across all three engines is 0.046–0.057, consistent with the finding from Paper 2 (Gemini EMS study, CV = 0.056). This invariance now holds across 1,200 responses, three architectures, and four rhetorical states.

The implication is that cohesion — the density of transition markers and reference pronouns — is structurally stable under batch pressure in ways that other dimensions are not. Batch load compresses r_v, suppresses θ, and elevates Δy in constraint-compliance responses; it does not meaningfully alter cohesion. This makes μ a reliable control variable for cross-engine studies: a large μ shift indicates something more fundamental than formatting pressure has changed in the response.

3.6 BPE-to-Word Ratio: Surface Measurement Divergence

The kernel runs a parallel tiktoken tokenizer (cl100k_base encoding) alongside its own word tokenizer. The BPE-to-word ratio — the ratio of BPE tokens to surface word tokens — is a diagnostic signal for how closely the kernel's surface measurements correspond to the model's internal token experience.

Claude shows the highest BPE-to-word ratio and the highest variance: mean 1.721 (std 0.492, max 4.662), compared to Gemini 1.603 (std 0.353) and GPT 1.528 (std 0.294). Claude's Science responses have a mean ratio of 2.035, the highest domain-specific ratio in the corpus. This reflects Claude's heavier use of mathematical notation in Science responses: LaTeX expressions, special characters, and subscript notation each produce multiple BPE tokens per surface word, inflating the ratio. The implication is that for Claude's Science responses, the kernel's surface word-level measurements correspond less closely to Claude's internal token experience than for Gemini or GPT responses on the same prompts.

4. Discussion

4.1 Batch Methodology as an Evaluation Instrument

The findings support treating batch submission as an independent evaluation variable rather than a methodological limitation. The structural behaviors that emerge under batch load — output trimming, formula fragility, sentence suppression, and allocation expansion — do not appear reliably in single-prompt evaluations. They are properties of how models manage concurrent demands, not properties of how models handle individual prompts.

The two failure modes identified in this corpus emerge from the same batch pressure through different mechanisms. Formatting collapse is triggered by prompt content (rendering-intensive derivations that exceed output allocation) and distributes roughly uniformly across batch positions. Constraint compliance back-loads — positions 9, 13, and 15 are the highest-frequency positions — consistent with cumulative load effects building across the batch rather than being triggered by individual prompt content. These are different pressure mechanisms requiring different monitoring strategies.

4.2 The Rendering-Intensive Derivation Prompt Class

The identification of a prompt class that predicts cross-engine collapse is the most novel finding of this study. The RID class — prompts requiring sustained multi-step LaTeX formula output — is not characterized by difficulty in the usual sense. Several RID prompts are standard undergraduate exercises. What distinguishes them is that a correct response cannot be produced without continuously rendering structured notation, and that rendering demand competes with response volume for the same output budget.

The practical implication is that batch evaluations of technical models should include RID prompts explicitly. A model that handles all twenty prompts in a batch with zero structural failures may still produce systematic collapse when two or three RID prompts are included, because the RID prompts alone can exhaust the rendering budget allocated to them.

4.3 Structural Resilience vs Capability

The difference between Claude and the other two engines in this corpus is not a capability difference. Gemini and GPT produced correct responses on the prompts where they did not collapse. Claude's advantage on the shared collapse prompts is structural: it allocated more output tokens to each response, did not compress under batch load, and maintained response completeness on the RID prompts that triggered failure in both other engines.

This distinction matters for how the results should be interpreted. The finding is not that Claude is a better model for quantum mechanics. The finding is that Claude's output allocation strategy under batch load is structurally different — more token-generous, less sensitive to batch position, and more resistant to mid-derivation truncation. Whether that strategy is always desirable (it produces longer responses with higher BPE-to-word ratios and more frequent lexical register shifts) depends on the deployment context.

5. Conclusions

This study demonstrates that batch-input methodology surfaces structural LLM behaviors invisible to single-prompt evaluation, and that the Linguistic Kernel v19 provides a deterministic instrument for measuring those behaviors without calling an external model.

Three conclusions follow from the results:

• The two-window detection requirement. Formatting collapse and constraint compliance are structurally opposite failure modes occupying different regions of kernel space (collapse: high r_v, negative Δy; constraint: low r_v, elevated Δy; both at large effect sizes p < 0.001). Production monitoring systems require independent observation windows for each mode. A single threshold cannot cover both.
• The rendering-intensive derivation class. Ten prompts produced cross-engine collapse in both Gemini and GPT. All ten share a structural characteristic — sustained multi-step LaTeX formula derivation — that distinguishes them from other Math and Science prompts in the corpus. This class can be identified prospectively by prompt content and should be included explicitly in batch evaluation protocols for technical models.
• Mu as a cross-engine control variable. Cohesion (μ) is invariant across engines and rhetorical states in a 1,200-response corpus spanning three architectures. This confirms its role as a structural control variable: cross-engine studies can treat μ stability as a baseline sanity check and treat μ deviation as a signal that something beyond formatting pressure has changed.

Future work should extend the batch methodology to larger corpora and additional engines, apply the RID classification prospectively in batch design, and examine whether the constraint compliance back-loading effect is reproducible across different prompt orderings within batches.

References

1. Parrish, R. (2025). Text Has Shape: Structural Operating Envelopes in Gemini Outputs (Paper 2, Gemini EMS Study). Artist Influencer. artistinfluencer.com.
2. Parrish, R. (2026). Text Has Shape: Linguistic Kernel Validation on a 1,597-Response Corpus with Permutation Null Models (Paper 1, v2). Artist Influencer. artistinfluencer.com.
3. Parrish, R. (2026). Linguistic Kernel User Manual, Version 19. Artist Influencer. artistinfluencer.com.

Appendix A: Effect Size Summary

Table A1 reports all significant Cohen's d values from pairwise engine comparisons on the eight kernel dimensions.

Table A2: Collapse vs assertive and constraint vs assertive effect sizes (all engines combined).

Appendix B: Corpus Details

Table B2: Shared collapse prompt inventory.

Appendix C: Statistical Appendix

Statistical Methods

All bootstrap confidence intervals are computed with 2,000 resampling iterations and 95% coverage. The null hypothesis for each test is no difference between the two groups. Two-tailed p-values are estimated as twice the proportion of bootstrap resamples on the wrong side of zero, with a minimum p of 1/2000 = 0.0005. Effect sizes are Cohen's d with pooled standard deviation.

Significance thresholds: *** p<0.001 (green), ** p<0.01 (blue), * p<0.05 (amber), ns p≥0.05 (gray).

C1. Pairwise Engine Comparisons: Key Dimensions

n = 400 per engine. Bootstrap mean difference = engine 1 mean minus engine 2 mean.

r_v (void ratio)

θ (flow instability)

Δy (segmentation strain)

ρ_r (packing density)

μ (cohesion): invariance confirmation

μ CIs all straddle zero. The dimension is structurally invariant across engines. This holds across n=1,200 responses and all rhetorical states.

d_s (structural thickness)

C2. Token Count Comparisons

C3. Formatting Collapse vs Normal: Within-Engine

Bootstrap mean difference: collapsed responses minus normal responses. Positive = collapsed higher than normal.

Gemini (collapsed n=22, normal n=378)

GPT (collapsed n=15, normal n=385)

r_v and Δy are highly significant in both engines (p<0.001, large d). θ is significant in Gemini but not GPT, consistent with GPT's higher baseline θ variance in Science/Math responses absorbing the collapse signal. ρ_r significant in both.

C4. Constraint Compliance vs Assertive: All Engines Combined

constraint n=127, assertive n=1,011. Bootstrap on mean difference: constraint minus assertive.

Δy effect size d=+2.54 is the largest in the entire corpus. The constraint compliance regime is defined by extreme sentence elongation: the model delivers multiple clauses without sentence boundaries. This is structurally opposite to collapse (Δy negative in both engines).

C5. Collapse Rate Comparisons (Proportion Bootstrap)

C5.1 Overall collapse rates

Gemini vs GPT overall collapse rate difference is not significant (p=0.308). The difference is real but the small event count (22 vs 15 collapses) produces wide CIs. Both engines are significantly higher than Claude.

C5.2 Domain-specific collapse rates

C6. Batch Position: Token Count Trajectory

Claude is the only engine that expands output across batch positions. Gemini shows mild compression. GPT is roughly flat. No engine shows dramatic monotonic compression, contrary to a simple 'batch fatigue' model.

C7. μ (Cohesion) Full Invariance Table

C8. Z-Space Displacement: Collapsed vs Normal

Euclidean distance from each engine's neutral centroid in r_v-Δy space, normalised to z-scores.

Reproducibility

All bootstrap CIs used random seed 42 with 2,000 resamples. Kernel version: lang-kernel-rebuild-v19. Data: EMS Full Set (Claude, GPT, Gemini), n=1,200 responses, March 2026. Raw kernel outputs, bootstrap code, and the full 1,200-response dataset are available on request. ORCID: 0009-0008-9781-7995.

Appendix D: Transparency

This appendix documents anticipated methodological objections and responds to each in detail. It is written in advance of peer review because the objections are predictable, structurally grounded, and worth addressing before they are raised rather than after. A note on context: this is independent practitioner research conducted outside an institutional setting, without access to model weights, training data, or internal APIs beyond the public inference endpoints.

O1. This is a small demonstration dataset.

O2. Why batch load testing? Real evaluations are done one prompt at a time.

O3. The geometric language is overblown.

O4. The kernel design is arbitrary.

O5. The formatting collapse regime falls under LaTeX rendering failures.

O6. The paper implies structural failures are predictable. That is not proven by this dataset.

O7. What does this paper actually contribute?

O8. Why publish now rather than wait for a larger corpus?

O9. The study does not control for prompt difficulty or prompt length.

O10. Note on AI-assisted research and transparency.

Summary: What We Claim and What We Do Not

This appendix will be updated in response to peer review. Reviewers are encouraged to raise objections not covered here.