Text has shape

What a deterministic reading of text can tell apart: the prompt from the run, one engine from another under load, a finished answer from a truncated one, and which instruction wins when instructions collide.

Russell Parrish · Parallax Metrology · Linguistic Kernel v19 · Gemini, GPT and Claude responses, collected 2025–2026

The instrument

Most ways of evaluating a language model's output either compare it with a reference answer or ask another model to judge it (Zheng et al., 2023). Both read for meaning. The Linguistic Kernel reads for form, in the tradition of multi-feature text analysis (Biber, 1988; Graesser et al., 2004). It turns any text into eight numbers built from things that can be counted on the page: sentence lengths, paragraph breaks, content words, connectives, rare vocabulary, clause markers. No model is called, nothing is learned, and the same text always returns the same numbers. Every number comes with the counts that produced it.

The kernel also runs a tail check on every response: unbalanced brackets, braces and dollar signs, dangling operators, an unfinished LaTeX command, an unclosed code fence. That check matters in section 3.

1. What the prompt sets, and what the run sets

Sixty prompts, ten each from six task types (customer support, extraction, reasoning, tool use, rewriting, summarisation), were each sent to Gemini five times. Across the 300 responses, the question for each reading is how much of its spread comes from which prompt was asked and how much from which run happened.

The readings fall into two groups.

• Set by the prompt: clause density (dₛ, 80%), rare vocabulary (xₚ, 74%), content-word density (ρᵣ, 73%) and cohesion (μ, 57%). Ask the same thing again and these come back close to where they were. They describe the kind of text a prompt calls for.
• Set by the run: average sentence length (Δy, 20%) and how far each sentence moves from the last (θ, 18%). These change a great deal between runs of the same prompt. They describe how a particular answer happened to be put together.

This is the first thing the instrument can do: say which differences between two answers are about the prompt and which are run-to-run variation. A difference in dₛ between two prompts is likely real. A difference in θ needs several runs before it means anything.

2. Three engines, the same prompts, under load

The same 400 prompts were sent to Gemini, GPT-5 and Claude 3.5 Sonnet: 148 everyday questions, 100 undergraduate abstract-algebra problems and 152 undergraduate science problems, mostly physics with some chemistry. They were submitted twenty at a time in a single call, so each engine had to divide its output across twenty answers at once. Batching prompts into one call is a common way to cut cost and latency (Cheng et al., 2023), and it is a condition single-prompt testing never creates.

The engines allocate output very differently:

• Length. Claude wrote about twice as much per answer (90 words against 42 for Gemini and 44 for GPT; Cohen's d 1.6 and 1.4). Gemini and GPT were close to each other.
• Answer form. GPT and Gemini answered in one or two sentences 64% and 65% of the time; Claude 16%. Many of those short answers are complete, delivered as a single long sentence.
• Flow. On everyday questions Claude's sentences move further from each other (θ = 0.68) than GPT's (0.25) or Gemini's (0.51). Claude tends to cover several angles even in a casual answer.
• Breathing room. Claude's answers are denser (rᵥ = 0.27 against 0.35 for Gemini, d = 0.48).
• Cohesion. Connective density is the same for all three engines (no measurable difference, η² = 0.00).

3. Answers that stop mid-formula

Under batch load, some answers stop partway through a formula: an opening dollar sign with nothing after it, a fraction that never closes, a matrix left open. The kernel's tail check flags these. Every flagged response was read to confirm it; 36 were real truncations and two were complete answers flagged by a stray dollar sign.

The striking part is the overlap. Ten prompts truncated in both Gemini and GPT, and Claude completed all ten. They are not obviously harder than the rest of the set. What they share is that the answer cannot be written without a continuous run of notation, step after step:

• the index of a permutation subgroup, and the order of an element of ℤ₁₂ × ℤ₈;
• two true/false statements about direct products and the converse of Lagrange's theorem;
• a delta-potential bound state, and a density matrix on the Bloch sphere;
• three spin expectation values and probabilities, and the expectation of momentum squared for a given wavefunction.

Call these rendering-intensive derivations: the correct answer is a chain of formulas, each step must be written in structured notation, and it cannot be shortened without becoming wrong. When output capacity is being shared across twenty answers, these appear to be the answers that run out first. On those ten prompts, Gemini's answers averaged 17 words and Claude's 77.

What does the detecting is the tail check, together with answer length. The eight structural readings on their own separate truncated from finished answers no better than word count does, which is expected: a truncated answer is mostly a very short one.

4. When instructions collide

Instruction-following benchmarks check whether verifiable constraints were met (Zhou et al., 2023). Some prompts stack constraints that cannot all be met, or add a new one that contradicts the old. Repeated runs show which constraint a model keeps and which it lets go. These are observations from five runs per prompt on Gemini, not general rates.

The pattern across these is consistent. Constraints that change the visible form of the text (sentence count, punctuation, alternation) are kept. Exact counting is the first thing to go. When a newer instruction contradicts an older one, the newer one usually wins. A prompt that stacks too many at once can fail completely: one asking for a Hamlet summary with alliteration in every sentence, no letter S, and sentences shrinking by exactly two words from twenty returned nothing in two of five runs, and none of the other three kept the shrinking lengths.

What it is for, and what it is not

• Monitoring without a judge. The readings are cheap, deterministic and auditable, so they can run on every output of a pipeline. A shift in the prompt-set readings is worth investigating; a shift in the run-set readings needs more samples first.
• Comparing engines on the same prompts. Length, answer form and flow separate engines clearly under the same load, without reading a word for meaning.
• Catching truncation. The tail check catches answers that stopped mid-notation, which a reader skimming for meaning can miss and a semantic score may not flag.
• Checking constraint compliance. Sentence counts, lengths and punctuation are counted directly, so whether a formatting instruction was followed is a measurement, not a judgement.

Data and method

All readings computed with Linguistic Kernel v19 (default configuration) on the stored response text. Engine comparisons use Cohen's d with pooled standard deviation (Cohen, 1988). Truncations were flagged by the kernel's tail check and confirmed by reading every flagged response. Sentence and word counts in §4 were made on the response text, with the model's own "(Banned: …)" annotations excluded.

Sources

This page is compiled from the author's working studies on the Linguistic Kernel: Text Has Shape: Structural Operating Envelopes in Large Language Models (2026), Text Has Shape: Local Deformation Spectrum of a Language Model (2025), Structural Telemetry of Gemini Responses Under Batch Constraint (2026), and the cross-engine study published as Structural Telemetry for LLM Outputs Under Batch Load. The archive of the original papers is kept at artistinfluencer.com.

References

• Biber, D. (1988). Variation across Speech and Writing. Cambridge University Press.
• Cheng, Z., Kasai, J., & Yu, T. (2023). Batch prompting: Efficient inference with large language model APIs. Proceedings of EMNLP 2023: Industry Track.
• Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Lawrence Erlbaum.
• Graesser, A. C., McNamara, D. S., Louwerse, M. M., & Cai, Z. (2004). Coh-Metrix: Analysis of text on cohesion and language. Behavior Research Methods, Instruments, & Computers, 36(2), 193–202.
• Halliday, M. A. K., & Hasan, R. (1976). Cohesion in English. Longman.
• Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420–428.
• Zheng, L., Chiang, W.-L., Sheng, Y., et al. (2023). Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. Advances in Neural Information Processing Systems 36 (Datasets and Benchmarks Track).
• Zhou, J., Lu, T., Mishra, S., Brahma, S., Basu, S., Luan, Y., Zhou, D., & Hou, L. (2023). Instruction-following evaluation for large language models. arXiv:2311.07911.

See also: Structural residue, whether the same readings carry a trace of a model's token-level uncertainty · The Linguistic Kernel.