Standalone research edition · no dependencies

THE TRAINING DATA LEDGER

A field guide to what can—and cannot—be tracked from training data to a model output.

01 / LIFECYCLE

Eight places where provenance changes meaning.

Choose a stage. Formal math and ordinary language describe the same mechanism; the ledger columns say what must be retained.

Formal view

Plain view

Minimum viable evidence

    Hyperparameters / choices

      Defensible claim
      Claim the evidence cannot support
      02 / IDENTIFIABILITY

      Same weights. Different pasts.

      If two training histories land on the same parameters, no inspection of final weights alone can reconstruct which units were present.

      Constructive collision

      One endpoint does not encode a unique training ledger.

      A(D₁, ω₁) = θ* = A(D₂, ω₂), while D₁ ≠ D₂

      Switch histories below. The observed parameters and inference remain fixed; the exposure record changes.

      Observed endpoint θ*
      0.20
      −1.10
      0.71
      0.04
      −0.33
      0.98
      0.52
      −0.08

      fθ*(x) is identical under both histories.

      03 / ARCHITECTURE

      Layer by layer, with the receipt beside it.

      Runtime activations explain computation on this input. Training-data attribution asks which past interventions changed the learned computation.

      Formal mechanism

      Layman’s mechanism

      What to log

      Hyperparameters

      04 / ATTRIBUTION LAB

      Five methods. Five different questions.

      The bars are a toy system with three training units. Toggle interactions to see why “percent contribution” is not generally unique.

      Estimates:

      Does not establish:

      05 / COST

      The exact question is usually the expensive one.

      Leave-one-out retraining approaches the desired causal effect, but its bill scales with dataset units, seeds, and the cost of one training run.

      Naïve leave-one-out

      O(N × seeds × train-cost)
      06 / OBSERVABILITY

      What the system can honestly know.

      Capabilities differ sharply between a controlled research stack and a hosted model endpoint.

      Evidence / claimFinal weights onlyHosted APIInstrumented trainingCounterfactual reruns
      07 / LEDGER DESIGN

      Keep two ledgers. Do not fake one.

      The exact ledger records deterministic lineage. The influence index stores method-dependent estimates, assumptions, uncertainty, and validation.

      Schema simulator

      Change the unit.

      The answer changes before the method runs. A token loss, document, creator, and source collection are not interchangeable atoms.

      Influence scores should be signed estimates with confidence and an explicit target—not percentages that are forced to sum to 100.

      
    
      

    
      
      
      08 / RESEARCH BASE
      Primary sources, not vibes.
      Foundational architecture papers and the main families of influence, reasoning, alignment, and tool-use methods.
      
      
      
        
      Architectures & diffusion
        
          
      1. Vaswani et al. (2017), Attention Is All You Need
      2. Ho et al. (2020), Denoising Diffusion Probabilistic Models
      3. Rombach et al. (2022), Latent Diffusion Models
      4. Peebles & Xie (2023), Diffusion Transformers
      5. Ho & Salimans (2022), Classifier-Free Guidance
      6. Meng et al. (2021), SDEdit
      7. Austin et al. (2021), Discrete Denoising Diffusion
      8. Li et al. (2022), Diffusion-LM
        9. 
        
      
        
      Attribution & memorization
        
          
      1. Koh & Liang (2017), Influence Functions
      2. Ghorbani & Zou (2019), Data Shapley
      3. Pruthi et al. (2020), TracIn
      4. Ilyas et al. (2022), Datamodels
      5. Park et al. (2023), TRAK
      6. Grosse et al. (2023), LLM Training Data Influence
      7. Georgiev et al. (2024), Journey-TRAK
      8. Zheng et al. (2023), D-TRAK
      9. Wang et al. (2024), Diffusion Attribution Score
      10. Carlini et al. (2021), Extracting Training Data
        11. 
        
      
        
      Reasoning & alignment
        
          
      1. Wei et al. (2022), Chain-of-Thought Prompting
      2. Ouyang et al. (2022), InstructGPT / RLHF
      3. Rafailov et al. (2023), Direct Preference Optimization
      4. Lightman et al. (2023), Process Supervision
      5. Turpin et al. (2023), Unfaithful Explanations
      6. Lanham et al. (2024), Measuring CoT Faithfulness
        7. 
        
      
        
      Tools & execution
        
          
      1. Yao et al. (2022), ReAct
      2. Gao et al. (2022), Program-Aided Language Models
      3. Schick et al. (2023), Toolformer
        4. 
        
      
      
      
    
      

    
      Track the receipt exactly. Estimate the influence honestly.
      A defensible system does both—and never quietly substitutes similarity for causality.