Training is a forward pass

Think about what backprop does for a single prediction. The forward pass maps inputs through activations (the intermediate values the network computes along the way) to an output; backprop runs the chain rule in reverse, tracing a small change back through each step that depended on it, so every input gets its share of credit for the output. Nobody finds this mysterious anymore.

Now look at a training run with the same eyes. It maps data (each example carrying a weight, all 1 by default) through a sequence of checkpoints (each training step producing the next) to a final measurement (the trained model’s loss on our gold-star test point). Same shape: inputs, intermediate states, output. So run the chain rule in reverse through training itself, and each training example gets its share of credit for the final measurement. That reverse pass has a name in this literature, unrolling (Bae et al. 2024), and the idea goes back to Hara et al.’s SGD-influence (Hara, Nitanda, and Maehara 2019). The analogy is exact, not decorative:

viz.trainingForward(d3)

The exact formula, and its price

Make it concrete on our classroom, where training is full-batch gradient descent with learning rate \(\eta\): \(\theta^{(t+1)} = \theta^{(t)} - \eta\, \nabla L(\theta^{(t)})\) (each step measures the loss’s slope \(\nabla L\) using all sixty points at once, which is the “full batch,” and moves the dials downhill by the step size \(\eta\)), run for \(T = 1500\) steps. How does the final \(\theta^{(T)}\) depend on example \(j\)’s weight? The chain rule answers with a sum over every step where \(j\) was felt:

\[ \frac{d\theta^{(T)}}{dw_j} \;=\; -\sum_{t=0}^{T-1} \underbrace{\left(I - \eta H_{T-1}\right)\cdots\left(I - \eta H_{t+1}\right)}_{\text{transport: how the rest of training carried it}} \;\cdot\; \underbrace{\frac{\eta}{n}\, \nabla \ell_j(\theta^{(t)})}_{\text{the pull of example } j \text{ at step } t} \]

The \(1/n\) is the batch average, a positive scale that changes no sign and no rank. The leading minus is gradient descent moving against the gradient, and it does matter, just not alone: removing an example flips the sign once more (its weight steps from one to zero), the two minuses cancel, and part 1’s convention survives. Positive still means helpful.

Read each factor in plain terms. At step \(t\), example \(j\) tugs the model’s dials (its parameters \(\theta\)) down its gradient: that is the pull. Then every later step transforms that tug: each transport factor \((I - \eta H_t)\) says how a small parameter difference at step \(t\) survives into step \(t+1\), shrinking along stiff directions, persisting along flat ones. Multiply the whole chain and you know how much of the step-\(t\) tug is still standing at the end.

Sum over all the steps and you know how the final dials move when example \(j\)’s weight changes. One last conversion turns dial-movement into credit: dot it with the test gradient, the direction in dial-space that raises the gold-star point’s loss fastest (part 2’s move), then flip the sign once because removal steps the weight from one to zero, and you have a prediction of the change in the measurement, good to first order in the weight change, the same small-step honesty as part 2. What is exact here is the derivative itself: no convexity assumed, no convergence assumed, no unique minimum required. Every one of part 3’s first three causes, dodged.

The price is written in the transport chain: it contains a Hessian per step, the loss’s local curvature at that checkpoint, part 2’s expensive matrix all over again. For our three-parameter classroom, fine; this post computes it exactly. For a real model, \(T\) Hessians is the impossible object of part 2 multiplied by the length of training. Every method in this family is a strategy for not paying that bill, and the honest way to present them is exactly how the field’s own tutorial does it (Ilyas et al. 2024): as deletions from this formula.

The approximation ladder

TracIn deletes the boldest thing possible: the entire transport chain, treating every \((I - \eta H)\) factor as the identity (Pruthi et al. 2020). What survives is beautifully cheap: the sum, over saved checkpoints, of a dot product between the test gradient and example \(j\)’s gradient. No Hessians anywhere; checkpoints you were probably saving anyway. The bet is that a tug now is a tug forever, uncarried, undamped.

SOURCE deletes less: it segments training into a few phases and freezes one Hessian per segment, so every transport factor in the segment is the same matrix and the whole chain collapses to a single formula you can evaluate directly (a closed form), the way \(2 \times 2 \times \cdots\) collapses to a power (Bae et al. 2024). The bet is milder: training’s curvature changes slowly enough that a few snapshots of it suffice. And SOURCE’s closed form contains a small revelation we will come back to below: the transport chain acts like damping, with the damping constant set by the training run itself.

MAGIC deletes nothing. It computes the exact unrolled derivative, made affordable by recomputation-friendly checkpointing of the training run, and pays the honest storage bill for near-exact single-run attribution (Ilyas and Engstrom 2025).

That is the whole family, and on the classroom we can afford to grade every method against the sixty true retrainings from part 1:

viz.unrollLadder(d3, unrollData)

Three honest annotations to the ladder. First, cheap does not mean useless: TracIn’s checkpoint sums scale past anything that has to carry a Hessian, and “which examples pulled hardest, ever” is often the right debugging question even uncarried.

Second, cheap does not mean safe: part 3’s opening exhibit, the CIFAR-trained model attributing an MNIST digit to photographs, failed for TracIn and its checkpoint variant too (Hammoudeh and Lowd 2022). The ladder buys you compute; it does not buy you out of checking your answers.

Third, the classroom flatters SOURCE: on this small, fully converged problem the segmented estimate lands on the exact unrolled value to machine precision, because the early steps’ differences contract away entirely. SOURCE’s savings, and its gap from exactness, only show at scales where per-step Hessians are unaffordable (Bae et al. 2024). One more honesty: even exact unrolling grades at 0.9992 against the true retrainings, not 1.000. The derivative is exact; removal is a finite step (a weight dialed from 1 to 0), and any derivative predicts a finite step only to first order, part 2’s tangent line again. That finite-step gap, not unrolling error, is why MAGIC’s own authors say near-optimal rather than exact.

The revelation inside SOURCE

SOURCE’s closed form does something profound. \(K\) repeated shrinkages by the same factor \((1-\eta\sigma)\) compound into exponential decay, the way compound interest compounds growth, and summing what survives gives each curvature direction \(\sigma\) (an eigenvalue of the segment’s Hessian: the stiffness of the loss along one axis) a factor \(\frac{1 - e^{-\eta K \sigma}}{\sigma}\), where \(\eta\) is the learning rate and \(K\) is the number of steps taken with that frozen curvature. Compare that to the endpoint influence function’s \(\frac{1}{\sigma}\), the divide-by-zero machine from part 3:

viz.sourceFilter(d3, unrollData)

Part 3 ended with damping as an embarrassing knob: an arbitrary \(\lambda\) added to every curvature \(\sigma\) before dividing, capping \(1/\sigma\) at \(1/\lambda\), and someone has to pick it, changing the answer as they pick it. The unrolled view says the knob was never arbitrary: training itself damps flat directions, with an effective \(\lambda\) of \(1/(\eta K)\), because a direction training barely moved along is a direction removal barely matters along. Set part 3’s knob to \(\lambda = 1/(\eta K)\) and the endpoint formula’s \(1/\sigma\) caps at the same ceiling \(\eta K\) the plot shows. The formula could not tell us what to divide by in the flat directions because the honest answer lives in the trajectory, not the endpoint.

Where the two roads meet

One loose end from part 2: if unrolling is exact and the influence function is a local approximation, they should agree somewhere. They do, and the somewhere is instructive: run training long enough on a well-behaved (convex) loss, and the unrolled attribution converges to the influence function’s answer.

viz.convergeIF(d3, unrollData)

The theory behind that convergence has recently been made precise, and it runs deeper than a sanity check: influence functions turn out to be the limiting case of unrolled differentiation, no convexity required, with the flat directions handled by a pseudoinverse rather than a divide-by-zero (Mlodozeniec et al. 2025).

The endpoint methods of parts 2 and 3 are not a different theory from the dynamics methods of this post; they are the equilibrium limit of one theory. And everything that went wrong in part 3 went wrong where the limit does not apply: valleys the run never settled into, directions it never explored, stages it was still passing through.

The bill, again

This family has its own bill to pay: what MAGIC has to keep is the run. Exact unrolling needs the trajectory, checkpointed cleverly (only \(O(\log T)\) checkpoints held at a time, paid for with an \(O(T \log T)\) recompute bill), and needs Hessian-vector products along it (multiplying the curvature matrix against a chosen vector, which can be done without ever writing the matrix down) (Ilyas and Engstrom 2025). TracIn needs only checkpoints and gradients, but Fig. 2 shows what the discount costs. And every step of the ladder that keeps any transport still carries part 2’s other giant: curvature in a parameter space that, for a frontier model, has more Hessian entries than any machine has bytes. Even TracIn, which dodges the matrix, pays for per-example gradients in that same space.

Part 5 goes there: what actually ran on a 52-billion-parameter model, how the Hessian was factored into something storable, and a stranger idea, replacing the inversion with sampling. That last idea lives on the endpoint road of part 3’s fork, but it shares this post’s respect for what training actually did rather than what optimality says it should have done.

Further reading

The family, in order of deletion

• Pruthi, Liu, Kale, and Sundararajan, “Estimating Training Data Influence by Tracing Gradient Descent” (Pruthi et al. 2020). TracIn: the transport-free end of the ladder.
• Bae, Lin, Lorraine, and Grosse, “Training Data Attribution via Approximate Unrolling” (Bae et al. 2024). SOURCE: segmented transport, and the training-effort damping made explicit.
• Ilyas and Engstrom, “MAGIC: Near-Optimal Data Attribution for Deep Learning” (Ilyas and Engstrom 2025). Exact unrolling with a checkpointing budget.
• Hara, Nitanda, and Maehara, “Data Cleansing for Models Trained with SGD” (Hara, Nitanda, and Maehara 2019). The unrolling idea applied to SGD, before the family had a name.

The reconciliation

• Mlodozeniec, Reid, Power, Krueger, Erdogdu, Turner, and Grosse, “Distributional Training Data Attribution: What do Influence Functions Sample?” (Mlodozeniec et al. 2025). Influence functions as the limit of unrolling, flat directions and all.
• Lee, Smith, Adam, and Hoogland, “Influence Dynamics and Stagewise Data Attribution” (Lee et al. 2025). Part 3’s sign-flip result: the trajectory structure this family exists to capture.

The map

• The ICML 2024 tutorial “Data Attribution at Scale” (Ilyas et al. 2024), whose deletion-based presentation of this family Fig. 2 follows.

References

Reproducibility. Every number on this page is computed by a seeded script in the site repository (scripts/influence-data/): the exact unrolled derivatives are propagated step-by-step alongside the classroom’s actual 1500-step training run and validated against finite-difference retraining before shipping; TracIn sums over 12 evenly spaced checkpoints of the same run; the segmented (SOURCE-style) estimate reuses the same recorded trajectory with the Hessian frozen at each of 3 segment midpoints. The convergence curve truncates the same recursion. The classroom’s exact leave-one-out values are part 1’s, reasserted at build time.

Acknowledgements. The backprop-through-training analogy that organizes this post, and the reading of SOURCE’s filter as training-effort damping, follow Louis Jaburi’s lecture notes “Data (Attribution) for Alignment” (Iliad Intensive, April 2026).