You Are Overfitting Your Training the Same Way You Overfit a Backtest

Maciej Lewandowski · July 2026 · Engineering notes

I keep a training log for the same reason I keep metrics on a service: I do not trust my memory, and a number written down at the time beats a feeling recalled later. What I did not expect was how fast that log started producing the exact class of wrong conclusion I spend my working hours preventing in a production time series.

So this is not a training article. It is an article about a small, noisy dataset that I happen to generate with a barbell, and about the fact that almost every failure mode in it has a twin in quantitative finance. If you have ever shipped a strategy that looked magnificent in simulation and did nothing live, you already know this material. You have just never pointed it at yourself.

Your log is a time series with n = 1 and no control group

The structural problem comes first, because everything else is downstream of it. You cannot run the counterfactual. You will never know what the other program would have done to the same you, in the same year, with the same sleep and the same job stress. One subject, one arm, no randomization, no blinding, and an experimenter who is also the subject and badly wants a particular answer.

Every conclusion you draw is therefore a single realization of a noisy process: a sample of size one from a distribution you have never seen the shape of. "This program worked for me" is a true statement about what happened and a nearly worthless statement about what caused it.

The noise floor is larger than the weekly signal

Now the measurement problem. Day-to-day performance moves with sleep, hydration, food timing, stress, caffeine, ambient temperature, time of day and where you happen to be in the week. Stack those together and the spread between a good day and a bad day routinely exceeds the real strength gained over that entire week. The signal you are trying to read sits under the noise, not above it.

Then add instrument error, which people forget entirely. RPE is a subjective sensor with a drifting calibration. Bar speed is not measured, it is guessed at from the inside, which is the worst available vantage point. "One more rep in the tank" is an estimate produced by a tired person with an opinion about how the session should be going. These are not bad tools, just noisy ones, and sensor noise adds to plant noise.

The consequence is uncomfortable: a single bad session carries almost no information. It is exactly what you should expect several times a month from a process this variable. And it is, reliably, the data point people act on.

Program hopping is parameter search on noise

Here is where it becomes the same article I already wrote about markets. Changing your program every four weeks on four data points is structurally identical to running two hundred parameter combinations over a price series and shipping the best equity curve. You are not selecting on effect, you are selecting on variance, and the more configurations you try, the higher the best observed result you should expect from chance alone.

The cruel part is the feedback. The switch is very often followed by improvement, which feels like confirmation and gets logged as such. It is not confirmation. It is the next section.

performance weeks you change your program here the only thing that was ever real

Regression to the mean explains most breakthroughs and most crises

If a week was unusually bad, the most likely reason is that several independent noise terms landed on the same side at once, and the most likely next week is closer to average. Not because anything healed, and not because of whatever you changed on Sunday night. Because extreme draws from a noisy distribution are, by construction, usually followed by less extreme ones.

The same mechanism runs in the other direction, and that is the one nobody wants. An unusually good week is also mostly noise, and it will also come down. Whatever you happened to be doing in between collects the credit or the blame in both cases. Neither inference has any content.

Regression to the mean is the single most useful statistical idea available to anyone who trains, and the reason is blunt: it is the default explanation for the two events that trigger the most decisions. Before you attribute a bounce-back to your intervention, ask what the bounce-back would have looked like if you had done nothing at all. Usually: about the same.

Survivorship bias in training advice

You hear from the people the program worked for. That is not a mild bias, it is the entire selection rule. The lifters who stalled, got hurt or quit do not write the retrospective and do not post the log, so the visible sample has been filtered on exactly the outcome you are trying to predict. Five enthusiastic write-ups tell you five people liked it. They tell you nothing about the denominator.

Elite-athlete programming is the extreme case. Those templates are the surviving output of a brutal filter: genetic outliers who also happened to tolerate very high volume, selected out of a much larger pool who did not and left the sport. Copying the program copies the treatment without copying the selection, which is the same error as backtesting today's index constituents and calling the result alpha.

What actually carries signal

Everything above is an argument for longer windows and aggregates. The daily numbers are not wrong, they are just mostly variance, and variance shrinks when you sum or average it. What survives the noise floor:

  • Estimated 1RM trends over months, not weeks. A monthly aggregate has enough sessions behind it to average out the bad days; a weekly one does not.
  • Total tonnage, or hard sets per muscle group per week, read as a moving average. This is your actual dose variable, and the one people track least.
  • Bodyweight as a 7-day mean. Daily bodyweight is dominated by water, glycogen and gut contents. The 7-day mean is a measurement; a single morning reading is a mood.
  • Sleep as a rolling average, because one short night is an event and a fortnight of short nights is a state, and only the second one is steering information.

That gives a single rule that does most of the work: if a metric can swing meaningfully between Tuesday and Thursday, it is not a metric you steer by. Look at it, log it, do not turn the wheel on it.

Autoregulation is closed-loop control

Fixed-percentage programming is open loop. It computes today's prescription from a number you measured weeks ago and assumes the plant responds identically every day, which we have already established it does not. There is no feedback path at all: the controller cannot see the output it is producing.

RPE and RIR-based autoregulation closes the loop: the setpoint moves with today's readiness, which is what you want when the plant's gain varies with sleep and stress. Worth being precise about the size of the claim, though. The sensor is subjective and noisy, so this is a controller with a bad sensor, not a good one. That still beats no controller, on one condition - you must not chase every wobble. A high-gain controller reading a noisy sensor oscillates. Deloads and long averaging windows are the low-pass filter that keeps the loop stable.

planned load the session measured response RPE, bar speed, sleep adjust autoregulation: the plan reads the signal an open loop skips the middle box and calls the result discipline

Progressive overload is constrained optimization

The objective is adaptation. The constraint is a recovery budget that is itself a function of sleep, stress, calories and time, and which you do not get to set. Most training arguments are really arguments about the shape of that constraint, conducted by people who have not noticed it is a constraint.

The modelling detail that matters: it behaves like a hard constraint, not a soft penalty. Overshoot a soft penalty and the objective degrades a little. Overshoot this one and the objective collapses - the trajectory stops rather than dips, and you spend a while optimizing a different problem. An optimizer that treats a hard constraint as soft walks straight off the edge of the feasible region.

The honest ending: the data is mostly for adherence

Here is the part that undercuts everything above, and I think it is the true part. The log does not work because the analysis is clever. It works because writing it down forces consistency and stops you being stupid, and those two effects dwarf every insight you will ever extract from the numbers.

Over a five-year horizon the variable that dominates every model is how many sessions you actually did, and it is not close. Program selection, periodization scheme and exercise order all compete for the residual left after attendance has taken its share. No dashboard has ever improved attendance. Showing up improves attendance.

This is the same lesson as a codebase, stated in a different unit. The sophisticated tooling matters far less than doing the boring thing consistently for years. A test suite nobody runs, a metrics stack nobody reads and a training log nobody acts on all fail identically: the instrumentation was never the bottleneck.

The checklist

  1. Do not change a program on fewer than eight to twelve weeks of data. Four weeks is four data points from a process whose noise exceeds its weekly drift.
  2. Judge by monthly aggregates, never by individual sessions.
  3. Average bodyweight over seven days. Treat the daily number as an input to the average and nothing else.
  4. Expect regression to the mean after any extreme week, in both directions, and refuse to attribute it to whatever you changed.
  5. Treat a bad session as noise until it has repeated enough times to be a trend.
  6. Change one variable at a time, or you have learned nothing about which one moved the result.
  7. Track the dose variable, hard sets or tonnage per week, as a moving average. It is the input; everything else is output.
  8. Remember that consistency dominates every other term in the model, and optimize that first.

None of this makes a training log true. It makes it falsifiable, which is the most a single-subject record can offer. The smooth trend line is the hypothesis, the noisy one underneath is what you actually measured, and the discipline is refusing to act on the difference until it has survived a few more months of not going away.

  • Your Backtest Is Lying to You — Look-ahead bias, survivorship, point-in-time data and overfitting - why a backtest is a replay engine, and what that demands of it.
  • AI Coding Agents in Production: What Actually Changes — A field-tested operating model for using AI coding agents in production: context, task decomposition, review, guardrails and the real verification cost.
  • Balance at 14:00 UTC Does Not Exist — Chains never close, so a point-in-time balance has to be derived, not queried. The anchor problem, the per-chain traps, and why reproducibility is the real requirement.