Write the full rule before testing: eligible market, decision time, setup, entry, invalidation, sizing and exit. Keep an untouched sample for validation. Searching many variations until one looks profitable increases the chance of selecting noise.
Include realistic costs, failed fills and the sequence of trades. Evaluate drawdown, sample size and performance across different conditions alongside average returns. Paper trading can expose process problems but does not reproduce all emotional and execution effects of real money.
Worked example
A rule selected on 2020–2023 data can be tested on later data that was not used for parameter selection. Repeatedly tuning against that later sample turns it into another training set.
Check your understanding
Why does the best result from thousands of trials deserve extra skepticism?
Show explanation
Selection favors lucky noise. It needs independent validation and realistic implementation assumptions.
Progress is stored on this device.