Does Your Setup Work—or Do You Only Remember the Winners?

The chart examples that stay in memory are usually the dramatic ones: the perfect breakout, the exact support bounce or the clean pattern that reached its projected objective.

Backtesting forces a harder question: What happened across every valid example—not only the memorable ones?

What backtesting is

Backtesting applies defined rules to historical data to examine how a strategy would have behaved.

It can be:

• Manual: move through past charts one candle at a time and record each valid setup.
• Rule-based or coded: translate objective rules into software and test a larger dataset.
• Hybrid: use software to find candidates, then review contextual elements manually.

Backtesting cannot recreate the future. It can reveal whether an idea deserves further investigation, where it struggles and whether its rules are clear enough to repeat.

Start with a hypothesis

A testable hypothesis connects a defined condition with a measurable outcome.

Weak:

Support and resistance works.

Stronger:

During a four-hour uptrend, the first pullback to prior resistance turned support followed by a one-hour higher low has produced positive expectancy under the specified exit rules across the selected historical sample.

The stronger version identifies:

• Market condition.
• Location.
• Setup event.
• Timeframes.
• Test metric.
• Need for defined exit rules.

It can be supported, rejected or refined.

Convert every judgement into a rule

Before opening historical charts, define:

Market and timeframe

Which instruments, sessions and periods are eligible?

Context

How exactly is an uptrend, range or transition defined?

Setup

Which structural and price conditions must exist?

Entry

What observable event creates the theoretical entry, and at what assumed price?

Invalidation

Where does the premise fail?

Exit

Fixed objective, opposing structure, trailing rule, time-based exit or another method?

Costs

What spread, commission, slippage or financing assumptions apply?

If you decide these after seeing each outcome, the test can unknowingly adapt to history.

Hide what happens next

Manual backtesting should move through history chronologically with future candles hidden. Make the decision using only information that would have been available at that moment.

Seeing the full chart creates look-ahead bias. Once you know the outcome, unclear structures suddenly look obvious and failed setups become easy to exclude.

Record every valid occurrence before revealing the next candles.

Build a representative sample

Ten examples can expose unclear rules, but they rarely establish confidence.

Test across:

• Different market conditions.
• Quiet and volatile periods.
• Several instruments if the strategy claims to generalise.
• Both successful and unsuccessful examples.
• Enough occurrences that one outlier does not dominate the result.

There is no universal sample size that guarantees reliability. More observations are generally helpful, but data quality and rule consistency matter as much as quantity.

Record more than wins and losses

For each theoretical trade, capture:

• Date and instrument.
• Context and setup timeframe.
• Screenshot before the outcome.
• Entry, invalidation and exit according to the rules.
• Planned risk unit, commonly expressed as R.
• Result in R after assumed costs.
• Maximum favourable and adverse movement if relevant.
• Rule-compliance notes.
• Market condition tags.

One R represents the amount theoretically at risk if the invalidation is reached. A result of +2R means twice that planned risk; −1R means the full planned risk was lost. R makes outcomes more comparable across different prices and instruments.

Expectancy combines frequency and size

Win rate alone is incomplete. A strategy can win often but lose more on its losing trades than it earns on winners.

A basic expectancy calculation is:

Expectancy = (Win rate × Average win) − (Loss rate × Average loss)

Hypothetical example:

• Win rate: 45%
• Average win: 1.8R
• Loss rate: 55%
• Average loss: 1R

Calculation:

(0.45 × 1.8R) − (0.55 × 1R) = 0.26R

The sample expectancy is +0.26R per theoretical trade before considering uncertainty, execution differences and future change.

Positive historical expectancy is not a promise of future results.

Metrics worth reviewing

Beyond expectancy, consider:

• Number of observations.
• Win rate.
• Average win and average loss.
• Largest historical losing sequence.
• Maximum historical drawdown in R.
• Profit factor: gross theoretical gains divided by gross theoretical losses.
• Results by market condition.
• Results before and after costs.

Do not optimise every weak metric by adding another rule. Complexity can fit the past while making future behaviour less reliable.

The main backtesting biases

Look-ahead bias

Using information that was not available at the decision point.

Selection bias

Choosing instruments or periods because you already know the setup performed well there.

Survivorship bias

Testing only markets or assets that survived and remained prominent, while excluding failed or delisted ones where relevant.

Overfitting

Adding filters until the rules describe historical noise rather than a durable idea.

Ignoring costs

Assuming perfect fills with no spread, slippage, commission or financing.

Inconsistent discretion

Accepting visually similar setups when they win and rejecting them when they lose.

Separate development from validation

An in-sample period can be used to develop and refine the rules. A separate out-of-sample period tests those frozen rules on data not used during development.

If the strategy performs well only in-sample, it may be overfitted. If the rules are changed after seeing the out-of-sample result, that data has now influenced development and is no longer a clean validation set.

After historical testing, use a forward test in a simulated environment before considering real financial risk. Real-time decision pressure, missed entries and execution conditions are not fully captured by a chart replay.

A losing streak is not automatically strategy failure

Random distribution means losses can cluster even when a strategy has positive historical expectancy. Conversely, a short winning streak can make a weak process look excellent.

Compare current performance with the tested distribution, rule adherence and market condition. Do not rewrite the strategy after every outcome—but do not ignore evidence that the current environment or execution differs materially from the test.

The backtesting worksheet

1. State the hypothesis.
2. Freeze the initial rules.
3. Select the sample without choosing known winners.
4. Hide future candles.
5. Log every valid setup.
6. Include realistic costs.
7. Review expectancy and drawdown in R.
8. Segment results by condition.
9. Validate on unseen data.
10. Forward-test in simulation.

Part 2 final knowledge check

Return to Speed Funded Chart School

Important educational notice

This lesson and its calculations are provided for general educational purposes only. Historical, hypothetical and simulated results have inherent limitations and do not represent actual trading or guarantee future performance. Real execution can differ because of spreads, slippage, liquidity, missed trades, changing conditions and human decisions.