Manual or automated backtest: which to choose?

There are two main ways to backtest a strategy: by hand, replaying the chart candle by candle, or automatically, letting a program apply your rules over years of data in seconds. Each has its strengths and weaknesses, and the right choice depends on your strategy and your goal. This guide compares manual and automated backtesting to help you choose.

• Manual backtest builds intuition but is slow and limited in volume.
• Automated backtest is fast and massive but requires codable rules and clean data.
• Manual suits discretionary strategies, automated suits mechanical ones.
• Many combine both: manual to learn, automated to validate at scale.

When we talk about backtesting, we often imagine sophisticated software testing strategies over decades of data. But that's only one of the two approaches. The other, manual, only requires a chart and patience, and remains one of the most instructive for a trader. The two meet different needs, and confusing them, or picking the wrong one for your situation, is a mistake that costs time and sometimes money.

Understanding the strengths and limits of each approach saves you from wasting time with the wrong one. A discretionary trader doesn't have the same needs as a systematic trader, and wanting to automate a strategy that rests on judgment is as futile as testing by hand a strategy that requires thousands of trades. This guide helps you choose the approach suited to your case, and avoid the pitfalls specific to each, which can turn a promising backtest into a false promise.

Manual backtesting

Manual backtesting consists of scrolling through a historical chart, one candle at a time, and applying your rules candle by candle, noting each trade you'd have taken. It's slow and tedious, but it has a virtue nothing else replaces: it makes you live your strategy from the inside. By replaying hundreds of situations, you develop an intuitive recognition of your setups and a fine understanding of their behavior.

This approach is ideal for learning and for discretionary strategies, those that rest partly on the trader's judgment and don't reduce to purely mechanical rules. It's also accessible to all, without code or expensive software, which makes it the natural entry point for beginners who want to understand trading from the inside before investing in more sophisticated tools. Its flaw is obvious: it's slow, which limits the number of trades you can reasonably test, and it's subject to hindsight bias if you lack rigor. An afternoon of manual backtesting rarely gives more than fifty trades, which is valuable for understanding a strategy but insufficient to validate it statistically.

Automated backtesting

Automated backtesting entrusts the application of your rules to a program, which executes them over years of data in seconds. Its advantage is spectacular: it lets you test thousands of trades across multiple instruments and periods, giving far more robust statistics than a necessarily-limited manual backtest. For a mechanical strategy, it's incomparably faster and more complete. Where manual testing takes hours to cover a handful of months, automated testing runs through decades of data while you make a coffee.

But automation has a price. It first requires your strategy to be entirely codable in objective rules, with no judgment zone, which excludes discretionary approaches. It also demands quality data and technical skills to code and verify the test, or failing that, a budget to pay someone who has them. And it hides a major trap: the ease of optimizing dozens of parameters until a magnificent but overfitted backtest, which will never work live.

The comparison table

Let's summarize the essential differences between the two approaches, to clarify the choice based on your profile and strategy.

This table shows that no approach is superior in absolute terms: they meet different needs. Manual wins on training and accessibility, automated on speed and volume. The right choice doesn't depend on which is best, but on which matches your strategy and your goal of the moment. Keep this table in mind next time you hesitate between the two: it settles most cases in seconds.

Which to choose based on your strategy

The decisive criterion is the nature of your strategy. If it's discretionary, resting partly on your judgment, your reading of context, your feel of the market, then the manual backtest is the only one possible, because these elements can't be coded. Wanting to automate a judgment-based strategy amounts to betraying its essence, and will give a test that doesn't really test what you do.

If your strategy is purely mechanical, defined by objective rules with no judgment zone, then automation becomes possible and often desirable, because it gives you the statistical robustness of large numbers. Many systematic traders in fact start with a manual backtest to understand their strategy, then move to automated to validate it at scale once the rules are well fixed. Between the two, there are also hybrid, semi-discretionary strategies, where only part of the rules is codable: in that case, a manual backtest on the discretionary part usefully complements an automated backtest on the mechanical part.

Combining both approaches

For many traders, especially those transitioning from a purely discretionary style toward something more systematic, the best solution isn't to choose but to combine. The manual backtest first serves to learn: replaying hundreds of candles to understand your strategy, refine your rules, and develop an intuition of your setups. It's an irreplaceable learning phase that automation can't offer.

Once the rules are clarified and stabilized by manual work, automation takes over for large-scale validation, over many trades and periods. This sequence, manual to understand then automated to validate, gets the best of each approach. It avoids the double pitfall of the trader who automates too early a poorly-understood strategy, and the one who stays stuck in manual without ever validating their edge over a sufficient sample. In practice, a few weeks of manual backtesting are often enough to know whether an idea deserves to be coded, before investing the time needed to automate it.

The pitfalls of manual backtesting

Manual backtesting has a specific pitfall you need to know: hindsight bias. When you scroll through a historical chart, you already know what comes next, even without consciously wanting to, and this knowledge colors your reading of the context at that moment. You see a setup and think, that was obvious, when in real time, without knowing what followed, that setup was probably far more ambiguous. This bias artificially inflates the results of a poorly run manual backtest.

The other classic pitfall is insufficient logging. Many traders scroll through dozens of candles in their head without noting each trade with the same rigor as live trading: exact entry, stop, target, reason for taking it. Without this logging discipline, it's impossible to know afterward whether the backtest truly reflects your strategy or an idealized version of it. The fix is simple: treat every backtested trade with the same rigor as a real trade, down to the screenshot.

The pitfalls of automated backtesting

Automated backtesting has its own pitfalls, more technical but just as destructive. The most insidious is look-ahead bias: your code uses, without you realizing it, information that wouldn't have been available at the time of the trade, for example the closing price of a candle that hasn't finished forming. This bias produces results impossible to reproduce live, and it's particularly hard to detect without rereading your code line by line.

The other pitfall is data quality. A price feed with gaps, errors or insufficient granularity silently distorts every downstream result, with nothing looking abnormal. You also need to include realistic costs (spread, commission, slippage), otherwise the backtest systematically overstates real performance. An automated backtest is only as reliable as the data feeding it. Always check your data source against a known sample before running a full backtest, comparing a handful of candles to another provider: a gap of a few points on a price is enough to turn a marginal result into a losing one.

How many trades do you need to trust it?

The volume question comes up often, and the answer depends on the method. In manual backtesting, imagine a trader scrolling through three months of hourly chart on one instrument: they might get 40 to 60 trades, which starts to show a trend but remains a fragile sample for solid statistics. In automated backtesting, the same strategy tested over five years of data can generate several hundred, even thousands of trades in seconds, giving statistical robustness manual backtesting can't reach.

That's not a reason to dismiss manual backtesting though: a sample of 50 carefully documented trades, deeply understood in context, is often worth more than 5000 trades generated by a poorly understood strategy you never lived through. The question isn't only how many trades, but also what you understand about each one. Numbers reassure statistically, understanding protects you live when data diverges from the past.

Documenting your backtest, whatever the method

Whether you backtest by hand or with a program, documentation makes all the difference between a useful exercise and a waste of time. For manual, that means systematically noting each simulated trade with a screenshot, as if it were a real journal entry. For automated, that means keeping the code, the exact parameters and the tested period, so you can reproduce and verify the results later.

This documentation gets a second life once you go live: it becomes your point of comparison. You know what to expect (win rate, typical drawdown, losing streak) and you can verify, trade by trade, that your real system behaves as predicted. Without this trace, every backtest stays an isolated exercise with no follow-up; with it, it becomes the first chapter of ongoing tracking.

A concrete example: manual then automated

Let's take a concrete example to illustrate the approach. Imagine a trader who wants to validate a reversal setup on a pullback to a moving average at the end of an uptrend. In manual backtesting, they scroll through six months of a 15-minute chart on an index, note every occurrence of the setup, its theoretical result respecting a 1:2 risk/reward ratio, and get, say, 45 trades with a 55% win rate. That's an encouraging starting point, but the sample remains too small to draw a definitive conclusion.

To go further, this same trader then codes the setup's precise rules (position relative to the moving average, pullback distance, trend filter) and runs an automated backtest over five years of data and several comparable indices. If they get, say, 800 trades with a win rate close to 52% and a profit factor above 1.3 across most tested instruments, the statistical robustness confirms what the manual test had only suggested. If instead the automated results collapse outside the initial period, it's a clear signal the setup doesn't generalize, despite looking good on the manual sample.

How Tradoshi complements your backtests

Whatever your backtest style, Tradoshi takes over on your real trades: it tracks your statistics in live conditions to verify that your strategy, tested manually or automatically, keeps its promises.

• Real statistics to compare to the benchmarks from your backtest, manual or automated.
• CSV import to integrate the results of a backtest or demo account into your tracking.
• Breakdowns by instrument and period to see where your strategy holds live.
• Tracking over time to detect overfitting, when a backtest edge doesn't hold live.
• Searchable history at any time, to compare a new idea against previous backtests.

Frequently asked questions