Backtesting stocks, forex and futures: what each market changes in the data and the simulation

Backtesting stocks, forex and futures means applying the same method to three kinds of data that have little in common: stock prices that must be adjusted, forex quotes that belong to your broker alone, futures contracts that expire. This guide says, market by market, what the data changes and what the simulation has to change with it, so that the backtest measures your strategy and not the flaws of its price series.

• Stocks: an adjusted or raw series depending on your strategy, a sample that keeps the companies that disappeared, an opening auction and halts.
• Forex: no central market, so a spread and ticks specific to each broker, a volume that counts price changes, a swap every night.
• Futures: expiries, a splice between contracts that distorts past prices, a tick value and margins set by the exchange.
• Everywhere: rules written beforehand, simulated costs, a long enough sample, an out-of-sample test.

The method is detailed in how to backtest a trading strategy honestly, and the choice between a test by hand and a coded test in manual vs automated backtesting. Here, only what depends on the market.

What does not change from one market to another

Whatever the instrument, an honest backtest rests on four things: rules written before looking at the chart; simulated costs, spread, commission and slippage; a sample long enough to cross several market regimes; and a part kept aside to check the parameters where they were not fitted. Then comes the forward test, before a cent is at risk.

Options are a case apart, because their price depends on time and implied volatility: that is the subject of option backtesting. The vocabulary of testing is in the glossary under backtesting and replay, and the tools are compared in the guide to free backtesting software.

Stocks: prices to adjust and a sample that forgets the dead

A stock is a share of a company, and the company lives: it splits its shares, pays dividends, sometimes disappears from the market. Each event leaves a mark in the price series, and when backtesting stocks, reading that mark without understanding it means taking an accounting event for a market move.

Splits and dividends: the adjusted series or the raw series

When a company splits its shares, the price drops at once without anything having happened: according to the Nasdaq glossary, a stock at $100 before a two-for-one split trades at $50 afterwards, and each holder owns twice as many shares. On a raw chart that is a fall by half, and a breakout strategy will see a signal that never existed. Data providers correct this with an adjusted series: Yahoo Finance's help page on the adjusted close defines it as the closing price after adjustments for all applicable splits and dividend distributions, the data before the event being multiplied by a factor. A dividend has the same effect on a smaller scale.

Which series to choose? The one your strategy reads. Rules based on price levels are tested on the raw series, because that is what the market saw; rules based on returns or indicators, on the adjusted series, otherwise every split manufactures a false signal. The LEAN engine names both in its source code: in raw mode, dividends paid in cash and splits applied to the portfolio quantity; in adjusted mode, splits and dividends backward-adjusted into the price. Stock backtesting software that does not tell you which one it serves leaves you guessing half your result.

Survivorship bias: the delisted companies are missing from the database

The second trap lies in what is no longer on the chart. If you test on the stocks that make up an index today, you test on companies that survived until today: those that went bankrupt or were delisted are no longer on the list, and neither are their losses. The phenomenon reaches even the reference databases: in 'The Delisting Bias in CRSP Data', published in 1997 in the Journal of Finance, Tyler Shumway shows that the Center for Research in Security Prices database of US stocks lacks correct delisting returns for most of the stocks delisted for bankruptcy or other negative reasons since 1962, and that the omitted returns are large. If you are backtesting stock market data over decades, this is the bias that grows with your sample.

The remedy: test on the index membership at each date, not on its current membership, and keep the delisted names with their last price. Otherwise, read your results as an upper bound.

Hours, opening auction and trading halts

A stock has a session. According to the New York Stock Exchange hours page, the core session runs from 9:30 a.m. to 4:00 p.m. New York time, opens and closes with an auction, and some days close at 1:00 p.m. The first price of the day is therefore the result of an auction, and an order placed 'at the open' gets that price, not the previous close. A test on daily data that buys 'at the close of the signal' acts on information known only at 4:00 p.m.: that is look-ahead, and it flatters the result.

A stock can also be halted. Nasdaq's list of trading halt codes distinguishes, among others, a halt pending news (T1), a volatility trading pause (LUDP) and an SEC trading suspension (H10). During a halt there is no price, so no stop can be filled: a backtest fills a stop there at a price that existed for nobody.

Forex: your broker's data, not the market's

Forex has no exchange. According to the Bank for International Settlements' triennial survey, trading in over-the-counter foreign exchange markets reached $7.5 trillion per day in April 2022, and each broker serves its own quotes. For forex backtesting, there is therefore no such thing as 'the EUR/USD data', there is your broker's, and a test run on another broker's proves nothing about what your account would have lived.

A spread and ticks specific to each broker

Forex backtesting software shows it in its own settings. The 'Every tick based on real ticks' mode of the MetaTrader 5 tester runs, according to the MetaTrader 5 Strategy Tester documentation, on the real ticks accumulated by brokers. At cTrader, the help page on backtesting a cBot presents tick data from the server as the most accurate source, and lets you set the spread to a fixed value, to the current one, or at random between a minimum and a maximum. A forex backtester set to a fixed, narrow spread tests a version of the market that only exists in quiet hours and overstates every short-term strategy; the honest value is your broker's average spread in the hours you trade. Free forex backtesting software is not what is missing, the MetaTrader Strategy Tester ships with the platform; what is missing is your own broker's real ticks.

A volume that counts price changes

Forex volume is not a volume. The MetaTrader 5 documentation of the Volumes indicator says so: for the forex market, the indicator measures the number of price changes within each period, whereas for stocks it measures the volumes actually traded. A forex strategy that filters its entries on volume is therefore testing the activity of your broker's server: it will not transfer as is to another broker.

The overnight swap

Holding a forex position overnight costs or earns a swap, tied to the interest rate gap between the two currencies and set by the broker. MetaTrader 4's contract specification sheet defines swap long and swap short as the rollover size for a long or a short position, and the MQL5 documentation of symbol properties provides a day of the week on which a three-day swap is charged, to cover the weekend. Over several nights, it can eat the expectancy of a pair with a wide rate gap. A multi-day forex backtest that ignores it is not wrong, it is incomplete, and always in the direction that suits you.

Futures: contracts that expire, and a series that has to be stitched

A futures contract has a date. The March S&P 500 is not the June one: two instruments quoted at different prices, and the first ceases to exist at its expiry. The lesson on futures contracts lays the groundwork; for futures backtesting, ten years of testing cross forty quarterly expiries.

Expiries, front month and rollover

The CFTC glossary, from the US futures regulator, calls the nearest expiry the 'front month', and the more distant expiries the 'back months'; it defines the roll-over as lifting a position in the near month and re-establishing it in a more deferred delivery month. In a real account, the rollover is a trade: you close March, you open June, you pay the fees twice and the price gap between the two contracts. In a backtest, you have to decide when it happens and how it shows in the series.

The continuous contract: three ways to stitch, three different series

To backtest futures over several years, platforms build a continuous contract, a single series that chains the expiries. At the changeover, the new contract does not quote at the old one's price, and that gap is not a market move. The NinjaTrader 8 guide on the merge policy describes three answers: 'MergeBackAdjusted', where an offset aligns the historical prices with the next front month; 'MergeNonBackAdjusted', where the data stays raw; and 'DoNotMerge', which loads only the chosen expiry. LEAN's source code names the same methods differently: 'BackwardsPanamaCanal' adds at each splice a factor equal to the price difference between the two contracts, 'BackwardsRatio' multiplies the prices by their ratio, and the splice date is chosen on the last trading day, on the first day of the delivery month, or when the following contract has a higher open interest than the current one.

Consequences: the difference-adjusted series keeps the gaps in points, but its old prices were never quoted; the ratio-adjusted series keeps returns in percent, but distorts the gaps in points; and backtesting futures on the raw series manufactures a false gap at every expiry, which your strategy will take for a breakout. A strategy on absolute price levels therefore cannot be tested on an adjusted series, and before reading a result, ask your futures backtesting software which series it served you and on which date it rolled.

Tick, point value and margins

The CFTC glossary defines the minimum price fluctuation, the tick, as the smallest increment of price movement possible in trading a given contract, and the contract size as the actual amount of a commodity represented in a contract: each future has its tick, and each tick its money value. A futures backtest that counts its gains in points without converting them with the tick value of the contract tested is comparing things that do not share a unit. Position size is not computed as on stocks either: according to the same glossary, initial margin is the amount required by the broker when a position is opened and maintenance margin the amount that must be kept on deposit. The leverage of a future is therefore set by the margin: a simulation that takes positions your margin would not have allowed tests an account you do not have.

The comparison table

Where Tradoshi fits

Tradoshi has its own backtesting module, in beta: it replays history candle by candle without showing what comes next, simulates your orders with spread, slippage and commissions, and scores each session with the same statistics as your journal. Whatever the market, the journal then takes over: your real trades produce the statistics your backtest promised. The module is presented on the backtesting page.

Frequently asked questions