Access to foreign exchange trading has opened up exciting trading options for the retail trader. You can now trade alongside corporations and institutions in a highly liquid market that is global, traded around the clock, and highly leveraged. Before jumping into this market,
however, we must understand the factors that affect the forex market. With that in mind, STOCKS& COMMODITIES has introduced Forex Focus to better prepare the retail trader to participate in the currency market.
Does backtesting produce valid results for foreign exchange trading? Here's a study that answers this question.
by Patrick Nouvion
For many years, backtesting and simulations have been banished from my toolbox. I only use these tools as debuggers and I never look at the results. I have told people that backtesting is useless without actual historical tick data, and only a dreamer would believe all these simulation reports. I just got my hands on historical tick data from my broker, so I thought I would do some testing. I decided to check to see if I was wrong in steering people clear of backtesting and overoptimized systems. Or was I right in assuming that without historical tick data, simulations were a waste of time?
The first thing I did was choose a system to test. The moving average convergence/divergence (MACD) sample that comes by default with MetaTrader is available for free and everyone who uses MetaTrader has access to it. Using the default settings on the MACD sample, I ran a simulation on a one-minute chart of EUR/USD from January 1, 2002, to April 1, 2007, using every tick option (Figure 1). The results are displayed in Figure 2.
FIGURE 1: SIMULATION ON ONE-MINUTECHART OF EUR/USD (1/1/2002 - 4/1/2007)
As you can see, the modeling quality was only 24.97%, which implies I did not have enough data to begin with. Note, however, that this was the data I had by default.
If you are a fan of backtesting, you will have heard people on other forums talking about downloading datafiles from different brokers to get a better modeling quality. To further test my stance on backtesting, I decided to run the same test (with the MACD sample) using some of the historical data trying to bring my test quality up to the level that hard-core simulators try to achieve.
After downloading a 10-megabyte file and importing the data into my platform, I was only able to achieve a 0.03% percent increase in my modeling (Figure 3). However, even that small of a difference in the data turned a positive end-result into a negative end-result. I would like to point out here that I do not give much credit to the modeling quality percentage and I will not focus on that number. What I am focusing on, however, is the difference between datasets.
FIGURE 3: NEW HISTORICAL TICK DATA. This resulted in only a 0.03% increase in the modeling.
It is a little-known fact that most trading packages have limits as to how much data can be stored in a datafile, and MetaTrader is no exception. That is why in a one-minute datafile not all of the ticks will be stored. For example, if I take a look at the tick data I downloaded from the other broker, the files are about 10 MB per currency pair and are only available for download as one-minute datafiles. This doesn't constitute a valid backtest.
So I went searching for valid tick data. I found that Interbank FX had the data, just not a method of distribution. The datafiles I received were about 500 MB on average per currency pair. Can you tell how many ticks are missing from the other files? The issue behind distributing historical tick data that so many want is that the files are just too big. They need to be converted into the right format so they can be imported into my trading platform easily. Steven Retz, one of the software engineers at Interbank FX, has written an application that will do just that. It will go through these enormous files and create a file with a compatible format.
The application looks like what you see in Figure 4. A couple of things should jump out. First, there is an option to add a spread. Second, there is an option to add swap values. In order to get a truly accurate simulation, or backtest, you will need historical values for both the spread and the swap, which is obviously not available. In this case, I will simply put in the current values and finish my test, hoping to prove that simulations as we know them are of little value for debugging logic and programming.
FIGURE 4: CONVERTING HISTORICAL TICK DATA. In order to get accurate backtesting, you will need historical values for the spread and swap.
I imported the historical data from my broker and ran a third and final test. The results can be seen in Figure 5.
FIGURE 5: A LITTLE CLOSER TO PERFECTION. Although the new historical tick data improved the results, it still didn't produce valid-enough results.
Ignoring the modeling quality and other options of the simulator, take a look at the profit. From the basic extrapolated data to the actual tick-by-tick data from my broker, the MACD sample system went from a positive $4.41 to a negative -$50.20. You should also note that the amount of trades placed were different between the two datasets, as were maximum drawdowns and other statistics.
My conclusion? This new historical tick data will bring backtesting
closer to perfection, but it will still not produce valid results. Now
you see why I have always warned people about simulations and optimizations.
And once more, nothing is better than real live trading, even if it is
only with micro lots.
Patrick Nouvion can be reached at psnouvion@gmail.com or through https://www.mistigri.net.Interbank FX
MetaTrader
Return to September 2007 Contents
Originally published in the September 2007 issue of Technical Analysis
of STOCKS & COMMODITIES magazine. All rights reserved. © Copyright
2007, Technical Analysis, Inc.