SYSTEM DESIGN
Don't Curve-Fit That System
Building A Workable Trading Model
by Seth Weinstein
Statistical models allow you to reduce, if not remove, emotion from the trading process. Knowing the model's past performance gives you some comfort when you enter a trade, something a discretionary trader does not have. However, there are drawbacks to using quantitative models if they are not designed correctly. If a model is not properly designed and tested, it can be a dangerous tool. The key to using a quantitative model is to develop a robust, non?curve fitted system for trading.Having trouble designing a successful trading system? Here's how you can develop a robust, non?curve fitted system for trading. DEVELOPING A ROBUST STRATEGY
When you create a quantitative strategy, you need to find a statistical pattern from historical data that will continue to work going forward. A robust model should perform in actual use in a similar manner as it did in historical tests. So what can you do to increase the odds that a model will be a long-lasting one?
OUT-OF-SAMPLE TESTING
One precaution you can take to ensure that a model is robust is to use out-of-sample testing during the optimization process. Optimization is the process of testing for the most effective values for the inputs of the system being designed. Do not use the full range of historical price data to perform out-of-sample testing when optimizing. By leaving out a section or sections of historical data from the optimizing, you can see how the system would perform on data outside of the optimization. This process helps you avoid curve-fitting? the system.
There is great value in testing a system over market conditions that have not been used in the optimization process. Another way to look at out-of-sample testing can be seen from this example: You would like to develop a system and watch how it performs for a year before implementing it. Theoretically and statistically, designing and optimizing a system over historical data, while leaving out the last year of data for out-of-sample testing, performs the same function.
LOW NUMBER OF INPUTS
One of the best ways to reduce curve-fitting is to keep the number of inputs in the model relatively low. The larger the number of variables in the system, the easier it is to create an overoptimized system that will underperform in real time. By increasing the number of inputs to extremely high levels, you could even write a system that trades around a price series made up of random moves.
...Continued in the February issue of Technical Analysis of STOCKS & COMMODITIES
Excerpted from an article originally published in the February 2005 issue of Technical Analysis of STOCKS & COMMODITIES magazine. All rights reserved. © Copyright 2005, Technical Analysis, Inc.
Return to February 2005 Contents