TRADING TECHNIQUES
Using Statistics With Trading Systems
by Jeffrey Owen Katz, Ph.D., and Donna L. McCormick
In part 1, Katz and McCormick looked at the underpinnings of how statistics can help the trader determine the feasibility of a system. This month, in part 2, they explain the steps necessary to evaluate trading system behavior with the use of statistics.
Last month, I laid the groundwork for how statistics can help the trader determine whether the performance of a system is due to chance, or if the trading model is valid. I explained how to use samples to make inferences about the populations from which they are drawn. Another issue I addressed was that of optimization Ñ improving system performance by adjusting its parameters until the system performs its best on what the developer hopes is a representative sample. If the result is sufficiently good or the sample on which it was based sufficiently large so as to bring the probability of getting something by chance alone down to a sufficiently small value, there may still be a very significant result, even if many parameters have been optimized.
Next, I presented two examples. First, I optimized a system on one sample of data (the in-sample data) and second, I ran the system on another sample (the out-of-sample data), in both cases calculating a variety of statistics. I used the lunar model trading system from my "Lunar cycles and trading" STOCKS & COMMODITIES article, the TradeStation code for which can be found in Figure 1. I used the Standard & Poor's 500 index as the market, employing end-of-day, continuous contract data. First, let's recap the two examples introduced in the last issue and the information pertinent to both, after which I will interpret the results.
FIGURE 2: THE USE OF STATISTICS TO EVALUATE AN UNOPTIMIZED SYSTEM. Evaluating an unoptimized system is the same as evaluating an optimized one on out-of-sample data not seen during the optimization process. In both cases, you're running one test on a system without tweaking any parameters. Here, the out-of-sample test data may be found together with a variety of statistics.
Jeffrey Owen Katz, Ph.D., is a professional trader and consultant in Selden, NY. His firm, Scientific Consultant Services Inc. (516 696-3333), specializes in custom programming of trading systems and tools, provides expert consultation on systems development and the use of Omega Research tools, and develops cutting-edge software for traders. Donna L. McCormick is a writer and vice president of Scientific Consultant Services Inc.
Excerpted from an article originally published in the August 1997 issue of Technical Analysis of STOCKS & COMMODITIES magazine. © Copyright 1997, Technical Analysis, Inc. All rights reserved
.