January 1997
Letters to the Editor
Editor,
REGIONAL RULES
As I'm relatively new to technical analysis and computerized trading systems, your magazine has been very helpful in my learning process.
I was fascinated by Dennis Meyers's article, "The siren call of optimized trading systems," in the October 1996 STOCKS & COMMODITIES. However, Meyers failed to point out a most important conclusion.
Briefly, Meyers optimized a simple RSI trading system (that is, buy when the four-day RSI breaks through 55 and sell when the four-day RSI returns back through the 85 level) by backtesting 10 years of randomly generated prices (that is, daily price changes randomly chosen between +3% and -3%). This system produced $185.39 of net profit over 10 years on a starting investment of $205.17, without any reinvestment of profit/loss to compound the returns, yielding an acceptable 9% simple annual return from a market that clearly has no fundamental driving forces.
Meyers concluded that backtesting random price movements can give the impression of predictable profitability that, being random-based, may not be repeated in the future. He rightfully advises validating the prediction reliability of a trading system using out-of-sample testing (that is, optimize nine years of data and then test the trading system with the 10th year's data to judge the system's probable profit-per-performance).
More important to me, Meyers clearly demonstrated the value of rational rules for buying and selling. His study proves that, even in a purely trendless situation, rigorously adhering to intelligent decision rules can produce good profits. Logically, if the price action is driven by fundamental forces and the optimized trading system (decision rules) can correlate to those forces, the system should generate profit on an ongoing basis, especially if those decision rules are adjusted periodically, through out-of-sample optimizing, to detect any real-world shifts in the fundamental forces.
Most investors and traders I know make most of their buy/sell decisions on subjective and emotional judgments. Those who harness these judgments with rational and measurable decision rules generally have much better and more consistent results.
Thanks for providing a fine and very helpful magazine.HERBERT E. GEISSIER
Upper St. Clair, PA
Editor,
ASYMMETRY IN SYSTEM RULES
In the October 1996 issue of STOCKS & COMMODITIES, two articles commit what to me is an error of logic: They sanction the optimizing of trading systems that use asymmetrical buy and sell rules.
In "The siren call of optimized trading systems," author Dennis Meyers uses different range values while optimizing his buy zone (which ranges from 20 to 70) and his sell zone (which ranges from 40 to 90), whereas in your interview with Nelson Freeburg ("Modeling the markets with Nelson Freeburg") in the same issue, Freeburg suggests optimizing a system that buys when silver climbs above its eight-day moving average and sells when silver drops below its 17-day moving average.
Do any other readers see the problem with such "lopsided" systems design?PETER PARKINSON
Ottawa, Ontario, Canada
Editor,
PERILS OF OPTIMIZING
I enjoyed Dennis Meyers's article in your October 1996 issue about the perils of optimized trading systems.
It seems to me that this article, if given credence, essentially destroys the basis for all technical trading systems, not to mention eliminating any reason for reading your magazine or buying any of the products advertised in its pages.
Meyers claims that optimizing a trading system over a set of historical data offers no guarantee of improving results unless it is then tested against what he calls "out-of-sample" data -- that is, data not used to optimize the system.
Suppose one does just that and finds the system doesn't work for the out-of-sample data. What next? The logical thing would be to vary the trading parameters to create a new starting point, then reoptimize in the hope that the next optimal parameter set might also work on the out-of-sample data.
In other words, you keep trying until you've found an optimal parameter set from the sample data that also works on the out-of-sample data. The end result: Eventually your system is optimized over both the sample data and the out-of-sample data! You're right back where you started!
So there's no way to optimize a trading system, period.
This would seem to leave us where critics of technical trading systems have attempted to lead us before: The market is a computer that processes information about the future. No amount of information about the past can beat buy-and-hold. The only way to beat buy-and-hold is to have information about the future that other traders don't have, or to be better than other traders at evaluating public information about the future.STAN JONES
Anchorage, AK
via E-mail
The Editor replies:
There certainly are traders who are successfully using technically based systems. Our readership can attest to that. I presume with confidence they've backtested their methods to ensure viability. Do you believe that all successful technical traders are simply flukes?
Dennis Meyers replies:
I've encountered this type of argument many times before in many different guises.
Before I proceed, let me point out that the two professors who received the 1996 Nobel Prize in economics propounded the theory of incomplete and asymmetrically distributed information. Buyers and sellers don't all have the same market information at the same time, and this informational advantage can often be exploited strategically. Thus, an individual with superior information -- whether gleaned from mathematics, computers, superior intelligence or other means -- can use that edge to take advantage of the market. In the framework of technical trading, this means superior returns over buy-and-hold strategies. Thus, your statement, "No amount of information about the past can beat buy-and-hold," no longer has the support of even renowned economists (not that we ever needed their support)!
Your first error in logic stems from not recognizing that if a system, no matter how rational it may seem, and its optimized parameter values fail in out-of-sample testing, then the system doesn't correspond to the real world and should be dumped -- not fiddled with until it works.
Your second error in logic revolves around the definition of future. The future is unknown. There is no "information about the future," as you state, public or otherwise. There are only guesses about the future. These guesses can only be formed from experience and knowledge gained from what has happened in the past. All a system does is formulate a set of rules and procedures. Systems are supposedly created from the same experience and knowledge that creates the guesses. However, with systems, we have a means of testing their validity before future profits or losses give us the final and conclusive test. One of the most popular validation procedures is out-of-sample testing. As I already mentioned, if the system fails in out-of-sample testing, it means that the system's formulation does not correspond to the real world. The system should be abandoned, not played with until it works!
It should be noted that successful out-of-sample testing does not guarantee that a system will perform well in the future. For example, successful out-of-sample testing can occur purely by chance or from incorrectly selecting a data sample for the out-of-sample testing. A successful out-of-sample test increases the probability that you've captured the price dynamics for the price series with a given trading model. However, only experience in model development and knowledge can give you a feel for what that probability is. Nothing can guarantee future results!
Finally, the point of my article was to demonstrate in a simple manner that hypothetical curve-fitted results and anecdotal examples are worthless in determining future performance, because it cannot be known whether a system has correctly modeled the price dynamics or has curve-fitted a random price series. The article in no way demonstrated or implied that properly applied optimization procedures do not work.
Contributing Editor Dennis Meyers has a doctorate in applied mathematics in engineering. He is a member of the Chicago Board Options Exchange (Cboe), a financial institution consultant and a private trader. He can be reached through E-mail at meyersx@enteract.com. -- Editor
Editor,
ERRATA
In case someone hasn't already pointed it out, a technical error exists in the November 1996 issue in the article titled "Stay in phase" by John Ehlers.
On page 74, two additional lines of TradeStation code are needed at the start of the indicator for the phase calculation to work correctly:
RealPart=0; ImagPart=0;
Thus, the entire Tradestation code should look as follows:
inputs: DomCycle(15); vars: RealPart(0), ImagPart(0), Weight(0), Phase(0), J(0); RealPart=0; ImagPart=0; for J = 0 to DomCycle -1 Begin weight = close[J]; If DomCycle <>0 then Begin RealPart = RealPart + Cosine(360 * J / DomCycle) * Weight; ImagPart = ImagPart + Sine(360 * J / DomCycle) * Weight; end; end; If AbsValue(RealPart) > .001 Then Begin Phase = ArcTangent(ImagPart/RealPart); end else Begin Phase = 90 * Sign(ImagPart); end; If RealPart < 0 then Phase = Phase +180; Phase = Phase + 90; If Phase < 0 then Phase = Phase + 360; If Phase > 360 then Phase = Phase - 360; plot1(Phase, "Phase");JOHN VALENTINE
via E-mail
John Ehlers replies:
Mr. Valentine is correct. The two additional lines are required. The reason is these two variables are accumulated for each price bar over the last full cycle length. Thus, when the calculations for a new bar are started, these variables must be reinitialized. I thought the settings in the vars: statement did this, but it does not. I salute Mr. Valentine for properly identifying the code error.
