INTERVIEW
An Academic Turns Trader - Turns Academic
J. Doyne Farmer
by John Sweeney
The number of physicists and engineers who have dented their axes on market prediction is so vast that it led to the derisive term "quant." But who are these guys, and what are they up to? We tracked down one man who made the journey, successfully, from academia to research to trading, then back. We found him holed up in New Mexico, still pushing the boundaries of speculative thought and willing to chat.J. Doyne Farmer studied physics, receiving a bachelor of science degree from Stanford University and a doctorate from the University of California at Santa Cruz. He worked in the Theoretical Division at the Center for Nonlinear Studies at Los Alamos National Laboratories, where he was an Oppenheimer Fellow and founder of the Complex Systems Group. In 1991, together with Norman Packard, Farmer founded Prediction Co., a firm whose business is the automatic trading of financial instruments founded on time series-based directional forecasting methods. He is currently the McKinsey Professor at the Santa Fe Institute and editor-in-chief of Quantitative Finance.
STOCKS & COMMODITIES Editor John Sweeney caught up with Doyne (pronounced "Dowen") Farmer in early June 2001.
Over the long term, prediction is impossible. But over the short term, it is possible, if you can only figure out the process.
How did you get from physics to finance?
I worked in what's come to be called chaos theory. Part of the idea of chaos is that effective randomness can be generated from very simple, deterministic processes. Over the long term, prediction is impossible. But over the short term, it is possible, if you can only figure out the process. The catch is the processes that do this have to be nonlinear.
I became an expert in modeling chaotic processes. This involved building models from data and using them to make predictions. Some of the time series I analyzed included sunspots, glaciation data, medical data, and many other forms of data. The nice part about this kind of modeling is that you can make predictions without understanding all the details.
Without understanding the details? How is that possible?
More specifically, without making a model of what is going on. It's called empirical time-series modeling.
What's that?
The usual approach in science is to build models based on first principles -- that is, explaining the underlying behavior from a fundamental point of view, then building a model that reflects this understanding. By contrast, in an empirical time-series model, you just apply an automatic mathematical procedure that tries to find patterns in the data. And then you can use these patterns to make forecasts without having to actually understand what is going on.
Sounds like prices, but were you able to forecast sunspots?
Sunspots are relatively easy to forecast, at least if you compare them to something really difficult like prices. The question was whether we could beat the best competing forecasting methods. Most of the work prior to ours used linear models. We used nonlinear ones, and briefly held the best score for predicting sunspot cycles before other nonlinear models edged us out.
What turned you on to prices?
Whenever I would talk about my forecasting work, somebody would always ask, "Have you applied this to the stock markets?"
Which piqued your interest.
I had been at Los Alamos for almost 10 years, getting a little bored
with what I was doing, and I thought it would be an interesting challenge
to apply these methods to prices.
...Continued in the September 2001 issue of Technical Analysis of STOCKS & COMMODITIES
Excerpted from an article originally published in the September 2001 issue of Technical Analysis of STOCKS & COMMODITIES magazine. All rights reserved. © Copyright 2001, Technical Analysis, Inc.
Return to September 2001 Contents