A Critique of Advanced Analyers - Beyond "Rocket Science"?

The advanced analyzers - the neural networks and bayesian correlator - are the most ambitious techniques available in the timeseries analysis module of StockWave; they are an attempt, in a precise and predictive sense, to 'do better' than simple random walks when calculating price movement probabilities. Of course, 'doing better' means in practice having the potential to make lots more profit, with much less risk; however, having reviewed their usage we feel that using them may seem difficult to the new user.

This short article is thus an attempt to give an honest appraisal of these techniques, hopefully in sharp contrast to the persistent shower of crackpots and sleazebags who pollute the internet with their outrageous claims and ridiculously misleading sales pitches - the advanced techniques can give good, or even excellent results, if used correctly, but we must tread warily, and never forget that there are no magic bullets. The main problem is that within the advanced analyzers we have an infinite possible choice of models; given that running a single test may take several hours one can see the challenges we face in doing any proper test. Thus -   

(Anyone who wishes to donate to us their high performance computing cluster for a few months, we would be most grateful.) But we did what we could, for as long as we had, and this is what we found.

Anecdotally, it seems that with the right network structure/pattern depth you can get very good results indeed, i.e. trends and changes of market direction and their timings, but model selection is not automatic, and our users cannot be expected to waste their time searching unfruitfully. Working on the basis of a 1000 point dataset we found a priori 'reasonable' parameter values for our analyzers - these are the values you will find already set when you fill-in the model selection form; we take these parameter sets as being sensible starting points. While experimentation and initiative are to be encouraged among the user, certain parameters are more important than others - for the neural networks these are the number of inputs and number of hidden nodes; the main parameter for the bayesian correlator is the pattern depth. The basic model selection strategy is to start with low values of these parameters then gradually ramp them up until you find suitable performance in backtesting; then you can try for an actual prediction.

Some useful and perhaps non-obvious, hints from our own experiments -

• Always start with a basic random walk; use about 20000 runs; this will be your baseline
• Don't use the preprocessing filters, at least to begin with (- see discussion below)
• More than one hidden layer in the feedforward networks seems to be unnecessary; it may be very interesting, but it is hard to see the practical benefit at this time.
• The effects of varying the minor parameters, e.g. bias, momentum factor, pattern metric, are as yet unknown.
  
• The bayesian correlator seems to be the most accurate analyzer, but is also the slowest.
  
• The neural network-based advanced analyzers will run in a few hours on a fast machine.
  
• The usual measures of training performance used in neural network training can be very misleading in this application. See more below.
  
• The recurrent and backpropagation neural analyzers should produce similar results due to their similar architecture.
• The analyzer based upon the Kohonen Self Organizing Map has a different structure to the feedforward networks - it gives a very useful comparison.
  
• All the analyzers are different and so give us, effectively, 'independent' opinions; so use all types of analyzers, not just whatever happens to be your favourite; so what if it takes a long time?! - set your tasks to run over the weekend, that's what its for, then go get pissed or climb a mountain ... your predictions will be ready for start of trading Monday.
• Use the analyzer preview in the task processor to get a look at 'work in progress' - this can give you an idea as to how things are developing and most importantly, if the analyzer is in some way, pathological. Occasionally you will come across an analyzer which is clearly not working - but these are easy to spot visually.
  
• Look for corroboration among your predictions.
• The predictions, i.e. the probability heatmaps, may look strange to you and you may be perplexed as to how to trade with them - don't worry, this is what the TradeSearcher is for; it may find you some really interesting possibilities, especially when combined with options strategies.
  
• Within the heatmaps, a crisp trajectory may eventually form - do not read too much into this; do not think 'I have predicted the stockmarket' and go on to trade on the basis of the apparent future share price trajectory; maintain the probabilistic mind-set at all times. (These 'crisp trajectories' may or may not form, and sometimes you can get more than one!)
  
• If you are going to trade for real, then check-out the news files, do a news event analysis and use the web agent to do some 'digging' about the company; this will give you an idea for the human factors involved and what external events could take place. Don't trade if you are getting any sort of  'bad vibe'. Ever. Just stop. Give it a miss.
  

Filters

Using signal processing techniques on share prices is interesting in its own right; by applying various transforms we can try to get a handle on what is the 'true signal' and what is the 'noise'. (Those of you who are fans of various 'moving average'-style indicators should also appreciate the filters we offer as they have no lag.) Our original expectation was that a preprocessing stage to filter the - very noisy - share price data, prior to any machine learning phase, could work wonders for our analyzers; our initial result, and we have to admit that it is slightly disappointing, is that we cannot discern any performance improvement.

To be clear on this - we do maintain our original belief in the hypothesis - that preprocessing, especially using wavelets, may allow significant predictive advantages, but that it is as yet unproven. Even if subsequently found to be correct, there are still several open questions we need to know the answers to -

• What filters to use with what analyzer; wavelets, in particular look like they should be the right thing for studying share prices, but there are many possible choices here.
  
• What threshold level should be used? If you 'overfilter' the timeseries, then you are effectively creating patterns/determinism where there is none - you are creating spurious order.
• Are there some stocks or classes of stocks which are more amenable to this kind of filtering than others?
• Is there a procedure for finding the 'right values' of filter and threshold, other than simple trial and error? Having such a process could make for enormous speed benefits.
  

Preliminary work has only been to differentiate in crude terms between smooth and spiky filters -

• Smooth filters, the FFT and 'high number' wavelets tend to smooth-out the timeseries during the denoising phase; in naive terms this seems to aid the neural network training immensely; you will find that the conventional error measures converge quickly to very small values. On the face of it this seems excellent - unfortunately, and surprisingly, we have found that it is possible to produce extremely bad predictions in this case - the analyzer heatmaps are totally pathological. Conversely, very poorly trained networks - ones which we would typically classify as having 'failed to converge' can often produce very good predictions - this serves as a useful caution about the dangers of naive assumptions - it also tells us that conventional error measures are not what we should be looking at, in this case. The good news is that the 'pathological' situations can be easily identified early on by checking the analyzer preview, so we don't need to waste precious clock cycles on garbage predictions.
  
• Spiky filters, i.e. the 'low number' wavelets seem to produce similar results to using raw data. These need a lot more investigation. All we can say is that they do not seem to produce the strange behaviour of the smooth wavelets.

Try using the filters only once you have experience, and have some time on your hands. Share your experiences with us.

Your Possible Experiences

These will fall into one of the following categories -

• They work, fairly reliable, all of the time and across all types of stock.
• Some do, some don't, arbitrarily
  
• Works, but depends on the stock
• Works, but depends on parameter choices
• Works, but not much better than a simple random walk
  
• Works, but not exploitable
• Just doesn't work!?

We want to hear your experiences so we can make the product better - so get in touch!

Lets relate our possible problems to the what we want to solve. Consider the variation in our target stock -

Types of Stocks

Share prices aren't smooth functions, they aren't even true random walks, nor can they be well approximated by superpositions of these; they seem to have a qualitatively different character all to themselves, furthermore this character also varies across different stocks, the nature of which may change over time. The advanced analyzers are an attempt to exploit some vestige of hidden, unseen, 'predictive information' within the share price - one way that this idea could go wrong is if there is 'nothing there' to exploit in first place; we must discern our victories, we cannot manufacture them.

Thinly traded, illiquid stocks are something to be avoided, as are news-dominated stocks; and even when a stock does not seem to be 'news-dominated' there is always the prospect of our old nemesis, the trend-breaker news event. Another thing to watch out for - which some may think of as a 'controversial theory' - is the existence of large amounts of 'funny money' in the markets from time to time; government institutions stuffing the futures to prop up the markets, hedge funds playing games, big players making large trades to trade the feedback.

What Information is Useful to a Trader?

Or in other words, what do we need to know to make money? We know perfect crisp predictions are impossible, but we don't need a crisp trajectory to make money - all we want to know is UP or DOWN or STAYS-THE-SAME at the time we close the trade; its like a fair game of find-the-lady. 'All' we need to do then is to gain a few percentage points advantage over dumb-luck (to beat the commission and the spread) to tip the odds in our favour, then we start making money instead of losing it. (The compounding of a small advantage leads, after many repeats, to a large profit - all bookmakers, market makers, card-counters and casinos know this very well.)

Depending on the type of analysis you do, the analyzers may show you, from the simple random walk -

• The general, long term trend; this is the white centre line.
• The likely size of the variation about this trend; from the isocontours labelled 50,66 and so on.

In studies we have done, across a variety of stocks, the common general behaviour seems to be an oscillation about the general trend with the sizes of each oscillation set by the 50% isocontour; within its obvious limitations the random walk analyzer is pretty accurate - what it cannot see is any 'substructure' within the probability distribution (- the 'heatmap'), which is where the advanced techniques come in.

From the advanced analyzers we may also see -

• highs and lows
• timing of turning points
• tighter probability contours

Which obviously gives you a lot more!

These types of information obviously influence the kinds of trades you can make, and the kind of trader you can be.

What kind of Traders are there?

There are a number of ways you can play the market -

• Daytrader. Professional private traders making hundreds of trades per day try to profit on very small movements. We don't really approve of these guys - its too manic, too frequent, too leveraged; an ordinary person would probably die of stress within a month.
• Spreadbetter. Much like the daytrader but not so manic; he plays on a timescales of hours up to several days.
• Swing trader - he is looking for the next change in direction; his strategy is based on the idea that prices just keep bouncing around. He is doing multiday trades with specific entry/exit points.
  
• Options trader - trades options; options are contracts with specific expiry times - what therefore interests you are the price probabilities at this expiry time. From these you can calculate the odds of making a profit or a loss. The option trader uses his pricing model to sell 'expensive' options and buy 'cheap' options.
  
• Leveraged, active trader - he believes he has specific knowledge about direction and timing; with such confidence he can make much heavier bets.
  

As you can see, what you get from your analysis determines what you can sensibly attempt in your trading.

If your stuff actually works, doesn't it invalidate all previous theories of market behaviour?!

If you are talking about the Efficient Market Hypothesis (EMH), then the situation is a bit like the idea of 'chastity before marriage'; everyone says they believe in it, but few actually pay any attention in practice; if EMH is right then we should all just buy tracker funds and go home - but only 'mug punters' do this in any numbers. In short, no one really believes in it; it is of interest only to academics and market-zealots.

Conclusion

We don't intend to put people off our product - far from it! - we just wish to be totally scrupulous about our claims. All we wish to do is to impress  upon you that if you decide to pursue your experimentation in certain directions, you are effectively starting a research project; while this may be of interest to you, we believe that the practical considerations of trading should dominate your priorities.

To recap:

• No one really knows why share prices behave like they do
• There is no equation we can even try to solve; if there was, its likely its parameters would be unmeasurable; and it would be too hard anyway.
• All we have is data; we must learn from the data itself - the advanced analyzers use techniques which can do this; a signal processing pre-filtering stage might be a vital step, though as yet unproved.
• Exact predictions are impossible; probabilities are the best we can do, and we need Monte-Carlo simulation to calculate probabilities
• You need to play the odds when in your favour; find the trades that maximise our winning chances but protect yourself from wipeout situations; this is risk management.

This is the equation you should be thinking of -

Future

The future looks good for our approach - current trends in technology will help immensely.

• Multiple processors on the desktop. Manufacturers are already offering dual processor and dual core on the desktop - this will likely continue; its possible that we will see 4, 8 or even 16 processors on 'ordinary' PCs before very long. (It has been found in the chip design business that simply ramping up clockspeed and adding more cache memory has diminishing returns beyond a certain point - it also produces far too much heat.)
• Our algorithms scale up! That is to say, if you have a 4 processor system, most of our analyzers will run (almost) 4 times faster, and so on. With more processors not only will it be possible to do single studies faster, but the user will also be able to do multiple studies for a single stock

Our own research continues, of course; our next step will be to add a database with company fundamentals, then to add a news event model, i.e.

Copyright © StockWave Software Ltd. All Rights Reserved.