Frequently Asked Questions

The goal of this program is to allow a person of reasonable intelligence, and of suitable diligence, who can afford to spend four hours per week of his time to generate absolute returns over a long period which are in excess of what could be achieved by handing his money over to a fund manager, and to be able make money whatever the general direction of the market.

Why invest in the stock market?

Because the rich do. Or at least they did, or have done. Note that in bad times, the rich can always move their money out — fast — when they need to.

How do I win the lottery?

Buy a ticket and then buy another one. Perhaps after a million or so purchases you will have won the jackpot, or perhaps got wise.

The point being made is that playing the markets should not be seen as a lottery — a game of skill and strategy perhaps, where the true winners are identified over the long term, but trading and investing should not be seen as a get-rich-quick scheme. These never work. Even pure gambling does not work this way; when punting on horseracing the professional gambler is usually testing his skill against the bookmaker — the skill here being not predicting which horse wins, but identifying favourable odds.

The answer to the original question is that you cannot win the lottery; it is a pure waste of money. The lottery is a tax on the stupid.

Will I lose all my money ?

Almost certainly not, if you follow our advice. Almost certainly yes, if you ignore every piece of advice we give you.

Paper trade to begin with, and teach yourself discipline. When trading for real, the use of stop-losses is to be recommended, certainly in the early times, and frankly, at any time at all — one of the major ways in which brokerage firms generate large profits is through people being emotionally unwilling to close a poorly performing position, they thus let losses pile up in the forlorn hope of the situation turning around. It seldom does, and then it takes a strong individual to 'take it on the chin' as it were; stop-losses take away the need for excessive moral character.

Having said all this, pointed out all the pitfalls, described the situation, offered good advice, there are still individuals who will be tempted to rush out and do stupid things; for these sad souls we offer the following answer:

How do I get run over by a truck?

Blindfold yourself, walk to your nearest motorway / freeway. Cross it.

On the other hand, please don't do this. Use the Green Cross Code, and please do not sue us; don't you Americans get 'irony?' (Hint: it is a bit like coppery. Metallic.)

What is risk management?

Understanding the nature and magnitude of risk; finding strategies which allow good results more often than not, while guarding against one-off exceptional, but highly destructive events. Risk management is naturally an area of some importance to the business community, and covers all of its aspects. In terms of investing, the phrase is often used in terms of portfolio theory — by distributing your investments across a number of different securities you can effectively insure yourself against negative events.

In theory, one should be able to hedge, i.e., limit risk exposure, to anything at all, but there are big problems here. One of the most worrying situations to be in is where one believes oneself to be insulated against disaster, but not actually being so. In a nutshell, the big problems are with very large, infrequent and unusual, but catastrophic events — exactly the kind of thing you need risk management for. These large events are naturally rare, and so one cannot gather any decent statistical samples on them, without which one is reduced to little better than guesswork. For the situation of the investor, the nightmare scenario would be of general market collapse, during which all stocks decline sharply, in which case having a diversified portfolio will not help you. The underlying reason for such a crash might be widespread consumer fear — ultimately all of the companies quoted on our markets must sell something to someone who wants to buy it. If everyone decides to stop spending then the entire economy will be in trouble — which affects every kind of business.

Why should I take the risk? Isn't this all very risky?

Do not indulge in anything which has a level of risk you feel uncomfortable with; just walk away. Do not trade on the markets, do not use this program. If you are not sure about something, then refuse to do it.

This advice applies in all situations. But consider this, whatever form you keep your assets in has some level of risk attached to it — you cannot get away from this, so the quicker you can accept this and start to manage this risk in a rational manner, all the better. Cash is not safe; nothing is completely safe.

Why don't I just put it in a bank?

Why not, indeed! Banks do not seem to go bust in the west; they are pretty safe here, but in other countries banks are not so solid — in these places you might prefer to keep your money in a hard currency buried in your back garden. Note also that the interest gained on an ordinary savings account would have easily beat the returns of the stock market in recent years.

Why don't I just buy a fund?

The 'genius' of fund managers to pick stocks is severely oversold; in recent times this would have been a very bad idea.

Won't data subscription and brokerage charges kill any profits I make?

Our general advice, the smart thing to do in our eyes, is to go for a highly probable, small profit on a regular basis. Brokerage charges and other costs can easily wipe this out, so you need to shop around. Note that these charges are explicitly included in the trading calculations done in StockWave™, and also that StockWave™ is designed to make use of free data sources wherever possible.

The algorithms used in Stockwave™ — how good are they?

You won't get the range we offer in any other application currently available. Some of our methods are novel.

We believe that in complex, real-world situations, there is no single solution which is guaranteed to work all of the time — it's 'horses for courses' — therefore we take the view that a hybrid approach is best, e.g., human beings have six senses — but why not only one or two? The reason is simply that we need six! As simple as that.

What causes stock prices to behave as they do?

Prices are generated on the markets by an auction process — a price is set, and numbers of buyers and sellers are found; when there are more buyers than sellers, the price will rise until the numbers of buyers and sellers are even, and when there are more sellers than buyers the price will fall. This is the market system — that is all there is to it. But this is only the 'how' of the process, as for the 'why' — no-one really knows, and theories are legion. All we know is that people want to buy because they believe there will be rises in the future, and vice versa — why traders believe what they believe comes down to many factors — reactions to news, reaction to past events, intuition, emotion — it is all there somewhere.

How random are stock prices?

Very, but not, we believe, totally — there may be some residue of predictive information in the timeseries, which can be exploited by the right kind of analysis algorithm. If stock prices are completely random then there is no way they can be predicted — in this case a random walk model is the best you can do.

Derivatives are so complicated!

They can be. So only trade with them once you understand them. Stick to vanilla options; absolutely avoid anything you are unsure of. Let's be very clear about this — 'not sure' means 'don't trade.' Use the trade creator to experiment with the trades and their combinations — in your early days with StockWave™ you should think of it as being an educational tool only.

Aren't derivatives very risky?

Yes, if you do not understand what you are doing, and most certainly, yes, yes if you are yourself a bit of a gambler — a natural risk taker.

Derivatives — being very flexible — allow the user the means to both hedge and to leverage a position; hedging is insuring against risk, while leverage means effectively taking on a multiplied risk in the hope of achieving multiplied profits.

Haven't derivatives been responsible for the most recent financial disasters?

Yes, in the sense that derivatives can create a dangerously magnified outcome — but in the final analysis, most disasters are really caused by good old-fashioned fraud and false accounting — the element unique brought by derivatives trading is that of leverage.

What's your advice?

StockWave™ and its creators do not give specific investment advice — we recommend no securities, products or experts; we provide an algorithmic toolkit for data analysis, that is all. We do have some general rules-of-thumb regarding trading though:

Never risk a loss you cannot afford; if you want to play the markets, please don't use the kids' college fund as your 'float.' An investment portfolio should be balanced; some gold, some cash, some foreign currency, property, bonds, shares and perhaps some derivatives — the exact percentage allocations to each category should reflect your own appetite for risk; if you are already wealthy, then you need to play a defensive game, and if you are poor, then the whole discussion is irrelevant. For the rest of us, it is a matter of personal taste.

Don't get emotional. A trading floor is probably the worst place to be; herd behaviour takes over. Reason is a major casualty. More specifically...

Don't panic, and...

Don't get greedy.

Use stop-losses, and also...

'Stop-profits' — close out positions in a timely manner.

'Trading on margin' is usually a bad idea. And by 'usually' we mean, say, 9,999 times out of 10,000.

Accept that sometimes you will lose money.

Accept that predicting very short-term phenomena is impossible.

Accept that sometimes even the best analysis will be wrong.

Only trust corroborated quantitative analysis.

Learn how to hedge, i.e., using combinations of trades put a floor on possible losses.

Check your alarms and any open positions regularly.

Once a week spend 3-4 hours doing more in-depth research.

If it's 'not for you' — then that is OK; if you find the returns don't justify the risk or stress level, quit.

I am getting weird results — what's happening?!

Maybe you are doing weird things. Maybe it's just the algorithms. Maybe it's a bug. Read the manual. Read these ./faq. Read the release notes. Then, tell us all about it — but be very precise. If it is very worrying to you, suspend trading.

One particular thing to watch out for is having crappy data — if there is not enough of it, or the sampling is extremely irregular, or you are trying to use a high-resolution sampler on low-resolution data — you are likely to get rubbish. The data analysis algorithms work best on high resolution, regularly sampled data of which there is enough to generate a good training set. The internal data pre-processing algorithms can account for a certain of irregularity, but there are limits. The old principle of garbage in, garbage out is applicable here.

The algorithms are too slow!

We have strived to find the most efficient implementations possible for our algorithms, but still — they can be very costly. There is nothing much that can be done about this — see the entry on computational complexity. One thing you must become aware of are 'sensible' parameter ranges for the data analysis algorithms.

Start with low values of parameters, then increase these until the results become acceptable — do not try to start your analysis with everything fully ramped-up. The basic situation is that Monte Carlo simulations and neural network training can take a very long time; in doing your analysis, start out with the faster methods, or low parameter values. Hefty computations can also be done overnight, outwith trading hours; during trading hours we will mostly be concerned with data capture, but even then we can use the lighter methods for real-time refinements.

StockWave™ can use all the processing power you can throw at it — get the fastest processor machine, with the most memory you can afford, and try to get broadband Internet connection; the more powerful your system, the greater is your trading potential.

What is (a)...

Computational complexity — a branch of computer science which studies how well computers can be used to solve different kinds of problem. The basic, and rather unsettling, result is that many, or indeed most common types of problem one might be interested in, cannot in fact be solved 'efficiently' by computers. An understanding of the complexity is often crucial in solving difficult computational problems; naïvely one might think it is possible to just solve a problem any old way you can, then find a computer big enough to solve it quickly — but this is the wrong way to go about things; there are many problem solutions which when implemented using inefficient algorithms will exhaust even the largest computer.

Neural networks are computational systems which are good at pattern recognition, even when using imprecise or noisy data. Applications might be facial recognition, speech analysis, or almost any signal processing task. Neural networks come in many varieties and with many different architectures and training algorithms, the main problem for the end user is in choosing something appropriate; a good, that is to say very hard research problem might be to find the best architecture for any given problem (one approach has been to use genetic algorithms for this structural optimisation).

A web agent is a program which traverses the Internet, looking for certain kinds of target information. Search engines are the largest users of them — their robots wander the Web to generate a catalogue of sites. If you have your own robot then you can tailor it to your own precise needs; this allows the so-called 'deep searching' of the Internet. Very useful.

The Fourier transform is a signal processing algorithm; useful for filtering data and finding periodicities (repeating behaviour). Probably the most important algorithm ever discovered (the FFT is described in the notebooks of the famous mathematician CF Gauss); for example, it makes real-time image processing possible; used everywhere.

A wavelet transform is similar to a Fourier transform, but can have superior qualities. Used for compression, de-noising and many other things. There are actually several types of wavelet transform, and many types of wavelets. In practice, not all wavelets are good at all tasks, but — and this seems to be their main selling point — it is relatively easy to find some wavelet and transform which will do a good job on whatever data you happen to be interested in. We don't know why...

A fractal is a limitlessly self-similar geometric shape; commonly used to model natural forms, shapes and textures, the key feature is that of repetition at ever smaller scales — this seems to be a quality of stock price data as we zoom in and out in resolution (try this with the stock chart). Some commentators have proposed fractals as an analysis technique for stock market data, but have yet to produce any convincing models. The fractal idea is one which you will come across frequently on Web sites devoted to stock market analysis — the reason for this is that most examples of fractals (which are 'of infinite complexity') are in fact generated by very simple formulae (e.g., the Mandelbrot Set comes from the equation x^2 = x), the idea then occurs to the observer that maybe, just maybe, the apparently noisy, random lurching of share prices is, in fact, generated by a really simple formula. All we have to do then is to find this 'really simple formula,' and we have perfect knowledge of the future for our share price — with this information we can then proceed to billionaire status in no time at all.

Natural language parsing — an AI discipline concerned with teaching computers to understand speech and language. The ultimate goal here would be computers we could simply talk to. Has been worked on since the 1950s but success has been much harder to come by than anticipated. It should be pointed out that while the problem of human-like understanding is incredibly difficult — imagine trying to get a computer to understand a joke — many of the techniques developed within the discipline can be applied very successfully to simpler tasks, e.g., filtering and extraction of interesting and relevant news articles.

A search engine — looks for information on the Web; typical usage — type in some keywords, then get back a list of relevant pages. Search engines vary in the quality of information they return to the user, and even the best of them have only probably indexed a small portion of the fully available Internet.

An expert system is a reasoning system based upon rules and logic.

Fuzzy logic — an alternative logic to the usual first order predicate logic; useful as it allows the use of imprecise, linguistic notions. First order predicate logic is in practice, a bit of straitjacket; it lacks expressivity. For a good comic example of the inappropriateness of logic in real world situations, read 'Mr. Logic' in Viz comic.

A genetic algorithm — a general purpose search technique. If you are really stuck, i.e., you have no idea what the search space is like other than it is very large, and no handle on the nature of the fitness landscape — try a genetic algorithm. (Conversely, if you have a lot of prior information, chances are something else will be a lot faster.) Genetic algorithms have had a lot of success with scheduling problems, traditionally a difficult area. The potential application to stock market trading would be this — suppose I wanted to develop a strategy which told me how to trade stocks, which would be a function of a number of observables about a company, including its share price, earnings figures, and perhaps many other numbers — where would I begin? The complication of the possibilities here is unlimited, but if I want to narrow my options, all I would likely succeed in doing is to reproduce a lot of the 'folk wisdom' of trading — which is what I want to get away from. With a genetic algorithm you would not have to start out with strategies that were any good, or were expected to be, but with successive evolution, involving back-testing on the historical data, the qualities of good strategies would gradually emerge and become concentrated.

All of these are 'good things;' all have their place; none, purely on its own, is a magic bullet. It is a common, somewhat unfortunate reality that all new techniques tend to generate a lot of ridiculous hype when they first appear, and then when they fail to 'deliver' are forgotten about. Methods fail because they are used in the wrong situations.

All of the above techniques have been used in StockWave™, at various places and in some measure, wherever considered appropriate.

Do you have a motto?

Yes. 'Reality bites.'

Why don't you have 'my favourite indicator?'

It was an express goal to eliminate anything which did not do anything useful; most of the favoured technical indicators fall into this category.

If you want my favourite indicator, buy another application. Sorry, but we don't do X, where X is moving averages, stochastic, RSI, Fibonacci retracements, GANN, parabolic SAR, MACD, Bollinger Bands, candlestick charts...and any number of other things. We give everything you need and nothing you do not. The intent is to be complete but minimal.

I am a fan of Elliott wave...

We are not — this is merely low-grade pseudoscience one step removed from astrology or the I Ching. There are no judgements too negative for this so-called theory. Please, do yourself a favour and forget about it.

But aren't Fibonacci numbers a fundamental law of nature?

Not any more than the Adidas logo is a 'universal component of human culture.' Quantum theory is a fundamental law of nature.

Is the stock market is based on quantum theory?!

Er, no. At least not directly. The 'laws' affecting the stock market are a branch of the discipline known as complexity. Physicists have recently become interested in such phenomena; convincing simulations have been produced which generate accurately the qualitative features of the markets; unfortunately, these are not capable of telling us if, say, Stock XYZ will hit $20 before Thursday. Or if they can — no-one is admitting it.

Why don't you search for the classic chartist patterns, e.g., double well, head and shoulders, crossing over MACD...

This is anecdotal folklore. If you like it, fine, but best buy something else.

There's this guy who swears that...

What? He saw Elvis? Aliens abducted his donkey? If he is barefoot without his shirt, you have an answer, and if he is sharp-suited with an expensive smile and wants to present an 'investment opportunity' — then it is high time you started to run in the opposite direction.

Sarcasm aside, there are loads of programs out there which do this sort-of 'chartist folklore' thing — if this is your cup of tea, then good luck to you; we think you will need exactly that.

Can you predict the market?

No. No-one can. Not Alan Greenspan, George Soros, Gordon Brown, Gordon Gekko, Uri Geller, Madame Blavatsky...no-one.

But what about Warren Buffett?

The legendary investor takes a very conservative line, which has proved to be most successful in the long term.

So why I am buying this?

We offer an algorithm set which produces probabilistic indications as to likely market movements. This lets you play the odds more effectively than otherwise. In the long term you can win out. It will give you an edge, no more. But if you use it well, that is all you will need.

Can the market makers predict the market?

No. They make their money on the spread — i.e., the difference between the buying and selling prices.

Investing versus speculation versus betting — what is the difference?

None. None whatsoever. They are all the same. Investing is respectable, speculation is not, and betting is seen as either disreputable, or in some backward nations, is even illegal.

They are all about making money. Wanting to make money in the short terms is seen as 'bad,' wanting to make it in the long term, for your pension, or for your grandchildren, is good. Cannot see the difference myself. All investing is really about the joy of the unearned dollar — you buy something at a certain price, then you do nothing whatsoever, and at a later time, that something is worth more than you paid for it. Earning without sweating — very sweet indeed!

What is snake oil?

Snake oil salesmen used to traverse the old American west selling gullible individuals 'medicines' which would cure all ailments known to man. These rarely did any good, and often did a great deal of harm. Eventually they were driven out; many went on to become investment gurus.

No such thing as a sure thing, and no-one will make money for you.

You can only make money for yourself, and only do so by playing the odds — when they are in your favour (and how do you calculate this?) We supply the tools to give the ordinary individual the chance to make good returns with DIY investing. We supply the hammer, the nails and the timber, you must make the shed yourself.

Why are the Rich, rich?

Because they invest in the stock market.

OK, so some of them stole it, and some earned it, and some inherited it, in the beginning, but their continuing and increasing wealth is largely to do with the stock markets. Money makes money, and with the right kind of investing strategy you can make a lot more.

What about crashes?

These do happen but so far, at least, the markets have always recovered - although this can take a period of years to occur.

You might like to think that because the markets have always recovered in the past, then they always will recover in the future, but this is a dangerous assumption to make.

Do markets always 'revert to the mean?'

The answer to this is a resounding 'don't know,' and we have to say, 'don't care' either. But we can say this for sure — if the answer is yes, then it must be yes...eventually. But what do you mean by eventually? How long is 'eventually?' Twelve years? Thirty years? Can you wait that long? This is a dangerous assumption to make. It is reckoned the collapse of LTCM was caused at least in part by this assumption.

The idea behind 'mean reversion' is that there is an 'elastic band' between the share price and some underlying trend — when the price rises or drops too sharply then the price is due to get 'snapped back' — but like many intuitively reasonable ideas it fails to give us any actually useful information, like for example, the timing and strengths of these corrections. If you were serious about building a model then you have to describe the parameters of this 'elastic,' like how stiff it is; so now we have to make a model of this stiffness, probably regarding it as a function of a number of other variables...which takes you back to where you started.

What is volatility?

How much the stock price wiggles around.

Is this good or bad?

Bad in the sense it make the price less predictable, but good in the sense it provides more opportunities to trade.

What about pensions?

Would I be better off using share trading as my pension fund?

Not really. Inadvisable. Pension fund managers do not know any more about the markets than you do, BUT, the government gives tax relief on pensions, and it is this that makes them a reasonably good deal, if you have a company pension based on final earnings. Keep a hold of that whatever you do. Alas, these are becoming rarer by the hour.

Without going too deeply into it, pensions have been one of the worst and yet most respectable scams ever perpetrated by the gurus of the financial services industry against the ordinary man. Rotten value in the past; likely to be rotten value in the future, the underlying and pressing question of exactly how we should save for our retirement is not in any way resolved. Chances are that the whole concept of retirement itself could be lost; first the retirement age will be increased, then gotten rid of entirely. In the future, we shall all work till we drop.

Is there a theory of price formation?

The share price represents the worth of the company. What could this depend on?

How much they sell
How much profit they make
How well their rivals do
The price of raw materials
Efficiency of their processes
Quality of workers
The weather (if you make orange juice)
Expected future demand for product
Are the management competent?

These are all tangible, concrete things; fundamentals, if you will. The trouble is that when one goes to look at a graph of practically any stock, you will see that it oscillates rather wildly and can increase and decrease by large amounts over even short periods.

So what the hell is going on?

The value of a thing depends on how it is traded; things in demand become scarce, the price goes up, and vice versa. The trades are carried out by human beings working in large dealing rooms, who are effected by their, gulp, — emotions and beliefs. This is a somewhat shocking fact for most people to grasp — that tangible real world phenomena can be altered, or indeed generated, by belief and emotion.

It would be nice to think that some set of fundamental physical laws, akin to the laws of physics governed market behaviour — once discovered, all would be known, all understood. But it would seem the markets have a different character. News, rumour, supposition and emotions are communicated to and fro, herd behaviour takes over, and from the individual actions of many traders large collective movements can take place. To the naked eye, price movements look pretty random, and they mostly are.

Share prices can do anything at all; no-one really knows why.

Can we predict or not? Is there a theory that can be used?

Data analysis algorithms used in science and engineering may allow probabilistic estimates to be made. Play the odds a bit better than the next guy, and you will win out in the long term.

Was Adam Smith or Karl Marx right about the market?

It is a good idea to leave your political ideology at home when making this sort of comparison; consider these great thinkers (yes, both of them) as being first class students of capitalism and the forces which shape our world, and remember that as far as we are concerned, practicalities are all that matter.

Both thinkers made insights; in a nutshell, for Smith that markets work best if left alone by governments, and trade is free; and for Marx that markets are unstable and will tend to crash by their own nature; however, both of these positions are extreme idealizations (i.e., simplified theoretical models), in practice governments and bankers are constantly attempting to stabilize the markets; trade is reasonably 'free,' and while there have been crashes, the system has never broken down completely.

Market behaviour is very rich and very complex; holding on to theoretical dogma is unlikely to allow you any profit.

Money is more important than politics.

What is Probability?

Probability is the mathematical discipline which deals with chance, i.e., events and processes which have some random component, and which we can no longer describe in terms of definite states.

Probability theory and statistics is often seen as a rather dull subject with everything 'interesting' about it having been worked out long ago, but there are lurking some philosophically difficult problems. What most people, even frequent users of statistics, do not realize is that probability theory as we know it today is based on a particular experimental scientific model — the ramifications of this result in what is known as frequentist statistics. The basic idea is that to define / calculate a probability one has to make lots of identical observations of the phenomena one is trying to describe — which is fine for a scientist in a laboratory.

Frequentist probability kicks in when large numbers of repetitive events are possible, via the law of large numbers. This means that for example, if you watch a toss of coins ten times, and it comes up heads seven times, then it is not inconsistent with the coin being weighted — there is not enough data to make the result statistically significant. If however, we threw the coin ten thousand times, and got 7,000 heads — it is fair to say the coin is almost certainly weighted. It is possible that such a result could happen by chance, but the chances are astronomically low (calculate these). But when you do not have enough data, frequentist statistics gives you nothing.

What is fine in the laboratory, is not so in the real world. If you had been playing roulette and the red came up seven out of ten, naïvely one feels that the 'probability' of the next ball being red should be higher than it being black — it is 'common sense' surely? But this is subjective, and for our trading purposes we would like some hard numbers, not some mere intuition.

This is where Bayesian statistics comes in — this allows estimates of state to be made based upon new information, so in the roulette case, the 'probability' of red would be increasing after each throw. There is an equation for calculating this, so mathematically it is sound. The objection to Bayesianism lies in the definition of a priori probabilities — i.e., the 'probabilities' which we assign to the process, without having any evidence to begin with, which just seems like a purely subjective guess. In the case of the roulette wheel, there is no real problem — punters can bet either red or black, and there is no advantage to the management doctoring the table towards either, so assuming a fair 50:50 split for red and black would be a reasonable. In other cases, particularly where a lot is at stake, it is not so clear — for example Bayesian statistics has made an appearance in the law courts, to somewhat mixed and confusing results.

Going back to the roulette wheel example, had the wheel thrown up another 9 out of 10 reds to make 16 of the last 20, you could be damn sure the management would have stopped play on the table! And quickly. This is really the heart of the matter — the difference in these viewpoints of probability lies in the intended usage; with Bayesian ideas we are usually trying to answer the question, 'What do I do, right now?' having been given an incomplete set of data; there is some kind of time pressure on us to make a decision, whereas with classical probability we are repeating a tightly controlled experiment over and over again until we can make a valid inference. Bayesian statistics is about the 'real world,' i.e., an uncontrolled, rapidly changing environment.

In general, be very careful of mathematical models which use probability theory to study the stock markets — these are often carefully constructed, using quite crude assumptions, so as to provide solvable models, to which the mathematician can provide a neat, elegant solution, and generally make himself look like a genius. Complicated equations can look very impressive to the layman, especially if there are a great many terms full of Greek letters and symbols — but do not be taken in, very often they express rather less about the problem than you might think. Most people think that once you have an equation it can automatically be solved, but this is very much the exceptional case — most of the equations which we do believe accurately model the realities of the world are almost impossible to solve, even with the most powerful computers. The mathematician's principal concern here is to get a paper written and published in a half-respectable journal; repeat this often enough, and he gets tenure. That's it. To be publishable, a paper has to say something definite — i.e., it is to one's advantage to fully explore and solve a gross simplification, rather than merely scratch the surface of the real problem. Be particularly skeptical from anything which originates from a 'Nobel Prize-winning economist' — remember that the poster-boys of the economics / financial mathematics community, Black and Scholes, almost destroyed capitalism itself when their hedge fund, Long Term Capital Management imploded.

Bookmakers would seem to be users of probability, but they aren't. They can't calculate probabilities any better than the man in the street — what they do it is quote odds in competition with other bookmakers, in order to attract customers. Bookmakers will lay odds at whatever level they feel confident of making a profit. You can of course take the bookies odds and turn them into probabilities, for example evens is 50:50 chance, i.e., a probability of 0.5. One of the axioms of probabilities is that the sum of the probabilities for all possible outcomes should be one — i.e., one of the list of all possibilities, must happen. When you take the bookies odds, turn them into probabilities, you will find they do not quite add up to one! (You should be able to work out why this is by now, and how it is related to the market makers in the financial markets.)

What's a butterfly spread?

A new type of low-fat margarine. You don't want to know. And, more importantly, you don't need to know either.

Actually, it is a kind of options trading strategy.

What's a privileged briefing?

A kick in the pants for the small investor. Hopefully a thing of the past.

In the bad old days, the management of top companies would entertain favoured analysts over lunch and golf, often to divulge advantageous information about the company. These analysts would then pass on this information to their best, i.e., biggest clients.

What's a conflict of interest?

When a party has responsibilities which are irreconcilable, e.g., being broker to a company, and yet having to give impartial advice to investors.

What is insider dealing?

The only genuine way to make risk-free profit. The only sure thing there is. And thoroughly illegal. As it should be. A criminal activity, thoroughly chastised by fierce legislation both in the UK and the USA. The laws are so strict in the UK for example, that someone was once nearly prosecuted for it.

Certain events will incontrovertibly affect the share price in a predictable way — this information is known as market-sensitive. Anyone who has this knowledge has an enormous advantage over those that don't. Markets police the release of this kind of information in order to give everyone a 'fair chance' — since if the game was rigged, no-one would play it, no-one would trade shares, and the stock market, merchant banks, brokers, fund managers, etc...would all go out of business.

If ever accused of insider dealing, you can probably get off with it explaining to the constable how lucky you are. Not at all like Charlie Sheen in Wall Street.

How can the little guy make money here? It's all a big stitch-up in favour of the well-connected city boys!

Perhaps. It certainly was in the past, and while experts tell us how things are a 'lot better than they were' — it is easy to feel that this is not the case.

StockWave™ is an attempt to level the playing field for the small investor.

What do you NOT do?

We don't give stock tips.

We don't do analysis for you.

We won't interpret your analysis for you.

We certainly don't recommend any particular brokers — except to say that we favour the cheapest.

We certainly don't recommend any particular analysts — the whole point of StockWave™ is to DO YOUR OWN ANALYSIS.

We don't tell you whether something you are doing is right, wrong, good or bad.

We don't give refunds if you lose money and we don't ask for a cut of your profits either (not like some hedge funds).

We don't say what the 'best method' is. Everything in StockWave™ is useful in some way; if it wasn't — we wouldn't have put it in in the first place.

Now this all sounds rather unhelpful — why ever won't we extend our customers a helping hand?

Simple, if we started doing this, then we would simply become another bunch of pundits / gurus / tipsters — and by now you should realize that this is against our entire philosophy. You want to make money?! Then you have to do it yourself. Do your own analysis. Make your own trades.

Can I make money on a falling market?

Most certainly. For a falling market the classic technique is known as short-selling, or simply shorting — it is a bet that the share price will fall. The thing people find hard to grasp about shorting is that typically one 'sells' a stock you don't own! How can you do that?

In the markets, no-one really cares about actual share certificates — accounts are settled in cash at the end of the day, so as long as you can pay, there is no problem. (It does freak people out a little.)

Here's how it works; you know, or feel very strongly that a stock valued at £20 is going down, and will hit £15. You start 'selling' it at, say £18.50, and indeed it does fall to the level you anticipated. You buy back all the stock you 'sold' for £18.50, when the price hits £15.50 — now you don't owe anyone any shares, plus you made a profit of £3 per share on the deal. Wow!

Of course, it can go against you. Suppose it dropped to £19, then rallied to £22.50. All those people you sold to want their shares, and you don't have them! No worry, just as long as you can pay for the shares you sold; the thing is, now you have to buy a load of shares at £22.50, and so you have lost £4 per share on that deal. Ouch!

On a generally rising (bull) market — one simply buys stock and holds it. It increases in price, you have made money. On a generally falling (bear) market — taking a short position can make you money. And if the market is simply oscillating, one can attempt to buy-the-dips, thus making a profit. Much trickier — analogous to sailing away from or into the wind.

Of course, the easiest way to profit from downward moves is not to buy or rather attempt to short sell stock, but to use for example, options, CFDs or spread bets; these are types of derivatives.

How is StockWave™ different from other programs?

It's better.

And it is cheaper.

What will be the coming hot stocks?

We do not answer questions like that, mostly because we do not know. Even if we did, we probably would not tell you. We have no wish to be considered as gurus.

What will be the coming hot technology?

Please stop asking for tips...

OK, a personal preference; big areas in the next 20 years:

Wearable computers
Immersive gaming
Molecular computing
Genetic medicine
Nanotechnology

Which is really just a statement of the obvious to any technologically literate person. Note that there are many companies with interests in these areas; I have no idea which of them will be winners.

Taking a more nihilistic view of human nature, I would say good long term investments will be the same as they always have been — booze, drugs (of whatever legal status), fags (the kind you smoke, American friends), guns, porn, food (the faster the better). Note that this investing advice dovetails smoothly with the classic deadly sins, which says rather a lot about the human race. Or at least my opinion of it. Seriously though, we cannot see any of these major sectors failing to survive in the foreseeable future.

Technology, when it comes down to it, is simply novelty — which is a very fragile thing.

What is the most important: the chart, news or fundamentals?

The chart shows the price of the stock as it was and as it is now — this is the basic reality we are dealing with, so it is the most important. News is important too, that is to say genuinely unexpected, sizeable events — these will create large distortions in the share price, the effects of which will take some time to realize; the time lag, and the size of movement caused means that serious negative positions can be achieved, so, no...we cannot afford to turn off the news wire. Fundamentals, plus other kinds of information are also important — we know they have an effect, but this is less clear than in the other cases.

All are important and complementary.

What methods do people currently use to trade?

Current trading methodologies broadly separate into three, unusually distinct and mutually exclusive camps. There are the chartists who believe that all truth lies in the chart of the share price; news hounds who listen constantly for the latest event or tip, trying to get ahead of the market, and fundamentalists who believe that the 'real' truth about a company lies in its report, i.e., its sales, turnover, cash flow and who use indicators like the P/E ratio. The intriguing thing to us, as outsiders, is the extent to which these groups separate themselves from each other.

Caricatures of the trading styles?

Chartists are cranky, mad-hatter-ish, pseudoscientists, addicted to a world of strange jargon and arcane 'trading systems.' They will talk of Gann angles, Fibonacci retracements, Elliott waves, the Kondratieff cycle, head and shoulders, double top, triple top, false bottom, support, resistance, momentum, overbought and oversold...please don't listen to any of it.
News hounds are permanently paranoid and overreact constantly on the slightest whiff of any news; never calm, in groups they exhibit the typical herd behaviour of sheep and lemmings. Most common habitat is the trading floor itself, barking into telephones, eyes glued to their clusters of monitors. Equipped with real-time, state-of-the-art data-charting and worldwide news feeds, they carry with them the fatal flaw that they cannot reliably assess the importance of the information that they are receiving; under constant pressure to react instantly, to be seen to be doing something, they do so excessively.
Fundamentalists are the smuggest of the lot, believing that their 'insights' from relentless porings over company reports and analysts briefings, somehow represent the 'true' value of a share price; hardly ever right about anything, they have taken a great blow recently as it has been revealed that their sacred numbers have largely been accountancy fantasies all along; we believe that the worth of company is exactly what its share price says it is, at that moment in time, and nothing more or less; what it is 'really' worth is a fantasy, as is 'should be worth' and 'could be worth.' We also believe that the numbers to be found in company reports give only a very partial picture of a company's financial health, even when they are not simple lies.

Let's think different; why not use all the information available to you, then combine it into a single piece of analysis? Become a fusionist!

In some of the documents you talk of Underlying Process — what do you mean?

Some unknown, and possibly unknowable, algorithmic procedure which generates the price movements.

In physics and engineering, there have arisen some extremely powerful techniques for processing data; take as prime example, the Fourier transform. Why does this work so well? The reason is because almost all physical systems consist of subsystems which oscillate in some manner (the underlying process); this applies equally well to the theory of sound waves, electromagnetic waves in space, and even atoms in a crystal. Whenever you have such a situation the FFT is a natural choice to use as a filter.

The analogous sub-entity with regard to the stock market is the individual trader/investor trading on the market; alas he will tend to be considerably more complicated than a simple oscillator, his actions being largely driven by pure emotion — cycles of greed and fear, plus reactions to random external events.

What are, in your opinion, the most common foolish attitudes?

The chart does not matter — we beg to differ
It is all in the chart — mostly, yes, but by no means all
Fundamentals are all that matters — just keep reading that report!
News does not matter — until something big happens!
All the analysts say — what do you care? If they said anything different they'd probably get the sack.
I got a great tip the other day — who told you and why did they tell you?
Stick to your system at all costs — so you forgo the possibility of learning!
The professional fund manager know best — unless you check his record!

If you rely exclusively on one type of data source upon which to base your analysis, eventually you will take a very bad loss.

General data analysis — a summary

How to use news

Get the news; develop an awareness of it. See where the events occurred in relation to the graph. Look for correspondences.
Filter the news for relevance.
Decide whether it is generally good or bad.
Make precise quantitative assessment of size, quality and favourability.
Aggregate news events, together with other information via an expert system.

How to Use the chart

Look at the chart; look for news events around large moves. Apply the StockWave™ analyzers to generate probabilistic estimates.

How to Use Fundamentals

Inspect aspects of interest.
Use the query tool to generate tentative stock picks. You supply the criteria of what you consider to be good or bad qualities and the database finds these kinds of stock for you.
Use the mining tools to look for signals of good and bad company performance. Use these signals to filter out stock picks. This is analysis of a different level from the simple query — instead of providing the question to be answered, the database analyses itself for internal patterns, then returns these stocks to you.
Do all from with a visual interface; see clusters of companies; see relationships; see relative performance.

Putting it all together

Data fusion is the key technology we must use:

Fusion of predictors
Fusion of price series analyzers and news events
Fusion of news and fundamentals
Fusion of everything

'Everything matters to some degree, at some time' is an unattractive reality to face. We want life to be simple; simple rules, clarity, folk-wisdom, home truths...the endearing, heart-warming emotional crutches that we cling to... wake up folks. The world is complicated, confusing, full of doubt, irrationality and randomness. Look at how the markets lurch around; look at an individual share price, see the chart of its graph; is that anything like the smooth functions and simple curves you drew in algebra class?

Now ask yourself this question: would you like to make real money to spend in the real word on real things, or would you prefer to make idealized money in an idealized model of the financial markets? Anything can happen, and it will — if we wait long enough. 'That shouldn't have happened' has no meaning; there is only what is, what was and a set of possible futures, each with its probability of occurrence.

Why is news important — can't I just use the chart?

Truly unexpected things can catch you out. By your persistent questioning I assume that you have heard of, and are thinking about the Efficient Market Hypothesis. Well, this hypothesis is exactly that — a hypothesis, an idea, a guess, a working assumption made by academics so that they can solve their equations more easily.

Empirical evidence suggests that the market is 'quite' efficient, but not totally. We do not believe that 'quite' is quite good enough to start basing trading decisions on. Consider this, if you lose money on a trade, do you feel happy that you 'nearly' made a profit?

No, we didn't think so.

How does the market react to news?

Good question. Sometimes greatly, other times not at all.

What we can say is that there are types of news which seem to cause an effect and others which do not; of course the sensitivity of the market can change quite dramatically; in good times, good news is magnified, bad new dismissed, and vice versa.

We can further divide our relevant news into fast news and slow news; fast news is an event the impact of which is obvious, and which the market absorbs readily; slow news is something which may be important, but whose significance is not clear at present — we cannot simply ignore these types of events as they are often the precursor of large moves; the interesting thing for the investor is this — if a set of slow news is the precursor of a large market move, there is the potential to predict the move since we already have all the information that we need; what we have to do is to 'put all the pieces together' in our minds, before the rest of the market does. This is not easy, very often we cannot see the 'forest for the trees' and like the rest of the market we do not see things till they are too late; the significant pieces of our jigsaw will simply appear to be 'irrelevant' news items at the times they occur. (There is also a third kind of news — that which has already been anticipated by the market and has already been factored in to the share valuations; this results in the 'big announcement — nothing happens' scenario.)

Let's take an example; a drugs company has a new drug in development, the approval process for which takes a long time. Over, say, a two year period we get the following kinds of news events; the drug progressing in its approval process, directors dealings, changes of top management, the expiration of patents and so on. Individually these mean very little at the time they occur, but suppose when assembling a timeline of events we see a steady selling-off of the company stock by the management, and a drop in revenues as the patents expire, all occurring as the new drug reaches the later stages of the approval process.

The likely valid inference from this is that the stock will shoot downwards at some point; sales and hence profit are set to become eroded by the likely presence of cheaper generics as the current patents expire, and the management, knowing this, are gradually eroding their own stock positions; this further makes it look like the drugs passage through the approval process has not been a smooth one and that in the end it will not get onto the shelves, and hence will not lead to any new revenue. Overall, we see that at some relatively near future date, the company will probably have to announce a radically reduced set of earnings forecasts, which will lead to a sharp selling-off of the shares and a subsequent thrashing of the share price. As an aside, drugs companies often have large step changes in their share prices due to such events.

Similar analyses could be done across a sector, looking for companies susceptible to takeover bids (these send share prices shooting upwards), for example.

In principle, an expert system could be able to draw such an inference.

How should I think about the news?

The trouble with 'news'/'analysis' is that there is so much of it, and much of it utter dross — recycled opinion, speculation, hot air, fluff, hype, misinformation — but amongst all of this may be hidden some nuggets which are genuinely interesting.

Textual information we broadly characterize as fact, opinion or rumour.

A fact is something which has actually happened, and will be reported on the major media channels; opinion accounts for about 99% of what is written in financial magazines and Web sites — the good news is that this is worthless and can be completely eliminated with little worry (as you should recognize by now, we are totally against the army of gurus, pundits, tipsters, analysts and so-called experts which seek to influence the general public; do not trust these people — do your own analysis, then keep it to yourself); rumour is the kind of thing which gets posted on bulletin boards or newsgroups — something which must be taken with a pinch of salt — for example, some guy could be trying to ramp a load of duff stock he has bought, or on the other hand be a disgruntled insider who is letting you know what a company is really like; you really cannot tell.

With StockWave™ we stick to facts; eliminate opinion and remain aware of rumour.

Once you get a piece of news, the questions you must ask are:

Is this favourable or unfavourable?
Can I believe the origin of the story?
How important is it, is it a large event or a small one?

When looked at this way, the vast amount of news and textual information can be hacked down to what is genuinely important, and furthermore, we can reduce a stream of news events to quantitative measures of importance and favourability, i.e., a simple numbers. Then the fun starts!

What is the Capsizing Boat?

A small dinghy is full of people; something happens and everyone shifts to one side to get a better look at it — because of this, the boat capsizes and everyone drowns. The important point to realize is that there was no real need for this to happen in the first place, e.g., if the boat was hit by a large wave.

You are being obtuse — what do you mean? What has thing to do with the stock market?

When everyone moves in the same direction, that is when disaster will occur. The market only operates because of liquidity, that is the ability to both buy and sell a thing; thus the system as a whole is stabilized by the fact that there is in effect a difference of opinion.

In an nutshell, herd behaviour leads to disaster eventually; what this means for the individual is that they must learn to think for themselves, to go their own way — sometimes you need to go with the flow, sometimes you need to go against it.

Explain Option

When you buy a call you are betting the market will go up (by a certain amount); when you buy a put, you are betting it will go down. If you sell ('write') rather than buy you are betting the opposite of the buy; e.g. if I sell a call, I am betting the market will not go up (by a certain level).

The benefits of options are leverage, and the ways in which you can combine them to your trading tastes. StockWave™ can automatically calculate the expected outcome for even the most complicated trades.

If you are not absolutely sure about how to use them, then don't.

Explain Spread Bet

When you buy a spread bet at say £10 per point, you are betting the market will go up (an up-bet); and when you sell it, you are betting it will go down (a down-bet). You receive £10 per point, or lose £10 per point, depending where the market goes.

Spread betting firms make their money on the spread, e.g., you can buy/sell at 507/497. It is quite easy to understand, but rather risky — you are fully at the mercy of market fluctuations; unfavourable positions can develop quickly.

If you want to try spread bets, try and obtain stop-loss limits with them. With spread bets it is unwise to take your eyes off them — even with the alarms built in to Stockwave™.

Explain CFD

A CFD is like buying a share, but without buying it — you do not get a share certificate, but neither do you pay stamp duty. You can also go short, i.e., short-sell more easily than with conventional share trading. CFD is short for contract-for-difference, which is a bit of a mouthful.

The benefits of all of these kind of trades are the ability to go short as well as long — i.e., you can bet on the market going down as well as up, they do not attract taxes, and you can gain leverage, i.e., you can take a position on a market movements for only a small percentage of what is would cost you to buy and equivalent amount of shares. The benefit is flexibility and precision. The trouble with sharp tools of course is that the unwary can injure themselves easily.

If you want to be part of the share owning democracy, by all means do so; pick your stocks and buy them through a discount broker. Just remember that in a bear market, picking winners is a thankless task.

I like your system and I want to use it for day trading...?

We do not endorse day-trading.

Day traders watch market movements in real-time, usually trying to spot a (linear) trend, then open and close a trade quickly so as to lock-in a small profit. Day trading is usually done from specialist stations, and does not attract the usual transaction charges. If you can do this successfully you can make a lot of cash.

People are attracted to day-trading, usually after they have had some success with a modest trade. They think it is easy to make money, and then fantasise about how much they could make if they traded more often and over small timescales, with suitably greater leverage. What they forget is that over very short time periods one is dealing with mostly noise, i.e., random fluctuations. The amount of leverage required to eke a profit from modest market movements is such that large losses can accrue very quickly; day-trading is thus a full-time, fully professional activity — you cannot do it part-time. Stress is a large factor among the day trading community — burn-out is common, as are large losses.

Overall this is a situation where 'less is more' — make your trade with StockWave™ based on your probabilistic analysis, then set your alarms; if anything 'bad' happens, you get a warning on your mobile phone. Alternatively, one can build in a degree of protection with a crafted trade, or simple stop-losses available from your broker. With StockWave™, you do not have to give up the day job.

StockWave™ is not a day-trading system; the data-feeds that are available are either too much delayed, or too coarse. There is also the fact that transaction costs will be higher than for a pure day-trader.

Day traders trade too often; buy-and-holders trade too little; but somewhere in the middle is just about right.

What is the difference between a CFD, an option, a spread bet and a universal stock future?

In essence, not much — all of these types of trade are derivatives, i.e. they depend on something else, and were designed for similar purposes, so, in many ways they are 'the same,' but there are technical details which are very important. The reasons for using these kinds of trades are all or some of the following:

Tax avoidance. You should(!) be aware by now that buying shares includes something called stamp duty. Profits from share trades are also, as far as I know, liable to Capital Gains tax. Plus there is also the broker to pay. The consequence of all of these charges is that small scale, or short term trading is unattractive for the small investor — even if you can see where the market is going in the immediate short term, you cannot profit on it, unless you borrow money to buy larger amounts of shares — but this is very, very risky.
Leverage. Sometimes shares do not move very much, and the return is very small. Buying more shares naturally leads to greater profits, but costs more. The above trades allow you to enter trades by putting up a small amount of cash, to gain similar results to what would be a much larger share-owning position. Leverage can be 'good' or 'bad,' depending on how your last trade went! — but it is good to at least have access to trades which allow this multiplying factor to be utilised.
Hedging. Reducing risk is a major use of futures and options — it is how they began. Buying an option in certain circumstances is like buying insurance.
Short-Selling more easily. In theory one can profit on any kind of market move, as long as you are betting in the right direction. Buying shares and waiting for the price to go up is something of a strait-jacket. Savvy traders of course have the ability to short-sell, i.e., to bet on the market going down — which is a piece of trickery that allows you to sell something you do not own, via the compliance of a helpful broker. Alas, this ability to short-sell easily is not widely available, especially if you want to do so in small amounts — most brokers will not want the hassle of offering such a facility.

In summary:

A CFD is most like share trading without trading the actual shares.
An option is the right, but not the obligation to buy/sell a security before or on the expiration date. A future is similar, but stronger, it is a firm contract to buy/sell or supply a commodity — with options you can simply let them expire worthless. Options are harder to understand, as is their consequences, but can be seen as somewhat safer than futures. Note that options and futures trades are almost never exercised (if this is a factor which is worrying you) — positions are closed by buying/selling the opposite contract on the 'secondary market.'
A spread-bet is an easy-to-understand trade (an important characteristic for derivatives trading); you are simply betting on a price going above or below a certain value, and your profit is your stake multiplied by the points differential. The overall effect is like a future or an option, but the bottom line, i.e., your profit or loss, is obvious.
Futures, options and spread bets are quite similar in character, deciding which one to use is a matter of taste. Universal Stock Futures offered by Liffe are a new product on the market. It is not clear what advantages these will offer over options and spread bets.

My advice would be to use whatever trades you understand well as your basic building blocks, then use these in combinations tailored to your own risk-reward requirement. Remember that StockWave™ payoff and probability graphs can handle any complexity of trade.

What is Teletext? Why use it?

Teletext is a data broadcasting service available in most European countries, but not in the US (they were supposed to have a system, but the corporations could not decide on a common standard). It is a datastream broadcast along with the TV signal.

The interesting point for us, is that share prices are broadcast via teletext, usually in real-time with 15 minute granularity, and all for free. Of course, the Internet is a far superior technology, but until there is widespread broadband availability without punitive costs, teletext is an adequate low cost alternative. There is also the small detail that the data providers want your money! They don't want anyone getting anything for free.

StockWave™ supports the Hauppauge WinTV range of TV/Teletext tuner cards; these are popular and, in their basic models (which is all you need), quite inexpensive. I use a WinTV-USB which plugs into the external port - you do not have to open up the case. Or better still, get a card fitted when you order your next PC. Even if you totally lose interest in the markets, you can always use them for just watching TV!

Having said that, if you have a broadband Internet connection, or you do not have to pay for Web access (somehow), then price data Web download becomes viable. The only residual trouble is the presence of a time delay. For true short term trading you will need to use one of the subscription services to obtain real-time non-delayed quotes. It is unlikely there will ever be free, real-time data available via the Internet. Perhaps if enough citizens wrote to their politicians demanding such a service as an inalienable human right (the right to 'fully participate in the free market of capitalism'), we might get somewhere.

There may be some attempts in the US to create/resurrect some form of 'interactive TV' system, but we are unaware of the details.

Do you yourself trade on the markets?

No.

I prefer writing software to stock-trading, and not being actively involved myself gives me a valuable sense of detachment; if I did trade myself, I would keep StockWave™ for my personal use.

You should put your money where your mouth is!

The experimental versions of StockWave™ do maintain a paper-trading account. Although paper trading is different from real trading — as many investors find out to their disappointment — the results are encouraging to the point of giving apparent validation to the overall rationale behind StockWave™, and also, they seem to be insensitive to absolute market direction. This tells me that I am not wasting my time, and am probably making a good product that is of practical benefit to the user — the intellectual satisfaction plus the monetary reward from the occasional sale of a license is enough for me. One of the pieces of advice I give to investors is not to get too greedy — here I am merely applying it to myself!

Compare and contrast the abilities of the different analyzers?

The Random Walk Analyzer treats the share price as a stochastic process; it is thus our basic algorithm as it makes the weakest assumptions about the underlying process. It is also fast — you can do many thousands of runs in a short period of time; it also acts as a worst-case predictor ( without the presence of trend-breaker news events.)

The wavelets are used for denoising non-stationary timeseries — usually as preparation for further analysis.

The Neural Networks are powerful, general purpose, nonlinear curve-fitting algorithms. StockWave™ offers a choice of architectures; some are better than others at different kinds of tasks, so it is good to try a variety.

The Bayesian correlator is a proprietary StockWave™ method which we believe to be slightly superior to the other methods. It involves some very heavy computations.

How do I configure the datacapture?

The datacapture module was designed to be completely flexible; to make it simpler and faster to configure for most users, some default options were included which correspond to the users likely available hardware, i.e. slow 56k dial-up modem, teletext and dialup, or a broadband 'power user.' If you need to customise the settings, use the 'create data source' functions on the main menu. You can also manually edit the data source file.

When using teletext it is your responsibility to make sure the channel pages and names match up (i.e., BBC2 must be called 'BBC2' in your channel suite and not for example, Antenna-29); make sure the card is tuned-in properly and try and get a good signal — weather can be a problem; rotten weather leads to rotten data, in particular the 'dropped-digit' problem, whereby a stock at 2230 gets read-in as 223. It is also important to be using a good quality coaxial cable to link the tuner card and the aerial socket — an obvious point, but something often overlooked (I speak from experience.)

Getting prices via a quote server will only work if the server accepts the stock symbols expressed as a QUERY_STRING via a GET request. The internals of the data sources are such as to try and use any Web servers in a 'polite' and sensible way, for example, trying to hit a quote server for 300 stock prices at one second intervals, is not very nice. If people started behaving in this way, it is likely these free quote services would be withdrawn, hence this kind of thing is internally forbidden within StockWave™ — we prevent you from making a nuisance of yourself.

Similar politeness rules have been applied internally to the Web agents — they all die after a certain amount of time, will not revisit Web pages and will not exhaustively search single domains (or rather, the link scoring algorithm makes this probabilistically unlikely) - rampaging robots are disliked intensely by Web administrators — you could find yourself blocked.

Web based information sources seem to be in a state of flux at the moment — services are being withdrawn, some which were free now require registration, or even a subscription. The big companies are flailing around trying to find ways to make money from the Internet — now that advertising seems not to work they are reverting to old business models. This is obviously a problem for the little guy — getting charged for data and news ( much of which is irrelevant anyway) is a proportionally higher burden on him than a big company; he doesn't have the £10K a year to rent(!) a trading station. We take the view that subscriptions to Web based news services are not likely to be worth the money.

I live in <country X> but you only have settings for the UK, US and Europe!

I want everyone to be able to use my program — believe me, I do. But since I am UK-based, the sensible original plan was to start with a UK-centric system, then add-in modifications for the next major overseas markets, i.e., US and the EU. If the program is successful, I would then expand it to other countries.

If anyone out there would like to volunteer their language skills, I would like to hear from you. There may even be some money in it for you!

I emailed you, but you haven't replied?

I get a lot of email and I am a busy man — got lots of programming to do! Genuine, polite, emails will all eventually be replied to — please be patient; folks who have actually paid for the software get priority of course.

Much of my email is not replied to, mainly because it falls into one of the following categories -

Cranks — usually with some theory about the stock market which they want to debate with me, or with some socio-political agenda that they want to push, or someone whose ideas I have 'stolen.' A new type of crank that has only appeared recently is the financial professional who believes that my product is 'dangerous,' and will bring 'chaos to the markets' if used by the general public.
Rude people who want to indulge in abusing me for some reason; usually cranks who feel they have been 'provoked' by the lack of a response.
Lazy people who will not spend 5 minutes reading the help information. Or who want pointless additions made to the application, failing to understand what it is all about. I have tried to make the program as simple as possible to use — in almost all situations there is only a small number of things you can actually do, and they are all valid.

Why are there gaps in the chart?

The StockWave™ chart uses real-time rather than any invented 'trading time.' This is done because we think real-time is a bit more important; for example, when you are checking when news events happened in relation to price moves, you want them to be in the right place. It is possible to use a 'trading time' which eliminates the non-trading portions of the day, but then this changes on geographic location. Trading time can be more cosmetically pleasing, but real-time is more accurate.

Why is start-up so slow?

StockWave™ can monitor a lot of stocks, and store lots of information about each on the hard drive — we do not load all of this information at start-up as that would be stupid, we do however have to load a minimal set of information about each stock, and this is what takes time. We are looking at ways to make this faster. In practice, this should not be a problem — our advice is to simply keep StockWave™ running all the time, do not switch it off.

Is StockWave™ magic?

StockWave™ is not Aladdin's lamp; it will not provide you with riches on your command. It is a tool and nothing more; but a good one at that.

What is StockWave™ about then really? if it's not magic?

Some observations for the private investor -

The stock market can make you money, but
The stock market is risky and you cannot simply assume that bull markets will last forever, or should we currently be enduring a bear, that another bull will be along in a minute.
The guides we place our faith in to help us through these dangers (IFAs, Fund Managers, Analysts, financial journalists) turn out to be very poor value for money; in fact they turn out to little more than overpaid con-men; we can only trust ourselves in financial matters. We must do our own analysis and investing.
With the Internet, the individual investor has all the potential information he needs to make sound investment decisions. In fact, there is so much information available, that only high speed computers and advanced data processing algorithms can make sense of it.
With derivatives, absolute market direction does not matter.

When you tie all these considerations together into a software package, you get StockWave™.

The financial world is like a jungle; full of danger, full of mysterious creatures and unknown vistas. If someone dropped you naked and unexpectedly into the Amazon, you would not last very long. If however, someone told you where to go, gave you a map and a compass, plus a Kalashnikov machine gun, then ...you still could get eaten alive. But at least you would have a chance, at least the ultimate outcome would be in your own hands; think of StockWave™ as the compass, the map and the machine gun you are going to need, should you venture forth.

When will it be safe to return to equities?

Conventional wisdom would probably say something like -

IF <list of conditions involving the FTSE, interest rates, macroeconomic data, etc. THEN the economy is on the upturn.

But this is really missing the point — by now having become a sophisticated active investor you should have the ability to make a profit irrespective of market directions; don't revert to hopelessly optimistic buy and hold strategies.

Help! My data looks wonky!

Data subscription services can be very expensive ~ perhaps £600 a year just for share prices and if you want options and futures prices, then it goes up even further still.

StockWave™ relies on free data sources, saving you money. The price to be paid for this is the lack of any guarantee of data service or reliability; the services can be withdrawn or can be changed. StockWave™ uses a highly configurable data extraction algorithm — this means that for most likely changes, any update to the software will only involve the download of a small text file. So, for the most part, changes in the services can be dealt with — total withdrawal cannot of course. Sources like the BBC for example, are funded by the public, so cannot be withdrawn; other commercial sources which supply free delayed prices over the Web could be — their rationale is to use the free delayed services as a useful giveaway, while actually making money from the real-time streaming subscription.

Further problems arise if the data we get has been garbled or corrupted in some way — we've got the numbers but they don't mean anything. A particular problem with teletext is that of bad weather and dropped digits; there is also anecdotal evidence that providers of free data will occasionally pump out some garbage values, just for the sake of it — naturally they make a point of the integrity of the data from their subscription services.

Filtering stock data for bad reads presents something of a problem - wild fluctuations of share price are quite common; excluding such values as garbage could be a costly error. At the average resolution of data feed we are using, say 15 minutes, it is not uncommon for a stock to lose 40% or more of its value on the release of some bad news. StockWave™ does have a simple cleaning algorithm which the user can use, but it is not foolproof — you may have to manually edit the corresponding data-file. Normally we would strongly discourage the user from changing any computer generated files as they can be rendered unreadable rather easily (and could cause your program to crash), but sometimes it is necessary.

Finally, it is very important to have good quality data for the analysis algorithms — pre-cleaning is not done automatically though, so the user must do it himself.

What do these regions on the stock chart mean?

There are two types of overlay we can have on the stock chart; the first type is the prediction overlay — if the currently open portfolio has some available analysis on the current stock, then the predictor overlays will be drawn on the chart; the colours of this chart represent the likelihood of a future share price being achieved and the contour lines within the predictor show the probabilities in percentage terms, i.e., between the two 50 lines, there is a 50% chance that the price will remain within them. (Isn't this a nice way and useful way of viewing our analysis?)

The other type of overlay relates to currently active trades within the current portfolio; there are three types of region, red, amber and green - these broadly correspond to good and bad/safe and dangerous regions for the share price to go. Note that — and it is very important to realize this - they do not represent regions of current profit and loss — they only show whether a trade is 'on track' or not, to make a profit at the implied closing time of the trade.

This needs some explanation; StockWave™ is set-up to let the user make any kind of complicated trade he can imagine — not all of these trades can be exercised early; with share trades, CFDs and spread bets we can exit a trade at any time, but with options and futures we can only exercise at the expiry date; to close options and futures trades early involves making the inverse trade on the secondary market, prices for which we do not have access to.

If you want to get an estimate of your current position, you can do this from your portfolio; estimated positions can be calculated from the last mid-price of the underlying equity (derivatives positions are estimated by pretending that American exercise is possible), and this may not be very accurate — to get an actual position you need to check with the broker you made the trade with — and you should be doing this fairly regularly anyway.

Remember that all trades, to begin with, start out at a loss — the spread and transaction costs ensure this. You are hoping that after a period of time, the trade will reach profit. The usefulness of the red/green/amber regions is that they give an indication of how well the trade is progressing in relation to our desired, final outcome. In practice, you let your trade run for a while, then get your position from your broker; have a look at the trade regions on your chart to see where it is heading — from this you can, hopefully, make good decisions about when to close out your trades. Whenever you hit a stop loss or a profit target you should close your trade; you should also have a maximum running time for your trade.

To recap — trade regions on the chart represent the 'On-Track-ness' of the trade; they do not refer to current position — for that you need to check with your broker.

What is the MFI scale?

The trouble with analyzing news is that there is too much of it, and most of it being utterly irrelevant; but perhaps buried within a vast pile of dross may lie some nuggets, the existence of which we cannot discount.

Most of what is called 'news' is nothing but; alas there is so much space to be filled in the various media that something must take up the space, so a lot of the time what you get are highly subjective interpretations-after-the-fact, or hand-waving speculations. The MFI scale is an attempt to usefully categorize news events so that we can quickly discover their relevance to our interests, so that we can answer the three main important questions -

Is this important? Does it actually matter — a lot of stories are just 'fluff.'
Is it good or bad for me? Remember that my interests may not be your interests — you could be long, while I may be short.
Can I believe it? The obvious caveats about rumours and the Internet apply here — but consider this further; it is common practice by many respectable organisations to 'plant' stories in the news media.

The MFI scale has three components —

M is for magnitude. It starts at zero, which would be the most innocuous story reported in a news channel, and grows open-endedly, but with a logarithmic scaling so that the a category 1 event is ten times greater than a zero, and so on. We calibrate the scale to a maximum of 13 — this would be an extinction event for the human race, e.g., global nuclear holocaust or 10 mile diameter comet striking the Earth. 13 is chosen as the upper limit as 13 is an unlucky number in most cultures. The analogy between the Richter scale for earthquakes is quite deliberate.
F is for favourability; 1.0 means totally positive, 0.0 totally negative, and 0.5 means neither good nor bad.
I is for integrity; 1.0 means totally credible, 0.0 totally incredible, and 0.5 means neither credible nor incredible.

Some of you may like to think of the F and I variables as fuzzy sets, but as yet they are not used as such by StockWave™.

Note the expressivity of the scale — we think it captures the essence of what you really need to know — notice how it is possible to have events which are very good, but too small to be of any importance, and highly important events but whose impact we don't actually know.

With a sensible looking MFI scale, a collection of news feeds, and some reasonable algorithms for making the assignments we can turn a stream of news into a stream of numbers; when you have something expressed as numbers a whole new world of analysis opens up — we are moving from the qualitative to the quantitative. With these numbers we can scan large numbers of news stories for what is interesting to us, and also improve our predictors more directly; news stories become news events, and the news events form a news event stream. The news event stream is the overall news influence - sometimes it is known as 'market sentiment;' we get the event stream from the individual events by summing the values (remembering the scale is logarithmic) and adding in an exponentially decaying factor to show the influence of the event. Note that events have effects which last for a period; the rate of decay of the events influence can be used to define a forgetting time for an event. Big events have longer lasting influences than short ones, of course. We hope this all seems intuitively reasonable.

Our MFI-scaled event stream is still an externally defined ad-hoc scale — to do useful things with it we need to calibrate the response of the share price. There are two main ways we can do this. The first way is attractively visual; we look in our historical news record, ascribe MFI values then look at how the share price behaved around this time. For this we define 3 characteristics of the share price in the measuring period (in practice, one day)

Step-jump — the change in the share price over the day of the event.
Trend shear — the change in the trend over the day.
Excitement — the change in the volume between the day of the event and the previous day

You can form scatter plots of these values; what you should find is that clusters are formed which show the qualitative nature of the event; these clusters are -

Irrelevant — it could be very positive, or very negative, but it is just not important enough to make any significant and identifiable impact on the share price; whatever effect it has will be 'lost among the background noise' so-to-speak.
Important, Good — unambiguous major and positive; e.g., your company wins a billion dollar contract. Yay!
Important, Bad — unambiguously major and negative; e.g., your company loses a billion dollar criminal damages lawsuit. Oww!
Important, Unclear — e.g., CEO resigns; its got to be important, but it could make the share price go either way, depending on what investors thought of him; genius or dickhead.

The important thing to know is the boundary between the relevant and irrelevant regions of the chart; this lets us set filter levels for our news gathering and our alarm system — irrelevant stories are ignored, relevant ones set off a warning. We can also use this analysis to generate expected share price changes.

The second method to apply to our MFI event stream is more direct and powerful, but not so visually appealing or expository; we use the event stream as an extra set of inputs into our neural net analyzers. Once we have trained our neural net and a big event happens we can re-predict with the neural net to get a prediction update. One nice feature of the neural nets is that they are very tolerant of the kind of arbitrariness and imprecision likely to be inherent in an MFI data set — perhaps it worried you that ascribing exact values to events was not possible, but with the neural net, as long as you are somewhere in the ballpark, it will work OK.

The MFI scale and its attendant algorithms is an attempt to produce a practically useful news event warning system by using statistical analysis to work out the likely identifiable immediate effects of a news event; this stops us getting 'caught out' by unusual events, stops us needing to try and make sense of vast amounts of verbiage (avoiding inevitable brainache), allows us to avoid the need for constant monitoring of the news feeds (and hold down a proper job, for example), and lets us trade through chaos; we can plot a steady course when the market is in an uproar, or take action when required. Reacting too much leads to poor performance as our trading costs mount; reacting too little means that one day we get burned — badly. But with MFI we can react like the Baby Bear's porridge — 'just right!'

Before we get carried away, we should point out that MFI does not cover all the bases; situations can arise whereby a stream of 'irrelevant' events - as so described by MFI — are nothing but. This would be when a lot of apparently small events over an extended period are in fact, 'little pieces of the jigsaw' with regard to the company's prospects; here we need to be able to retain our irrelevant events and then send them to our fuzzy expert system.

Do I really need broadband internet access?

Size matters in the field of advanced data analysis; big processor, lots of memory, and a good data input stream are what you want. Getting data has always been a problem in the past for the small investor — all the providers want you to sign up for their subscription services, which can be expensive. At StockWave™ we don't believe you should have to pay for just for the raw data, and it is here that the Internet comes to the rescue — it is the biggest information resource on Earth; whatever it is you want to know, if it is public-domain knowable, then it should be on the net, somewhere; it is this 'somewhere' that causes the problems for to actually find precisely what you want you need some smart algorithms to find it — but this is what StockWave™ has; so you don't need to worry about it!

So far, so good — to avoid signing up for costly subscription services, we instead scour the Internet picking up little pieces of our information jigsaw here and wherever. Trouble is, this will eat into your phone bill like a dog on meat — even trying to be very sparing in our usage, e.g., using Web news download only once per hour, only using the Web agent at night on the off-peak, making sure the auto-disconnect is set low on your dialup connection, the charges will still mount up.

Teletext only, i.e., price-only data download is obviously cheap, but it doesn't really allow StockWave™ to be used to its fullest potential; the next stage is obviously to use the teletext and dialup configuration, but because of high telephone charges in the UK set by British Telecom (still basically a monopoly whatever the Competition Minister says), you will probably find you end up paying even more than for a broadband user. So, what it boils down to in practice, is that you should get a broadband connection ASAP once you start using StockWave™ seriously. The best deals to be had will usually be available from independent telephone or cable TV companies, but your area may not be covered so you may have to rely on bad old BT; and here is more bad news — not all local exchanges can handle ADSL and BT will only upgrade them if they register 'sufficient demand' within an area.

Sufficient demand is, as you might expect, an entirely arbitrary figure - 'whatever we say it is' — and should your area somehow cross the threshold, the timescale for upgrading the exchange lies at BTs discretion — i.e. 'whenever we feel like it.' Registering interest in getting broadband in your area is done by signing up on the BT Web site — if you can actually find the signup page. Despite being heavily criticised by consumer groups, business advocates and even the government for dragging its feet over the rolling out of broadband, BT has remained largely unmoved, after all slow dialup modems accessing the Internet at local call rates brings in a lot of cash. All of this is because in Britain we have the worst of all possible worlds — a de facto privately owned monopoly; there is neither a genuine free market to drive prices down through competition, or a state-owned utility which would be run in the public interest.

Apologies to any American users who have been either baffled or sidesplit with laughter at the previous discussion about broadband availability in the UK.

I can't find any profitable trades!

StockWave™ tries to be at the very least, a good educational tool, i.e., it tells you, or even better, shows you in direct and obvious visual terms, what it is you want to know, you really want to know. As regards the stock market, the ultimate question is — 'how much money will I make?'

What you will find using the basic analyzers and the simple stock trades is that the expected profits, if any, are very marginal — much too low to justify taking on the risk. This is the first thing users notice soon after playing around with the trade creator and it illustrates one of our main points of argument — 'buying stock' is an inflexible straitjacket for the small investor and that because of stamp duty and brokers commission, all you are likely to do is lose money. So don't do it -

Do not trade shares; use CFDs or other derivative-based products
Use the cheapest execution-only broker you can find with the smallest spreads

Learn how to make combinations of trades to craft a specific risk profile, or alternatively use the automatic trade searcher to find a trade for you. You should also use the more advanced analysis techniques if possible, even though they can take a long time to run (set them running overnight) The trouble with the basic stochastic analyzer is that it is too conservative at short times and too restrictive at long times; the better analyzers will give you tighter contours, and hence better chance of profit.

I've done all that; I used the advanced analyzers and the trade searcher, but there's still nothing very profitable...

This is possible, and...it is a 'bitch' — but that's just the way it is; the advanced analyzers couldn't find any more information in the data, probably because there wasn't any there to begin with, and the available trades just don't have tight enough spreads or low enough trading costs. So why not sit this one out? It won't kill you. Look at it this way, half the battle in trying to make money on the markets is not simply making it, but also avoiding losing it; if you think you'll get your arse kicked, just take a break.

I don't like what you said about something!

Everyone is entitled to their point of view, so it is often said - but if you are, in fact, totally ignorant, your opinion — while keenly held- will be utterly worthless. Not all opinions are created equal.

You may not like what I have to say, but bear with me, hear me out — especially if I seem accurate in other matters - like has nothing to do with anything; the goal of this software is to give the little guy a chance at making some money in that Rich Mans Casino called the Stock Market. To stand a chance at this, with the odds stacked so highly against us, we need to lift the veil from our eyes to see things as they truly are. In this series of mini investment guides I am trying to 'tell it like it is' as best as I can, cutting through all the jargon and baloney traditionally found in the investment literature.

The website material is written in a populist style designed for a mass-market audience; you may well not appreciate the sense of humour, if for example ... you are a pompous ass in love with your own importance and authority. To be fair though, we probably didn't want you as a customer to begin with, and couldn't be bothered with you even if you did part with the readies as we know your type - clogging up the tech support helpline ranting over the least little thing or making dozens of trivial feature requests, demanding away like a 5 year old who's never rightly tasted the back of a hand.

Truth can be ugly. Get over it.

Your views on investing are highly cynical...

I am not a pundit or a moral guardian, your rabbi, priest, imam, shrink...and I do not want to be. You should make your own decisions based upon a complete appraisal of all available data and in accord with your own ethical standards. StockWave™ is the tool which can give you access to all you need to know about a company — it can find the information, but the decision must be yours.

If you want real cynicism, try looking at the traditional investment industry — a bunch of guys in sharp suits telling little old ladies about the next 'high return/low risk' investment idea, or the army of pundits currently trying to talk up 'the next bull market.'

Although it is expressly not the focus of this application, many people will be highly interested in the topic of long term investing. Our opinion is that truly long term predictions are impossible — even our probabilistic ones; these will mostly just say — 'if you wait long enough anything can happen.' In our wider discussions of stock picking we have noted however, that most bubbles arise from fashionable, technology based investments, i.e., novelty. If you want to invest for the long term you should go for what are undeniably recurrent human needs (of course this is only partially true; housing is a recurrent human need and property bubbles are common.)

Which brings me to a particular dislike of mine; tobacco companies. Whatever you think of the capitalist system as a whole, tobacco companies are certainly, an unacceptable face of it. Look at the facts -

Their basic product is lethal.
They suppressed research which proved this long ago.
No other industry can or could possibly get away with killing so many people as part of its normal business. They will kill more people, legally, than Al Qaeda ever will.

But, alas, as I have said above — truth can be ugly. Simple fact is, tobacco companies are great investments.

The demand for cigarettes is unrelenting. And just wait till they get a foothold in China.
They have huge political influence; their advisors and lobbyists are a who's who of high office holders.
They get sued all the time, and always win. Even if one day a class action is successful against them, the multi-billion dollar payout will hardly dent their profits.
Legislation has made no impact whatsoever on tobacco sales. When taxed, you end up with a black market run by criminals; when prevented from advertising they just find ever more subtle ways of getting their message across.
Everyone knows cigarettes are bad for them, but they still don't care.

The tobacco kings are slick, ruthless and well-connected — they sell a product people love, and will never stop loving. When stymied, they will find a way round. What you have is a concoction of oily political corruption on the Right, spinelessness on the Left and a complete disregard for reason from the general, smoking public.

When investing in tobacco, we are betting on the future incompetence of progressive politicians, and the stupidity of people in general. Its a growth industry — let's be honest.

How come I got 1-minute resolution data from a 15-minute resolution share price data feed?

You didn't, because you can't.

Since StockWave™ uses free data sources we cannot rely on their quality — we have to take what is available. Sometimes this will leave you with gaps in your price data. As long as these gaps are not too big, StockWave™ can make a reasonable interpolation for them. We need to do this whenever we are doing an analysis — all of the algorithms we use need as input a full complement of regularly sampled data points, so if the data is dirty, gappy or irregular we need to do some kind of a 'fix-up.'

It is this fix-up routine which is doing the interpolation and producing an apparently much higher resolution data feed from a lower resolution.

BTW — do not try this deliberately thinking it will improve your predictions! It will not. The higher sampled data feed will contain more data, but no more information than the lower sampled feed (there is a difference). All you will do will be to make your analysis take much longer to run and your predictions less crisp.

What are the correct parameter values for the analysis algorithms?

A good user interface should not be too fiddly and default parameter values should be supplied which give reasonably acceptable performance...

When you do some analysis with StockWave™, up pops a little form with values for you to choose or to fill in. What these things actually are will probably be unknown to you, and what values they 'should' take a complete mystery.

Trying to simplify things for the user is hard, eventually you get to some limit and just have to leave him to it, so, the short answer to this question is then that we simply don't know — if we knew what they should be, they would be set automatically; you wouldn't get a choice in the matter. What you are given are a range of a priori 'reasonable' values — whether they are practically so, is a matter of trial and error.

Let's take the example of the back-propagation neural analyzer and make some observations:

number of inputs sets the detail level of our analysis; too short and the predictions will be crap. As you increase it, performance should improve until a point, then it will level off. Higher values increase training time. So you want this to be as big as it needs to be, but no bigger.
using more hidden neurons means having better generalization/learning capacity, but will result in a longer training time. A rule of thumb for number of hiddens is at least twice the number of inputs, but probably not as high as three times that number.
using more than one hidden layers has similar consequences to the previous entry, except that it will result in much longer training times. It is thought one or two layers is the most you should ever need.
momentum, learning rate, and bias are used to speed up training; they may speed it up, but the performance could be affected. You are really on your own here.

Neural networks can work very well on noisy input data, once trained, but using noisy input data to train on can be very time consuming, and may not even work — it can slosh around indefinitely hardly improving its performance at all. What you need to do is preprocess the input data — separate it into a trend and a noise component; the neural net can be trained on the smoother trend, and when we later do our Monte Carlo simulation, the noise can be added back in.

In theory, you can do anything with a neural net — that is the common belief — but in practice, you will have to 'fiddle around' quite a bit. This may worry you, but shouldn't — the parameters you change should be found to give reasonable performance within a range, i.e., they are not too sensitive to exact values. With experience you can work out for yourself what is 'reasonable' in this respect. Experimentation is necessary!

Similar observations apply to all the other analyzers.

All you are saying is that I should just 'fiddle about' until some inspiration hits me!

Fiddling about — yes, but not without some effective criterion for judging the choices made; what I should have made more obvious is the need to use back-testing during our 'fiddles.' Suppose you are thinking about an option trade and want a prediction for next month, try predicting last months price from the previous 3 months data — this will let you find reasonable parameter values for the analyzer, and what level of preprocessing should be used; when you get some configuration which gives a reasonable prediction for last months chart, use this parameter configuration to make your desired prediction for next month.

What we want from our analysis is a firm prediction — but this is impossible, so forget it; instead you get a probabilistic prediction for which StockWave™ can construct an optimised trade. From the probability contours we can get an idea as to highs and lows, general direction and timing of changes of direction, plus an indication of how widely a stock will swing within a given period. This should be enough to make for a viable strategy. But remember also that in many cases, there will be little to exploit in the timeseries anyway; if there is nothing there to begin with, then you won't find it with any method.

When you are looking to construct a trade you should be —

hoping for the best, while
preparing for the worst; this way your surprises will be pleasant ones.

The most potentially profitable trade will occur when you are prepared to take some quite definite position as to market direction or timing; you should then look at what the worst potential outcome for this case should be and try to hedge against this case. Small losses are unavoidable and will occur, what we need to avoid is the wipe-out scenario.

Look upon data analysis as a discovery-feedback process; try it and see what happens — if it looks crazy, it probably is; if the analyses all disagree, then the job has not been done. Remember also that your analysis should not be brittle; i.e., if you tweak the parameters a little, then the final prediction should not change that much.

'There are no magic bullets' — which is exactly why we don't rely on any one analysis method, and why we believe all types of data are important.

I've read all your stuff, but I just got confused — what's your point? What idea are you really selling?

Go as slowly as you need to; there is no rush. Paddle in the shallows before you try to swim the Channel.

When it comes to investment, you — the small investor — have only two real choices available. You should either:

Put any spare cash you have into a high interest savings account from a reputable high street bank, after having paid off any high interest debts like credit cards, or other personal loans. (If you are truly paranoid, then you might even want to buy gold and bury it in a secret location.)

Or, if you want more than this, you have to:

Become an active, sophisticated trader-investor.

There is no middle ground here; this is extremely important so I will repeat it — there is no middle ground; using fund managers is a waste of money, buying small amounts of shares on your own is pointless due to the tax charged on it.

If you want to go the second route, then you need some other things as well:

Access to all relevant news and data in a timely manner
Scientific data analysis techniques
An understanding of derivatives, and their application to the creation of sophisticated trading strategies.

Which furthermore means you need something like StockWave™ — and since there is nothing 'like StockWave™,' then you need StockWave™.

Are you an <X>?

Where X is, for example:

Extreme free marketeer
Marxist
Libertarian
Anarchist
Hedge fund insider
Communist
Liberal
Renegade economist
Alien being, etc.

My answer to all such questions is a simple one...

No. But even if I was, what difference should it make?!

When people hear something which troubles them, something they do not want to hear — they need to find ways to discredit it, to put their minds at rest; rather than having to deal with the argument directly, which they often cannot do, they instead stick a label on the person. This label means 'bad' to whatever value system they hold — and since we do not have to listen to bad people, or take their arguments seriously, we can consider them dealt with.

Give me some guidelines for choosing parameter settings in doing the data analysis — I am getting some odd results?!

The most common truism in the field of data analysis is — garbage in, garbage out — and it is right enough. Don't stick gappy, dirty data into your fancy-pants algorithm and somehow expect it to compensate; it won't because it can't.

Some guidelines —

Always clean your data before you create a sample for analysis, if need be, manually delete any bad points — you can do all this from the stock chart Display Menu. Check the sample selection in the selection window — visually check that it looks like a rectified version of your price data.
Choose a sample length which is commensurate with what you want to predict; if you are interested in next months option and then choose, say three months worth of data.
Choosing a sampling resolution which your computer can handle; start out low and then work your way to higher resolutions.
Don't choose a higher sampling resolution than the data feeds allow; if you have been getting 20-minute data, then attempting to sample it at 1-minute resolution will not do you any good — all that will happen is that the rectification algorithm will add extra points to compensate for what is missing; since these extra points will be 'guesses' based upon the original sampled data, they will not add any new information to your analysis.
Too low a number of data samples will not give the algorithms enough to work on. Practical lower limit is about 500 data points
Too high an amount of data will simply choke the algorithms; practical upper limit depends on your machine — 10,000 is probably ballpark. Using data resolutions of 1 hour and 15 minutes are probably the practical choices for most people.
The advanced techniques are power-hungry in the extreme — use the simpler analyzers first. If needed, do the heavy number crunching overnight.

The reason why there are a range of analysis methods, and a range of parameter choices within the program is a strength, not a weakness — it allows us to check for parameter sensitivity — i.e., that our prediction algorithm is stable, and thus we can look for corroboration; independent analyses reaching more or less the same conclusions are more believable.

One aspect of human nature we have to deal with is the automatic reaction to 'turn all the knobs up to 10' — so to speak. Bear in mind what it is you are trying to achieve — in a nutshell what you want to know at some future date is whether the price will go up, go down, or just stay where it is; if you have this information consistently, then you make a lot of money. Note that this, potentially very lucrative, information is not microscopically detailed — we just need to be 'ballpark' or rather the right side of the fence.

"Respect the Data for Data is King"

- use all the data, but only the data, and if something looks funny, then it is...

You mention rectification a few times, what is it?

The rectification algorithm is a cute piece of code that allows us to accept irregularly sampled data, possibly with garbage and missing values — the sort of data feeds that you are likely to get with free services (since its free, we can't really complain about the quality can we?) and use them in our data analysis.

All the analysis techniques need for their input, clean, regularly sampled data at high enough resolution. The costs of data subscription services are partly to do with this quality guarantee; free data is just a useful giveaway, thought to be practically useless to the serious stock watcher.

Good Data Management — A Guide

The data gathered by your program is of the very greatest importance - everything else, every analysis, prediction, simulation that you do depends on it. One obvious consequence of this is that if the data is garbage, any analysis based on it will be also. This is one reason why normal data providers charge you a lot; since we get our data for free, we can't expect too much from our providers — which means we have to put some occasional effort in ourselves.

First of all, most importantly — DON'T WORRY! Garbage is very easy to clean, and we do not have to do it very regularly — just let the datacapture run and accept what you are given; we only need to get the scrubbing brushes out when we intend to do some analysis.

Prices

Teletext can suffer from dropped-digits; these are easy to clean from the stock chart window. If there are any values that are hard to shift but won't be cleaned, then you need to delete them — again from the stock chart. Zoom in to isolate the bad values, then delete values above or below a threshold — the program will delete these values from our datafiles for the selected timeframe in the stock chart window.

With teletext make sure you are getting a good reception — this is fundamental; note that your PC and monitor can itself cause interference. Possible solutions to this include:

tuning your receiver to a different transmitter for the station you want; these can vary in strength and quality
use a good quality connecting cable from the aerial to the WinTV card
use a different station, e.g., if the BBC has a dirty signal, try Channel 4
cable and satellite may also provide share prices, and have a better signal
small rearrangments of your PC and monitor position can help mask or accentuate interference — so move your system around slightly
your WinTV card has to have its channels setup up properly, i.e. the channel names which it uses have to be the same as the ones used in the program

Sometimes the BBC 'inexplicably' stops auto-updating, or will pump out spurious values; this doesn't happen very often, but it is something to watch out for. It is possible that this is done deliberately so that their data providers paid-for services maintain a quality advantage. The German station ZDF provides a very comprehensive service.

Tick data is very hard to come by — Teletext is thus a very cost-effective way of getting this type of price data; it amounts to a one-off payment to your friendly public service broadcaster — so don't diss teletext! Even on the Internet free tick data is elusive - the only game in town seems to be Yahoo, which can have quite a long time delay on it — at least 15 minutes and more likely 20 or 30. Internet prices are only practical if you have broadband — dialup charges would kill you. There is also the potential worry that a company like Yahoo — which is a commercial business and ultimately wants to sell you its services — might just pull the plug if it feels the service is being 'abused,' i.e., is failing to operate as a suitable bridge to casual users becoming paid-up subscribers.

News Stories

I think the news annotation feature of the stock chart is really good - seeing the news stories placed at the time they occurred in relation to share price moves is very intuitive; but what can mess this up is having stories taking place at the wrong times, or duplicate stories or stories which aren't stories at all. These kinds of problem are pretty much unavoidable — StockWave™ uses a crude but effective automatic algorithm which can scan and extract for interesting 'information chunks' or 'articles' — this lets us decimate the torrent of utter garbage that HTML has become to get at the content which is important, thus avoiding wasting our time on...unbelievable offers, cutting edge graphic design, animation which serves no purpose, tracking programs, pop-ups, pop-unders, view-sitters, etc., etc.

Getting a reasonable algorithm, working across many different types of pages was tricky — and so it is not 100% accurate, thus before we do any analyses we need to clean our news files as well; but this is pretty easy.

To start with make sure you load all the news stories, then:

remove duplicates
aggregate similar stories to one or a few good exemplars, e.g. you might get a takeover bid reported 10 times — reduce this to one or two articles.
edit the articles to your taste; in particular, get the dates and times right — this is crucial. The 'safest' date and time formats to use are, e.g., 23 November 2003 and 15:35, i.e., date long-month full-year and 24 hour clock.
delete any obvious rubbish, e.g., keyword lists used in the meta-content sections of Web pages, or stuff that is simply not relevant.

Once done you should have a good set of news articles, from here you have two choices — to update or reset the news files. Be careful that you choose the right one!

Choosing to update your news files will proceed to take the current list of articles and use them to update the news files on your hard disk, i.e., it appends the cleaned articles to the correct files, or creates a new file if it needs to.
Choosing to reset your news files does the same as update with one important exception — it deletes all of your current news files, before creating an entirely new set of files from the list of cleaned articles.

Mind you choose the right one.

Timelines

Similar comments apply to timelines as do news articles, the only difference being that a timeline is usually kept in a single file, having an overall cohesive purpose, e.g., AZN_directors_dealings.tln

There are spikes in my histogram — is this bad?

In generating a probability, we need to have enough samples, so when we see spikes, it is usually a bad sign. You probably need more samples.

Where is the user manual?

No-one ever reads the user manual...so there isn't one.

Most forms have a 'What is this?' button — so start there; try searching the local documents.

What's so great about this anyway?

Good. You are learning — 'Healthy Skepticism' — this is a very useful quality for a potential investor.

Some wee observations for you:

With other programs you have to sign-up for some data subscription; this can be expensive depending on what you get. It also locks you in to receiving your information from one particular place. Plus you will need to pay for your broadband Internet access as well.
Having signed-up for one provider, you will naturally be reluctant to use another; you will also tend to be using your providers trading software — this will not have the analysis tools that StockWave™ has.

But these guys give me real-time data, you use delayed sources...

Unless you are daytrading or are into statistical arbitrage, you don't really need real-time data; and these cases are not really available to the small investor anyway, or shouldn't be. (Please do not be tempted into daytrading — it is simple madness.)

StockWave™ has analysis techniques that the big data providers/brokerages do not provide and what is more cannot provide even if they wanted to; most providers own trading platforms are what is known as 'thin clients' — they offload most of the work to a server. It is unlikely any data provider will let you use their servers to do your Monte Carlo simulations — its just not on.
StockWave™ has some unique stuff in it. We can't provide more details as we might want to file some patents later on.

A lot of the buzzwords you use in the material I've heard of before — you aren't really doing anything original; this stuff is available elsewhere...

The 'buzzwords' represent the basic raw ingredients for our algorithms, nothing more; there is still significant work to be done. As for availability, well yes, perhaps...for example you may be able to get a neural network toolbox, or a Monte Carlo simulation module, usually as a bolt-on to some other software like Excel, Matlab or Mathematica. These other applications — very fine though they maybe are just not designed for trading; what is more these 'extra' programs that you will need will be very expensive in their own right, as will the bolt-on modules.
Some of the 'buzzwords' and their potential application to trading had been suggested as long ago as the early 1990s but interest in them died off for a number of good reasons; the computing power you need is substantial — ordinary PCs had nothing like the 'grunt' required back then; data was expensive — and the Internet hadn't really got started as far as the general public were concerned. The only players with the resources to try out the new techniques were the big finance houses — and then something happened; the biggest, rudest, most fill-yer-boots-ever Bull Market hit the City. And you do not need to be clever to make money in a Bull Market, in fact, you have to be unlucky not to — dart throwing chimpanzees could easily become 'star' Fund Managers; the argument then goes — who needs a bunch of Ivy League geeks talking technical jargon in a back room full of supercomputers, when all you need is a team of aggressive stripy-shirted barrow-boys in your trading room making the deals?!
When a new 'buzzword' comes along there is usually tremendous hype at the prospect of a new 'revolution' and then disappointment as it turns out not to live up to the inflated claims of its early supporters. What we tend to find is that some new technique turns out to be useful within a certain limited regime which has only a minimal overlap with the domain of practical application — nothing, on its own, appears to be the silver bullet which will let us for example, talk to computers, have them understand our needs and emotions, be our friends, etc. There is a growing realisation that using hybrid techniques is the way to go — even though the precise 'what' and 'how' can be tricky; for example, you could use genetic algorithms to optimise neural network architectures, or use combined neuro-fuzzy systems for high level controllers, or even extract rules from trained neural nets. The possibilities are endless. From this viewpoint one can see the possibility of using all different kinds of information — share price history, news events and fundamentals — to make investment decisions. This is surely the way to go as it frees the investor from having to choose a particular analysis-ghetto — and this is the program of StockWave™. As far as we know — no-one has managed this yet, and even if they have, they are not putting it into software for sale to the general public.
Nowadays, with broadband Internet and multi-GHz desktop PCs the man in the street has the power; and with uncertain markets we need all the 'cleverness' we can find.
StockWave™ is very inexpensive; the whole thing is a mere fraction of what other similar genre software costs. There is just nothing comparable to it.
StockWave™ steers you well away from the conventional voodoo — you know what I mean — the blizzard of technical 'indicators' you get overlaid on historical charts — what's all that about?! In our opinion, absolutely nothing at all.
Our interface is pretty clean when you compare it with other programs; for the last thing you want to be when trading is to become confused.
Nothing else give you as simple a conclusion as we do — expected profit, likelihood of profit and maximum profit or loss. Its as straightforward as it can be.

What is a Statistical Anomaly? What application does it have in Data Mining?

So far we have downplayed the importance of fundamentals in our data analysis, but we do think they are important and can be useful to the investor, alas not in the way they are currently used.

Currently the fundamentalist-investor will log-on to his favourite company information provider and go looking for companies with the correct stock-picking criteria he believes will act as an indication of performance, typically the PE ratio and maybe some other things as well. He will invest in companies with the 'correct' ratios and use a buy-and-moderate-hold strategy.

But looking at individual ratios for specific companies is not likely to work — the company books are likely to be cooked for starters, but looking at the ratios for a single company in isolation is unlikely to tell us very much on its own; it is only when you start to look at the data for entire collections of companies that genuine 'indicators' may emerge.

Let's take a real world example; the tanning salon business in the UK.

Britain is rather cold and rainy so we Brits like to 'get some colour' in our cheeks in the belief that it will make use more attractive — and why not. Tanning salons thus exist in all major cities. How many tanning salons a city should have is an interesting question. What could it depend on? Latitude, wealth — many things. If you calculate the number of tanning salons per capita for Britains major cities an interesting pattern emerges — most of the cities lie in a cluster, but the data point for Glasgow lies well outside the rest — it is anomalous. Glasgow has a much higher per capita ratio than anywhere else in Britain (about twice as many salons as you might reasonably expect) — so what is going on? Glasgow lies further north than London or Manchester, but Edinburgh is at the same latitude and Aberdeen is even further north.

We could surmise that Glaswegians simply love sun tans — and this could be the truth of it, or it could be something else. Like most industrial cities Glasgow has both a significant amount of drug usage and an infamous criminal underworld. Drugs cash provides something of a problem to the criminal — depositing 50 grands worth of small denomination bills into your savings account could quickly lead to detention at Her Majesty's pleasure — the money has got to be cleaned or laundered; putting it through the books of a legitimate business which processes a lot of cash is a good way to do it. Something like a pub, betting shop, petrol station or...a tanning salon.

The unusual preponderance of tanning salons in Glasgow could thus reflect the drug underworlds current fashion in money laundering schemes. Now, if you were in the position of being asked to invest in a tanning salon business, you would certainly check out the financials very thoroughly — and it might look very good, and with reputable people behind it; this is the usual way to make such a business decision. But suppose now you were given this extra, global information about such businesses as a whole and their geographical clustering — how would you react? Certainly, however you ultimately decided, this is information you would want to know — and it is information that is not available from doing the usual checks. Anomalies provide us with a justification for being suspicious — but they do not provide any specific information; our statistical analysis does not prove that the 'SunWorld Tan Bar' on your Main Street is run by criminals, but it provides us with stimulus for further investigation.

What applies to little businesses applies even more to big businesses; by looking at the data for many different types of companies in different ways we can hopefully find signals of good and bad performance, of being well-run or of being a house of cards.

What is a Timeline?

A timeline is simply a name for a collection of articles which are ordered by the time that they occurred.

A timeline is often the output from the Webagent module; typically one might use the Webagent to look for some specific events you are interested in, scan a lot of Web pages, extracting the information 'chunks' within and associate with them some date of occurrence. Once created, the timeline may be imported into a stock chart — this lets you then see if the types of events represented in the timeline seem to cause any significant effects on the share price, allowing use a nice visual check.

What is significant about Red Triangles?

Red triangles in the payoff graph for a trade, i.e., the profit-price graph, represent danger — triangles show potentially unlimited losses. Note that the summary performance figures in the trade creator form will give you maximum losses as indicated by the analysis you are using, i.e., the maximum losses that could occur in our probabilistic model, not the maximum theoretical loss.

On the contrary, a red flat horizontal line in the payoff graph represents a capped loss, i.e., no matter how the price moves, your maximum loss is set at a certain level. Note further that it is options trades which allow one to create capped-loss situations — this is a feature which does not naturally exist when you use spread bets, futures, CFDs or ordinary stock trades (don't buy shares, its stupid; say no to stamp duty!)

When you have a 'Red Triangle' situation you might want to arrange a stop-loss with your broker.

Effective Usage of the Advanced Analyzers — a practical approach

Predicting the future is...difficult — what else could it be?! Over the years anyone with half-an-idea or who just 'fancied themselves' has had a crack at predicting the stock market — the potential payoff is just so enormous; but the consensus reached over the years seems to be that you can't really do it (conspicuously successful traders have always existed over the years, but it is hard to prove that their success is due to skill rather than luck, or perhaps some other, illegitimate, means) — its such a hard, hard problem, and even if you could maybe do it, it would only be for a short time and other costs would stop you making a profit.

Real world observations seem inconsistent with the received wisdom — i.e., this understanding has not stopped people trying to make money on the markets:

It has not stopped analysts and pundits encouraging the public to have-a-go, usually under the cloak of managed funds, ISAs and with-profits insurance policies.
Market insiders do not seem to be giving up their jobs to start small farms
Hedge funds still offer top-dollar for PhDs to work on derivatives models
Some people may just be better able to trade than others, and in a systematic way
Some insiders may have access to subtle streams of information which gives them an advantage
Some simpletons who believe that they have some superior skill to the rest of us — they will still be 'game'
The brokerages make money whatever people do — as long as they keep trading
The entirety of society is interlinked with the markets, and our hopes for an enjoyable retirement depend on what our pension funds are doing.
The attitude of the market insiders seems to be — you can't predict, but we want you to think you can.

We should not be surprised at these attitudes, after all, hordes of people play the Lottery all over the world; but playing the Lottery is really, really stupid...it just shows how innumerate the average person is. And Las Vegas is not likely to be turning the lights off anytime soon.

But if you can't predict unless you have privileged information — which is illegal — and the only people guaranteed to make money are the brokerages, then you should stay well out of the stock market — don't buy stocks or funds, keep your money in cash. Volumes should fall to almost zero. Trading on the markets is itself, seemingly irrational. Why should anyone play a game they know they can only lose?

So its all just a 'rigged game' set-up to benefit a cabal of insiders who are conning ordinary people into thinking they can win, when they can't...

This is not far wide of the mark. But the problem with trying to stay out of the market is that...you can't. Even with cash you have inflation to worry about, as well as fluctuations against other currencies. Banks do go bust from time to time; property is not as 'safe as houses;' gold and diamonds — those favourites of the paranoid — are difficult to store physically, i.e., you need major security. The inter-linked nature of the modern world means that everything affects everything else — bond markets, stock markets, currency markets, commodities, property, insurance markets — all have relationships with each other and when highly stressed, as the international financial system often is, disturbances in one part of the globe can get propagated far from their origins; for example, right now you may be about to lose your job because your company is trying to cut costs since it is not doing so well — the reason it is 'not doing so well' is because its derivatives trading desk (which you didn't even know existed) lost a lot of money on a company called Enron... so, in fact, a bunch of Texas energy cowboys thousands of miles away, who you've never met nor are likely to — have put you on the dole queue.

- We are all linked into this thing, whether we like it or not. So we better get educated.

The question still remains, as beguiling as ever — can we predict the market, or more precisely to what extent can we predict, and what level of performance do we need to make money rather than lose it.

Purveyors of stock market software also have a vested interest here...no doubt !

Of course, but I think it is safe to say that current trading systems are little more than folklore and voodoo. We need to look at the problem with a fresh pair of eyes. Share prices seem to have a character all of their own — they don't seem similar to any other kind of timeseries which arise in the real world. They are certainly nothing like the smooth curves we learned of in school ("integrate the equation and get the trajectory of the particle", etc), and yet at the other extreme, are not quite the same as the random walks we learn about in the theory of Brownian motion; they seem to be somewhere between the two.

This is an interesting observation; the two examples given before — of smooth curves and brownian motion, demonstrate the ideas of determinism and randomness.

Determinism is when a system evolves with 100% certainty between definite states; typical situations will involve differential equations and known underlying laws — the basic idea is that there is something we can 'solve' and in doing so predict into the future. So, if a system is deterministic, then we can in principle find its underlying law and so predict. A clockwork universe — one thing follows another, in strict sequence.

Randomness is when a system can evolve to a number of different states at a given time; we cannot — even in principle — make a crisp prediction about the future, all we can do is calculate the probability.

The key point here is to realize that determinism and randomness are not two binary states, but a continuum of possibilities between the two extremes — this is where our advanced analyzers come in; with these we are trying to use any residue of predictive information in the timeseries to sharpen our predictions.

In practice, one should always start with the random walk analyzer — share prices are quite close to simple random walks, to a first approximation. The predictive isocontours will give an indication of the general trend plus the size of spread over which the share price should vary — it is, as far as it goes, quite accurate. You can trade with this — it may in fact be the most sensible thing you can do — but it is fascinating to think we might do a lot better.

From the example you can see how accurate it is — you will also notice how much the swing of the share price in the time period is well described by the contour lines. The goal of using the advanced analyzers is to try and discern more detail than the random walk can; you should realize how potentially lucrative this more detailed information is — we would have precise direction and timing information, giving us entry and exit points. With this more accurate information we can leverage-up our trades with confidence to massively increase our profit levels (if you found something that was a 'sure thing' then you could safely bet your mortgage on it.)

The advanced analyzers are types of neural network — back-propagation, plus the counterpropagation network. These are the basic architectures which are applicable. Each neural net by continued exposure to the data builds an internal representation of the data — this 'internal representation' depends on the architecture of the network, which is why we have a choice — some are better than others. The difficulties with the networks are in identifying their parameters - you do this via back-testing, which is described elsewhere, and with preprocessing the data; training neural nets on noisy data is quite hopeless, so you really need to identify a suitable smooth approximation — you do this using the advanced filtering tool. Note that this preprocessing stage is equivalent to splitting the price timeseries explicitly into a deterministic and a random component via a visual inspection. The other type of advanced analyzer is the bayesian correlator — this is a very direct method which can be used without any preprocessing of the data.

The one thing the advanced methods have in common is that they take lots of computing power — so get a fast machine, or do your simulations overnight, or both. The steps involved are then:

Do a random walk. This is your baseline predictor.
Use backtesting to identify model parameters for any or all of the advanced methods.
Use the advanced analyzers with the parameter sets identified via back-testing to produce your probabilistic prediction.
Look for corroboration between your predictions. Select a 'best-guess' advanced analyzer.
Go to your trade creator module and use the auto search to find trade combinations for both the 'best-guess' advanced analyzer and the random walk analyzer. Try and find a trade combination which gives good payoff characteristics for both analyzers.
Enquire about this trade with your broker, getting in particular, accurate trading costs. Decide whether to make the trade.
Make the trade (or sit it out); your alarms will be set; now go off and get on with life — anything goes wrong you will get a warning.

Wow! That's great! I'm off to make myself a million bucks...I hope my Porsche dealer has one in my colour...

Steady on. Just wait a minute. The above prescription shows you how to sensibly analyze noisy timeseries on their own — but this is not all we have to worry about; we also have the spectre of the trend-breaker news event, you know...those big, totally out-of-the-blue news events which take everyone by surprise — won't these totally screw any fancy-pants analysis we do?!

Potentially, yes...they will. But by their very nature, they do not happen very often — this is both good, as they are unlikely to hurt us, but also bad in that we don't have enough data to model them.

We just have to use our brains here...check out the news files and look at the occurrence of events on the stock chart — you will see what kinds of events have happened in the past and what their consequences were; do you think that there is 'something in the wind?' If you feel uneasy, try using the WebAgent module to do some deeper digging — try and find possible signals of things which affect share prices — takeover bids and mergers, lawsuits — try and get and idea of how the company operates. If anything makes you feel uneasy, or your exposure will be excessive, then you do not trade.

Let's look at an actual example.

Marks and Spencer (MKS:LSE) was a once highly respected UK retailer, but it had been in the doldrums for years, not really going anywhere — even changes of management had only achieved a very brief recovery; the share price was on the slide, had been for years, and looked likely to continue. The timeseries predictors would probably tell you this, albeit with the advanced analyzers giving an indication of the 'wobbles' as well. However, on July 2004, the share price shot up vertically due to the announcement of a takeover bid by the retail entrepreneur Philip Green.

No timeseries method on its could have predicted this move, which would have led to serious consequences if e.g., you had been selling lots of call options on MKS to take advantage of its slide — ouch!

Takeover bids are kept secret until they are announced, so it would seem that we are defenceless against this kind of event. But there is hope, even here; the decision making processes and analyses behind such a move can be surmised by us; Philip Green would decide to make his move based on information available to him, which is probably also available to us; if we can guess at the reasoning process we will not be able to predict 'takeover bid by Philip Green on the X of Y' exactly, but in theory it would be possible to say 'MKS is vulnerable to a takeover bid in the next few months' — and this information can be added to our analysis techniques — if you know what to do.

- But these latter speculations are the subject of a different piece of software, available...when its ready.

What is this News Alarm thing? What do I use it for? Should I create news alarms for every stock?

The news alarm is used to warn you if any significant news events happen concerning a particular stock of interest; you don't need to create one for every stock, only for some company you are interested in, like for example, one for which you have an open trading position.

The news alarm works by examining the historical news files, identifying types of events and then creating a function which can give an indication to the likely share price response. We would recommend you create one for each stock you are trading in.

Doesn't the trade alarm take care of this already?

No. It is different — the trade alarm checks that your trade is 'on track' i.e., that the share price development is sufficiently in accord with our favoured prediction to allow us to have confidence that out trade will be profitable. The way to interpret the trade alarm is that if the price stays within a green region, then we can say — 'the share price is moving pretty much as we expect, therefore our trading position will be safe,' and vice versa — if the price heads into a red region, it is telling us that our predictions are (probably) seriously wrong and that we need to adjust our position.

The news alarm is complementary to the trade alarm — here we are creating a warning system which can identify events which produce share price movements on the extreme tail of the histogram, i.e. trend-breakers — we might expect that these events could catch us out.

Come again! — how does this News Alarm thing work?

The big question for any market-watcher is —

Why do share prices move about as they do?

We want to find 'reasons,' mainly so we can have a stab at constructing a model which might give us decent predictions — and so make money. Reasons mean causes, and we know certain types of news events affect the share price...so that is where we will look. The trouble is, some events seem to have an effect, and some don't; what is more, any effect is not necessarily consistent, and will vary between companies; not all price movements can be explained in this way. Also, more seriously, trying to construct some algorithm which can go from raw news stories to a price movement...is just next-to-impossible.

If something is too hard to do in one giant leap, then it can be useful to break it down into stages. This is where the MFI scale comes in — this allows us to preprocess any news story into 3 simple numbers - this gives us an indication as to its importance, favourability and believability. We can classify a news event against a set of archetypes (these were extracted from the news archives) and assign it an MFI score; the ad hoc nature of this scale is then removed by examining the price response to it and creating an interpolating function via our old friend, the neural net.

The news alarm feature is thus an attempt to rigorously categorize, in a quantitative manner, the boundary between 'stuff that matters' and 'stuff that doesn't.' This feature can be accessed from the news analysis menu on the stock chart; note that at this stage, it is largely experimental — we would suggest that you get to know it well before you create any news alarms.

Be warned — news alarms using the MFI scale will not always work, for the following reasons:

you don't have enough news events; so get some more.
you don't have a good enough sample of news events; so get even more — go back as far as you can.
the measuring parameters of the price response stream have not been chosen well; some variation could solve the problem.
the MFI scale is just too crude to adequately capture the subtle complexities of financial news reporting and the traders response to such; we need a better model — this is already being worked on.
there is something 'odd' about the company and its share price dynamics.

Note about MFI : the I — 'integrity' component of the MFI scale is not used by StockWave™. The original reason for including it was to be able to include as influencing events, things about which we were unsure — rumours, speculations, gossip and so on. Stories which were reported in the major media we would take as being factual and things which originated elsewhere could be given a provisional integrity figure. The main source of such dubious material was expected to be the newsgroups and discussion boards on the major financial Web sites — we even created a new type of data source, a NewGroupSource to accommodate this. But like many good theoretical ideas, it did not survive contact with the real world; after a considerable period examining the quality of postings on newsgroups and the discussion boards we had to conclude that this source of information was practically worthless (as are the majority of the 'pundit pieces' on the so-called respectable Web sites) - the timely, the accurate and the insightful do exist, but only as a trace element — a small fraction of a percentage, if at all. Unfortunately, much of the discussion is dominated by the clueless, the troll, the lazy newbie, the stealth marketeer, the sneaky ramper, the rampaging egomaniac and a multitude of other bizarre creatures. You can do without this stuff — it merely lowers the signal to noise ratio. The clock cycles we would waste trying to filter out the nuggets from this raging cacophony would be much better spent elsewhere.

How do I use the News Scatter Chart?

The news scatter chart is designed to help us understand the relation between news events and the share price movements.

To start with, load all news articles into the news viewer — if necessary adjust/edit these before you do any further analysis. Now open the news scatter chart. Once open, choose the parameters of the price response stream — these parameters are measuring intervals for calculating the change in share price and the change in the gradient of the trend. The change in share price and share price gradient will be calculated for every piece of news in our files and plotted on the left hand chart.

The chart will probably show a cloud of points, clustered around the origin, with a smaller number of points towards the edges of the chart. The axes of the chart represent percentage changes in the share price and in share price gradient. The cloud of points at the origin — which should be the overwhelming majority — are the types of news events which don't matter much; they will be lost among the 'background noise' of the share price movement.

The interesting points will be towards the edges of the chart — these points show us events which have produced more extreme changes in the share price. (NB — changing the parameters of the response stream will move the cloud around somewhat — try and find a configuration which is stable and hence, we would hope, representative.) By moving the mouse over the chart we can inspect what these events actually were. What we are trying to find out is:

What kinds of event cause big changes? Are these of similar type? Is there any other connection between them?
What kinds of event don't seem to matter? Can we ignore these safely?

This lets us identify what we should interested in.

The left hand plot shows us where the news events are in relation to our MFI scale; this scale is a human-constructed measure which represents our ordinary expectations about how important or not an event should be. As we move the mouse around each panel, the corresponding point on the other panel will be highlighted.

The 'unimportant' cluster on the right should correspond to the lower plots on the left; the 'extreme values' should be higher up. If this general relation holds true then we can do something very useful — we can create a news alarm; this is an object which can make a precise quantitative judgement about the importance of a news event. The main usage of a news alarm is for monitoring an open trade — this way if something important happens, the system can warn the user without waiting for the share price to move. Which is useful.

If you have problems with this procedure, remember that garbage in, garbage out — applies as always (in particular, if the time of occurrence of the news events is incorrect, you will get complete rubbish). You should also want to have as large a sample of news events as possible — too few events means that we can infer little. Or if you feel the MFI scale is amiss in its classifications, then you can alter it to your own taste, even to the point of manually editing it. If you still find the found relationships to be 'odd' then you should do more investigation about the company — you could uncover something very interesting, and remember — a superior knowledge creates the opportunity for profit. In this case 'more investigation' would usually mean using the Web agent to dig for more, different, kinds of information than get reported in the news channels.

Now, you may feel the news scatter chart simply confirms what you already know, e.g., profits warnings and CEO resignations are important events — any market watcher knows this, but the scatter chart allows us to quantify our 'conventional wisdom' and furthermore identify where our conventional wisdom is wrong.

Data preparation in detail

Preprocessing is even more important than usual for using the news scatter chart and for creating news alarms. Open the stock chart and load all news articles. Now from within the news viewer:

Remove all duplicate articles.
Remove all 'crap' — i.e., articles that don't actually say anything, are not about the stock in question, or are ambiguous, or just plain confused.
Check the datetime values for these are correct. The datetime is a universal date and time coordinate — it is the number of days since 1900, and the fractional part is the part of the day. The datetime is what lets us associate the news event with the price changes at the same time.
Manually edit any article which needs it.
Remove/Edit multiple reports of the same event into one or two good exemplars.
Clean the price data

When you open up the news scatter chart, build a response stream and then, if you need to, re-assign MFI scores. There will now be two data clouds on each panel, as described above. The most important points on each panel will be the outliers, so inspect these manually to make sure their assignments make sense. Once you have a good data set, you can make yourself a trend-breaker calibration object for use in a news alarm.

Why do I have to fiddle about doing all this? Shouldn't it all be automatic?!

Getting computers to understand the meaning of textual information, i.e., natural language, is really difficult. Really difficult. The problem is a bit easier when dealing with these financial news 'chunks' but still the problems remain, mainly ambiguity. Meaning, it must be said, comes from context, which is a global feature — you need to look at everything, altogether — which is computationally expensive. Improving the MFI scale and its assignment algorithm is a longer term goal, but for now you will have to accept that sometimes the assignment will be wrong, or have the wrong sense — note however, that these cases will be easy to spot and easy to fix for the user.

Ambiguity can be lessened by using shorter input texts, but if too short, will be meaningless. The current MFI scale is based upon keywords and keyphrases — these are chosen to be as unambiguous as possible; regular expressions may also be possibly used (if you don't know what a regular expression is, then don't worry about it). User experimentation with the MFI scale is not discouraged, indeed we will be very interested to see any improvements you have found.

News event analysis is, admittedly, an experimental feature in StockWave™, but we hope our users appreciate our reasons for including it. The quantifiable response of the share price to news events is also a crucial component for our further developments of StockWave™, one of which is to be an expert system, using both fuzzy logic and NLP — the goal of this is to build an accurate statistical model of trend-breakers, i.e., extreme news events.

Why do we have a timeline facility as well as a news viewer? Isn't it the same thing?

News articles are textual information chunks which have a direct relevance to our share price information — these are, typically, headlines and article summaries which are collected on regular intervals from the more well-known news sources.

A timeline is something rather looser and quite different; a timeline is created as the result of a WebAgent search. The WebAgent is a program which we use to 'dig deep' for interesting information — it is an investigative tool which uses the algorithmic equivalents of guesswork and intuition to find nuggets of information which would not necessarily be reported on the news — which is purely fixated on current 'events' (chosen by an editor, which are in his eyes, 'relevant') rather than any deeper context or analysis. Not everything you need to know is on the newswire — and even if it is, by the time you read it, it might be much too late. As any investigative reporter or historian knows, obscure documents can hide dynamite.

In looking at the relation between news events and share price moves we may find much that we understand and much that we don't; of particular note are large share price moves which are presaged by little more than an obscure press statement (perhaps released late in the day just before trading is about to close) or even worse, nothing at all.

In nature, things can happen at random (e.g., the decay of a radioactive nucleus), and while in the human world the chaotic actions of many individuals can create 'randomness' (and at times, order), most specific events in the realm of society, happen by deliberate action or design. Remember also that while coincidences are possible, they are by their very nature unlikely; what is more, chains of coincidences, especially those which consistently favour one party over another — are astronomically unlikely. In practice, we should take the view that — things happen for a reason and coincidences do not happen.

What you are trying to do with the deep searching and creation of timelines is to look into the sub-structure of the company's internal dynamics. Take for example, directors' dealings — these are always of great interest to the ordinary investor. If you managed to assemble a timeline of directors dealings and found that there were clusters of activity prior to extended periods of share price growth or decay ( and the directors were mostly going in the right direction as well), it would be reasonable to infer that there was manipulation occurring — and suppose further still, within these clusters the largest trades were made by specific individuals — what should you do? Call the cops? Inform the SEC, the FSA and the SFO (the FBI, CIA, NSA...)?!

Absolutely not. The authorities should be monitoring suspicious activity anyway as part of their duties — its nothing to do with you. If the crooks have been clever they will get away and if not they will be caught.

Furthermore, by your cleverness and diligence you have uncovered a trading signal — so use it; next time the boardroom crook(s) start loading up or selling off — get yourself a piece of their action. Make some money.

You can import any timeline into your stock chart to look for a connection with your share price; you won't always find anything, but you might find something. Knowledge means profits — so go to it!

If this program is so good, then why not use it to make money for yourself — why are you selling it?

We have limited resources and could not do more than 'dabble' in our trading. Sticking to our 'core competencies,' i.e., software, is likely to be the best strategy for us at present. If we did ever get serious on the trading side, then we would be keeping our software purely for ourselves — we would get a stodgy new name, a fancy address with a brass nameplate and become very secretive, i.e., we would become a hedge fund.

But if your software really is any good, why sell it for only £849 — all your competitors sell for many hundred or even thousands of dollars?

We could sell StockWave™ for...let's just say a lot more than we actually do — but we don't, for reasons which are partly good business and partly philanthropic; although niche software can be very profitable, for a time at least (eventually someone will come in and seriously undercut you), the real money is to be made in the mass market — we want a large user base. We also have a secondary mission, as well as making money for ourselves, to educate the public about the markets and financial matters; the markets are central to the workings of the world, but we can't trust our money to the so-called experts — everybody has to get involved on their own behalf, and what is more, if everyone did exactly this, then by the Invisible Hand of Adam Smith, the markets, and hence the world, will work better than it does today.

Dealing with the share price and news together is confusing — when does news 'matter' and when does it not; and why is this?

In a nutshell most news 'doesn't matter,' but when it does, it really, really does, When it 'doesnt matter,' it is for the following reasons:

Too small, too unimportant to make any difference
Important, but widely anticipated, therefore already factored into the share price
Important, but unclear — half the market thinks its good, half thinks its bad; the net effect is zero

What counts when doing news analysis is therefore:

Important, but unexpected, events and with a definite positive or negative sense

The tools in StockWave™ will help you make these distinctions.

Why isn't the user interface "different"?

In what way would you like it different?

more buttons?
less buttons?
bigger buttons?
smaller buttons?
more popup menus?
less popup menus?
MDI - everything contained in the main form?
docking?
customisable toolbars?
more flashing lights?
less flashing lights?

... honestly people, if you want (QuoteTracker, eSignal, MetaStock ... etc) then just buy (QuoteTracker, eSignal, MetaStock ...)

Why can't I find <feature X>?

The interface is designed to lead the user through certain stages before he can do anything advanced or dangerous.

Why don't you support <datafeed Y>?

We probably can, but not out of the box; contact us, show us what you've got and we will try and write a translation file for the datacapture configuration.

I think StockWave Advanced is too expensive!?

It is still cheaper than anything -remotely- comparable; especially if you are running a single processor system.

Why is <feature Z> not it in the Basic Edition?

Because it's either in the Advanced Edition, where it is more appropriately sited, or it's not in either edition because we don't think it's much use.

Why are you so anti-Technical Analysis / Elliott Wave Theory?

it looks like science but it's not
it looks like it should give you answers, but tells you nothing
it looks like it should work, but doesn't really

I think it's safe to say a lot of investors have been lured by the promises of these theories, but found themselves made increasingly disappointed and desperate by their inconsistent results; and by the time you've got to this stage of realisation you are probably out of pocket by several thousand bucks - this kind of software is not cheap.

But what if we could use a genetic algorithm to optimise the parameters of a neural network that we have trained to generate automatic Elliott Wave counts!!

Please go away.

Isn't your stuff just a 'fancier' version of Technical Analysis?

You could look at it that way, if you insist on using the correct definition of the term, however I prefer to define terms by their common usage, in which case TA is a collection of techniques (- moving averages, oscillators, RSI, etc) which StockWave is not.

If you are so against TA, why do you have all these 'useless' indicators on your 'Classical Technical Analysis' chart?

A compromise - we have no wish to alienate potential customers; but we would hope that over time they would come to prefer our techniques.

Why can't I write my own "trading systems" in StockWave?

There are two reasons, the first one is that, you can't access the API right now, and the second one is that it wouldn't do you any good anyway. Let me explain - once you have become disillusioned with what (Classical) TA has to offer, many will persist in its goals, but shift their beliefs and expectations to thinking that some combination of technical indicators and trading rules WILL work - they only have to find it, this -magic- formula. They then go on to learn some kind of scripting language (- EasyLanguage, Visual Basic) with which they attempt to hand-craft some kind of 'trading system' which they will then back-test against historical data (- and will probably upgrade to the 'professional' versions of their favoured charting software in order to do so; more money down the drain.)

Several drawbacks to this approach come to mind -

indicators, useless on their own, might still be useless in any combination you can make of them
even if it can work in theory, the magic formula / trading system may be much too complicated for a human being to identify
backtesting can be very misleading; markets change over time - so all you may be doing is developing a system which would have made you a barrowload of cash ... in 1989 or 1994, ... but won't do it now.

What might work, (if the approach is theoretically valid), would be the use of a general purpose search technique like genetic algorithms or genetic programming to 'evolve' trading systems; this would be able to handle even extremely complicated rule-based systems. But then this still doesn't give you the probabilistic approach - if you are getting a signal, you still need to know how reliable it is, as this impacts upon your risk and money-management strategy.

Anything can happen; all we can know are the true odds - i.e. probability.

Why can't I do (classical chart) pattern recognition?

This is another variation of the 'trading systems' idea - one looks for patterns in charts (- the more common ones have well-known names, double top, head-and-shoulders, ) and associates these as being indicative of certain market moves.

But consider -

do you have enough samples to identify the actual behaviour
does it work for everything or only certain securities or at certain times intervals
'recognition' of chart patterns is itself subjective

In any case, our algorithms are already doing (a proper form of) "pattern recognition" anyway.

Why can't I access the API?

A scripting language offering full access to all StockWave algorithms and analysis functions is scheduled, however it is at a lower priority than other development tasks, i.e. it is the last thing we shall be adding.

How do I get my setups, entry and exit points, from StockWave?

You could try to use StockWave in this way, but it's not the intended method we would suggest; like eating soup with a fork, it is possible, but not really what you should be doing ...

The principal object of interest, the main trading tool, in StockWave is the probability heatmap; once you have one of these you then use the automatic TradeSearcher to find the best trade for you (- based on the kinds of trades you can make.) Once you make a trade, the alarms are set which continuously monitor the progress of the trade.

- isn't this easier than staring at a chart all day?!

(N.B If you are serious about trading then you will need Advanced Edition.)

Why isn't it faster!?

StockWave is written in C++ and is multi-threaded; it's about as fast as it can be - you could tweak 10-15% more from it (- I suppose), but the real issue is that Monte Carlo simulation - the only general purpose technique for calculating probabilities - needs lots of runs. With averagely-powerful current systems you can do simple random walks in minutes, and the more advanced techniques in a few hours (- we would suggest you do your analyses overnight, and limit trading hours to datacapture and portfolio monitoring.) Multiple cores will help.

Would you like to be partners with us?

Why yes, of course - but only if you have something we can use or is of interest to us; for example -

money for expansion, or
other resources, principally access to powerful hardware
other expertise; marketing expertise, business connections, or
some cool technology AT LEAST at working prototype level - either unique, or something we've thought of but haven't done yet.

Would you like to help me build my system?

Thanks for the offer but we're kinda busy right now.

But I've got this really great idea!

You go for it, buddy - here is what you should do -

first of all don't tell me what it is and don't tell anyone else either otherwise you won't be able to patent it
write some proof of concept software to implement it, a simple spreadsheet written in Visual Basic should do it
open an account and trade it yourself OR get a professional programmer to put a nice GUI around your algorithms and then sell the software to others OR patent it and just license the algorithms to the software companies
PROFIT!!!!!!!!!!!

But my system will produce <some ridiculously large figure> annual return!?

Then why are you talking to me? You don't need me! You don't needanyone ...

Where are the "numbers" for your approach?

Suppose in the course of our shakedown testing and debugging we did a small paper-trading test over two months, scored 13-3 win-loss rate (without even using the advanced techniques) and made 3K "profit" on trading capital of 10K (- that's ~500% annualised return)... should we tell the world about this?

Absolutely not! - the above is statistically insignificant - just anecdotal nonsense.

Here is what you really need to do, if you are going to present the "numbers" on your website -

make extensive parametric studies of the analysis techniques to identify their optimal operating parameters
trade the techniques across a wide range of securities, using corroborated analyses and probabilistically-optimised trades
do steps 1 and 2 over an extended time period
get the results notarized otherwise no one will believe you
now post the results

Setting up a study like this requires resources we don't have - so as far as "the numbers" go, we must remain silent ... for now.

If you're so good, mr genius why aren't you a gazillionaire hedge fund manager and keeping your software super-secret!?

Software development is expensive, time-consuming and error-prone - notoriously so, in fact; on any project that you undertake you have to bear in mind that you stand a good chance of outright failure; if you are trying to do innovative work, i.e. that no one else seems to be doing, the risks multiply ten-fold. Furthermore if you are in hock to the banks, vulture capitalists or have other investors pressing you for deadlines, you have extra pressures to deal with it - in particular, the threat of having the rug pulled out from under you if you don't "perform"; StockWave Software is employee-owned and entirely self-financed, this means we can do what we want technically - the price we pay for this is that we necessarily run on a very tight budget.

Given the situation, for us to try to trade professionally on top of our software development, would be near impossible given the substantial setup costs involved; you need hardware, datafeeds, infrastructure, trading accounts and enough cash to invest in the first place; there is also the costs, mostly legal bills, associated with financial regulation, which you can only (partially) avoid by setting up offshore, but that costs money too.

If your stuff works and we all start using it, then it will stop working, so it can't really work, right?

No.

If we were selling a traditional trading system which had some success in both backtesting and live trading as well, then sold it on to others, the inefficiencies which it was exploiting would be "arb-ed" out of the market - probably very quickly. The act of distributing the system would have the effect of adding information to it and allowing it, or rather compelling it, to adapt.

A trading system is an algorithm, our system is a meta-algorithm, an optimisation over a space of algorithms, these algorithms being embodied in our toolkit of analysis techniques and fusion process. You could think of it as a tool for finding trading systems, however there are important differences between our techniques and one which, e.g. uses genetic algorithms, or genetic programming to perform an optimisation of (hyper-) parameters - our choice of algorithmic techniques provides, we believe, a superior decomposition of the search space, and a better starting point for the meta-search process.

(- that last sentence may sound like technical gobbledygook, but I can assure you it does make sense, there's just no simple way to put it across.)

And if it really did work, you'd be crazy to sell it, because you could simply make money for yourself, but if you did sell it it would stop working and then you wouldn't make any money?

No. And No.

We don't have the resources to trade professionally. (Yet.)

And see above.

Would these "meta-technique" strategies scale?

The assumption is that any individual trades will have infinitesimal market impact - we assume, and recommend that you only trade markets that are "liquid" relative to the size of what you are trading. I would not expect any problem even for wealthy private investors - institutions, that's a different matter.

Suppose you were a hedge fund, having good results with StockWave, then you decide to get greedy, and to leverage up the bet to make more profits - naively upping your usual stake by a factor of 50 might cause problems - but this is well known anyway, and with well known techniques for dealing with it, e.g. there are specialists who will break up large trades for you, or you can just smurf it yourself, chunk it up and randomise the amounts, pushing it through different accounts, there are also "dark pools".

Crossing over to the dark side, I am sure the big boys will/have/are try using feedback and other information to exploit technical deficiencies in market operations and liquidity restrictions - but this is very close to illegal market manipulaton, and is an entirely different ballgame.

From a theoretical perspective, optimising a trade/prediction action taking into account your own market impact would be a very difficult calculation, an enormous computation; having said that, it would be a very profitable thing to have - you would be able to hit your market opponents multiple times, watching them lurch from one trap into another.

What is Python?

A very nice programming language. And it's free.

Why do I have to learn this? Why should I learn Python?

So you can program StockWave into doing exactly what you want it to do - now there's a lot in the help files telling you what to do, what you should do, what you should be trying to do, what you should be thinking about, what the best thing is ... and you should heed it! But also, once heeded, it's perfectly reasonable to "go your own way".

StockWave now has an API and command language which allows you to interface with the Python language - with Python you can write whatever you want and call all the great built-in algorithmics of StockWave.

And, once you know it, you can do a lot of other stuff as well - it's a fully featured "real" language. Hell, your new found "mad h4xor skillz" may even get you a job!

But why are there two types of scripting?

The batch system allows you to enter single commands, or lists of commands - the main use is to automate a lot of fairly boring data maintenance jobs - it is not a fully featured language with loops, branching, recursion and subroutines (- not "Turing complete")

Simple scripts are in the \scripts folder and have a ".script" extensions - there's a couple of examples to help you get started. More complicated programs are written in python - they are in \scripts\python and they have the ".py" exetension.

So I can now write my trading system - how do I do this?

Yes. Just write it ... in Python and do an "import stockwave" to use any of our groovy shit.

Are those Python guys putting you up to this?!

Yup. Me and Gerda vonPossum were in Skull and Bones together. The conspiracy is deep.

Very deep.

The Black Pope and the Chabad Lubavitch are in on it. But I've already said too much.

Why not something else?

Look, there's loads of programming languages out there, most of them absolutely terrible, and the ones that are any good, are mostly only good for one thing; there is a tension in programming language design between easy-to-learn, easy-to-use, "power", and widespread utility - it's a trade-off. Having done an in-depth survey I concluded that Python, really hits the "sweetspot" - you can cover shell scripting, web development, real programming, "objects" and even lower-level stuff without too much of a stretch.

Most of you guys have no interest in being professional software engineers ( - and I hear you brother, I really do!) - you just want to get the job done with as little hassle as possible - learning this will be a good investment.

But you have to put some time in - go to www.python.org, download their latest and install it; make sure the environment variables are set; have a read at the manual, it's got a tutorial; now download some example programs to get the feel of it, and later play around with the StockWave samples.

(I infer from your persistence that you have a whole load of stuff in other languages - well, since Python is "loosely-typed" you can probably just cut-and-paste your old formulae into a Python script, fix up the syntax a little, and it will probably work; that's the thing about Python - it's almost like writing runnable programs in "pseudo-code".)

Remember that proprietary scripting languages fix you into their applications, their platform, their way of doing things; this is called "lock-in"; C# is OK to write and has excellent free tools, but it locks you into .NET and as for (Excel, or indeed any kind of) Visual Basic, that's just a disease - and spreadsheets just rot your brain.

Who's looking at this website anyway!

Despite being geared towards the average retail investor, i.e. 'joe-public' (- more or less), a significant amount of our traffic comes from the corporate / academic / banking nexus; one of the great features of Google Analytics is that it lets you identify these types of visitor - at the risk of name-dropping, here are some of the more interesting -

Financial Institutions

European Central Bank
Abbey National Treasury Services
Asset Management Advisors (Greenwich)
Assurances Generales de France
Barclays Capital
Baring Asset Management
Bayerische Landesbank
Bear Stearns Security Corporation
BNP Paribas
Canaccord Capital
Cantor Fitzerald & Co
Citicorp Global Information Network
Credit Suisse Group
Datastream
Deutsche Bank
Deutsche Boerse AG
Egerton Capital Ltd
Fidelity Investments
Financial Times Information
Forex Capital
Fortis Bank
Goldman Sachs
Grand Investment (Securities) Limited
Grupo Financiero GBM-Atlantico
Halifax & Bank of Scotland
HSBC Bank plc
Infocredit D&B(S)Pte Ltd
Information Risk Managment
InfoWeb(Fujitsu Ltd.)
ING Bank
Interealty Corporation
Investec Bank Ltd
JP Morgan
Kantor Management Consultants S.A.
Knight Capital Group
Kotak Securities Limited
KPMG Peat Marwick
Legal & General Investment Managers
Legg Mason
Lehman Brothers
Macquarie Bank
Majestic Research
Merrill Lynch and Company
Nat West Bank Group
Northern Trust Company
Parex Bank Corporation
Philadelphia Stock Exchange
Pioneer Invest
PricewaterhouseCoopers GTS UK
Royal Bank Of Scotland
S3 Management Services Ltd
SAC Capital
Salomon
Softbank BB Corp
The Chase Manhattan Bank
Thomson Financial
Tinker Federal Credit Union
Town North National Bank
UBS
Videsh Sanchar Nigam Ltd
Wachovia Operational Services Corporation
Washington Mutual
Yorkshire Building Society

Government Agencies, Laboratories, Think-Tanks

Commission Europeenne
Argonne National Laboratory
Cambridge Consultants Limited
ESA
HQ USAFE / SCNO
LR Avionics Technologies LTD.
Lexis-Nexis
Los Alamos National Laboratory
Mimos Berhad
National Aeronautics and Space Association
National Defence College
Naval Ocean Systems Center
Software Technology Parks of India
U.S. Environmental Protection Agency
UK Defence Science and Technology Laboratory
USAMITC
Washington State Dept. of Trade

Corporations

Allianz Versicherungs-AG
Alstom ( Schweiz ) AG
AMOCO Corporation
Analog Devices
CEDEX
China Oil Co.
Cincinnati Gas & Electric Co.
Cisco Systems
CNC GROUP Beijing
Exxon Mobil Corporation
Federal Aviation Administration
Ford Motor Company
Google
Hewlett Packard
Hughes Network Systems
Intel Corporation
Microsoft
Northrop Grumman
Pilkington Plc
Raytheon Company
RCN Corporation
Saudi Arabian Oil Company
Sony Network Taiwan Limited
Talisman Energy (UK) Ltd
Texas Instruments
Toshiba
Tyco Electronics
United Energy
Westinghouse Electric Company

Academic, Educational

Bournemouth University
Columbia University
Cornell University
Direction generale de l'informatique
Ecole Nationale Superieure des Telecommunications
Gallaudet University
Georgia Department of Education
Glasgow Caledonian University
Hofstra University
Indian Institute of Management Khozikode
Indian Institute of Technology
Institut National de Statistiques
Institute of Mathematics and Computer Science
Iowa State University
Leeds University
Leibniz-Rechenzentrum der Bayerischen Akademie
London School of Economics and Political Science
Manchester Metropolitan University
Massachusetts Institute of Technology
Max-Planck-Institute of Micro Structure Physics
Max-Planck-Intitut fuer Biochemie
Medical Science University of Shahid Beheshti
Northwestern University
Oxford University
Prince of Songkla University
Purdue University
Queen's University
Rice University
Singapore Management University
Southern Methodist University
Stanford University Network
State of Tennessee Department ofEducation
Texas A&M University
The University of Winchester
Universitaet Graz
Universitaet Hohenheim
Universitaire Instelling Antwerpen
University of Central England in Birmingham
University of Florida
University of Leicester
University of Luton
University of Michigan
University of Minnesota
University of Natal
University of New Mexico
University of Newcastle upon Tyne
University of North Carolina
University of Rochester
University of Saskatchewan
University of Southampton
University of Texas at Dallas
University of Wisconsin - Milwaukee
UPC Sweden
Uppsala University
Vaal University of Technology
Victoria University
Wilfrid Laurier University

End Note

General speaking, StockWave might look superficially similar to any number of charting packages - but it is radically different under the surface, a whole different philosophy; just try to keep an open mind.

Why is the new price for new StockWave 849GBP - we thought it would be less?!

The price for the new product is in a way, arbitrary - we could choose any number, but 849GBP was chosen as it properly selects our target audience.

It was our original intent to make StockWave a mass market application, but on reflection this laudable, idealistic notion is not practical. Our intended audience is the serious, experienced, private investor - someone who has the cash reserves to trade for real, and the money necessary to equip himself with the high quality computers and displays needed to trade. StockWave really needs powerful hardware, while it will run on modest systems, the experience will not be enjoyable. To the serious investor, 849GBP is not a big deal, and even compares favourably with other software.

Another important trait of the serious, experienced investor is his expectations - he knows how difficult trading is and what constitutes a viable trading edge; on the other hand, the average person may well be holding some simplistic ideas and unrealistic notions about trading the markets - he will be looking for a "one-click magic box" which produces profits on demand. The "edge" possible with StockWave is not that of Aladdin's Lamp, but more of that of the card-counter in the casino. Unrealistic expectations can never be satisfied - we don't want disappointed or disgruntled customers - those we can do without.

There is an old saying, sometimes attributed to Einstein : "make things as simple as possible, but no simpler" - trading the market for real money requires discipline and knowledge; there are a great many "little details" which can cost you a fortune and a lot of simplistic bad advice that people all too readily fall for.

I'm still confused about it all — give it to me in a nutshell!

Try this mental checklist...

Market conditions

You can make money from any type of market move, non-move or condition
Information means opportunity for profit
All types of information are important
You can lose heavily in the markets, so risk exposure needs to be controlled

Trading

Spread bets are for short-term action
CFDs are for medium term trading; the cost effective alternative to share trading
Stock trades are for long term, or ownership/control if you are a big player
Buying stocks is for dummies, unless youre a big player
Options are for smart guys, especially when using strategies
Good brokerage is important — tight spreads, low fees, low margin and a full range of trades available — shop around
Keep your emotions out of your trading

Overall strategy

Play the odds — the probabilities — when its in your favour
When the odds are not in your favour — do not trade

This software

Stockwave™ calculates probabilities
Stockwave™ uses a range of techniques no other program uses
Stockwave™ is based on science, not pseudoscience

In general

Everybody needs to understand the workings of the markets and the financial world, whether it be for profit or self-protection
The Markets are the World; I don't necessarily approve of this situation as I am not a market evangelist; this is just a statement of fact. Deal with it

On advice

Financial journalism is full of people selling something — it's mostly worthless.
Really high quality information is expensive and difficult-to-get — they dont want your dollar, its not for the likes of you; you won't find it on the usual media.
You need to do your own analysis; stop looking for tips.

Program Documentation and Help Files

Frequently Asked Questions