Given a probability space , and a non-negative random variable Z satisfying (why 1?). We then defined a new probability measure Q by the formula, for all .

Any random variable X, a measurable process adapted to the natural filtration of the , now has two expectations, one under the original probability measure P, which denoted as , and the other under the new probability measure Q, denoted as . They are related to each other by the formula

If , then P and Q agree on the null sets. We say Z is the Radon-Nikodym derivatives of Q with respect to P, and we write . To remove the mean, μ, of a Brownian motion, we define

Then under the probability measure Q, the random variable Y = X + μ is standard normal. In particular, (so what?).

This text made no sense to me when I first read it in school. It was very frustrated that the text was filled with unfamiliar terms like probability space and adaptation, and scary symbols like integration and . (I knew what meant when y was a function and x a variable. But what on earth were dQ over dP?)

Now after I have become a professor to teach students in finance or financial math, I would get rid of all the jargon and rigorousness. I would focus on the intuition rather than the math details (traders are not mathematicians). Here is my laymen version.

Given a probability measure P. A probability measure is just a function that assigns numbers to a random variable, e.g., 0.5 to head and 0.5 to tail for a fair coin. There could be another measure Q that assigns different numbers to the head and tail, say, 0.6 and 0.4 (an unfair coin)! Assume P and Q are equivalent, meaning that they agree on what events are possible (positive probabilities) and what events have 0 probability. Is there a relation between P and Q? It turns out to be a resounding yes!

Let’s define . Z here is a function as P and Q are just functions. Z is evaluated to be 0.6/0.5 and 0.4/0.5. Then we have

This is intuitively true when doing some symbol cancellation. Forget about the proof even though it is quite easy like 2 lines. We traders don’t care about proof. Therefore, the distribution of X under Q is (by plugging in the indicator function in the last equation):

Moreover, setting X = 1, we have (Z here is a random variable):

These results hold in general, especially for the Gaussian random variable and hence Brownian motion. Suppose we have a random (i.e., stochastic) process generated by (adapted to) a Brownian motion and it has a drift μ under a probability measure P. We can find an equivalent measure Q so that under Q, this random process has a 0 drift. Wiki has a picture that shows the same random process under the two different measures: each of the 30 paths in the picture has a different probability under P and Q.

The change of measure, Z, is a function of the original drift (as would be guessed) and is given by:

For a 0 drift process, hence no increment, the expectation of the future value of the process is the same as the current value (a laymen way of saying that the process is a martingale.) Therefore, with the ability to remove the drift of any random process (by finding a suitable Q using the Z formula), we are ready to do options pricing.

Now, if you understand my presentation and go back to the textbook version, you should have a much better understanding and easier read, I hope.

References:

• Stochastic Calculus for Finance II: Continuous-Time Models, Steven Shreve
  
• Introduction To Stochastic Calculus With Applications (3rd Edition), Fima C. Klebaner

efficient frontier

Harry Markowitz won the Nobel prize for his work in mean-variance (MV) portfolio optimization in 1950s. The theory is widely regarded as fundamental in financial economics. It says, given a target return of a portfolio of m assets, the optimal (in terms of information ratio) weighting is given by

where is the expected future returns, and is the expected covariance matrix of future returns. This problem is readily solved by quadratic programming.

Nonetheless, the assumption that and are known in advance is very dubious. This has been referred to as the “Markowitz optimization engima”. The attempts made so far are to better forecast these estimators, namely and , as accurately as possible. The maximum likelihood estimate (MLE) from the training sample is an example. It turns out, however, that MLE performs poorly because the estimators are quite different from the realized values [Michaud, 1989]. Since then, three approaches have been proposed to address the difficulty. The first approach uses a multi-factor model to reduce the dimensionality in estimating  [Fan, Fan and Lv, 2008]. The second approach uses Bayes or other shrinkage estimates of [Ledoit and Wolf, 2004]. Both approaches attempt to use improved estimates of  for the plug-in efficient frontier. They have also been modified to provide better estimates of , for example, in the quasi-Bayesian approach of [Black and Litterman, 1990]. The third approach uses bootstrapping to correct for the bias of  as an estimate of

The innovation in [Lai, Xing and Chen, 2010] is to move away from the traditional approach of finding better estimators. They instead assume inherent uncertainty, hence risk, in these estimators. In other words, because the mean and covariance matrix of future returns are estimated from past returns, the uncertainties of these estimators should be incorporated into the portfolio risk. The authors formulate a stochastic optimization framework which solves a more fundamental problem. Given the past returns ,

maximize

where is now regarded as a random variable conditional on the past returns, and is a risk-aversion index (investor’s preference).

This stochastic optimization problem is not standard due to the term in (). To tackle this problem, the authors solve instead an equivalent stochastic optimization problem:

where

and

Next, define the function

where and are the first and second moments of returns. The problem now becomes maximizing which can be achieved by any univariate maximization algorithm such as Brent’s method (BrentMinimizer in SuanShu can be used to minimize the negative of this function).

The remaining task is to estimate and to plug into . can be estimated by, for example, bootstrapping, a multi-factor model, a GARCH model, etc. Similarly, can be estimated by first estimating the sample covariance matrix . As may be unstable when the sample size is small, our implementation uses a covariance-shrinkage method described in [Ledoit and Wolf 2004].

Putting all these steps together, the algorithm to compute the optimal weighting looks like this:

1. Estimate and from past returns
2. Estimate
3. Construct the univariate real function
4. Maximize over
5. Compute the optimal weighting using the optimal

We have implemented this algorithm in AlgoQuant as a component/module/signal. Using our component based programming paradigm, it is easy to modify the steps in the implementation without a lot of changes to the code. For instance, we can pass in the constructor of Lai2010NPEBModel a GridSearchCetaMaximizer object instead of a BrentCetaMaximizer object to change the algorithm to maximize .

In addition, we have coded up a trading strategy using Lai’s portfolio optimization algorithm as a signal. This strategy re-balances periodically a basket of assets according to the optimal weighting computed by the model. Specifically, onDepthUpdate(), collects the returns series using the component ReturnSeries with its update() method. Then, onTimerUpdate() is triggered periodically (say, every 3 months) to call the model (passing in the returns matrix) and to send orders to re-balance the assets accordingly. We backtest this trading strategy using the Simulator object together with the historical data from Yahoo! Finance (or other data sources).

An online demonstration can be found here.

References

• Black, F. and Litterman, R. (1990). Asset Allocation: Combining Investor Views with Market Equilibrium. Goldman, Sachs & Co., New York.
• Fan, J, Fan, Y and Lv, J. (2008). High dimensional covariance matrix estimation using a factor model. J. Econometrics 147 186-197.
• Lai, T. L., Xing, H. and Chen Z. (2010). Mean-variance portfolio optimization when means and covariances are unknown. Annals of Applied Statistics 2011, Vol. 5, No. 2A, p. 798-823.
• Ledoit, P. and Wolf, M. (2004). Honey, I shrunk the sample covariance matrix. J. Portfolio Management 30 110–119.
• Michaud, R. O. (1989). Efficient Asset Management. Harvard Business School Press, Boston.

There are a tutorial and examples that show you how to build a SuanShu application in Visual Studio. One major advantage of using SuanShu.net over the Java version is that it integrates seamlessly with Microsoft Excel. By incorporating SuanShu library in your spreadsheet, you literally have access to hundreds of numerical algorithms when manipulating and analyzing your data, significantly enhancing Excel’s productivity.

We hope that you enjoy using SuanShu.net in your work. If you have any interesting story, comments or feedback, we’d love to hear from you.

Starting downloading SuanShu.net now!

After you download the AlgoQuant package, you will find that there is a folder lib/blpapi-dummy which contains the Bloomberg API jar file (blpapi3.jar). This file is a dummy for AlgoQuant to compile. To use the Bloomberg Java API, you need to replace the file with the real API jar file. If your machine has been deployed the Bloomberg API, you can find the real jar file in your hard drive, for example,

C:\blp\API\APIv3\JavaAPI\v3.4.3.2\lib\blpapi3.jar

Note that the version number of the API may be automatically upgraded by Bloomberg.

The code for connecting, downloading and saving the tickdata is located in the package com.numericalmethod.algoquant.data.historicaldata.bloomberg.  You can find in the package a simple demo application “TickDataDownloadApp“, which accepts command-line arguments and downloads tickdata of a given period for a given security:

Usage: TickDataDownloadApp <security> <startDate (yyyy-MM-dd)> <endDate (yyyy-MM-dd)> <output dir>

Note that the start date should be within 140 days as it is the oldest history you can download from Bloomberg.

Here is how it works. The Bloomberg system provides a core service named “//blp/refdata” which allows downloading a wide range of market data. The code opens a session to connect to the localhost at port 8194 (change the settings in SimpleSessionFactory if you are using a different port). Then, it sends the IntradayTickRequest with the security symbol, start and end dates. Upon receiving the response, the BloombergTickDataFileWriter saves the data as zipped CSV files in the specified output folder (one file per trading day). For example,

TickDataDownloadApp "5 HK Equity" 2012-12-01 2012-12-21 data

will save the data as .csv.zip files in the folder “data/5 HK Equity“.

Since AlgoQuant is source-available, you are free to change the code to download different data, or save the data into your database instead of files.

Happy downloading data!

Reference: Bloomberg API Version 3.x Developer’s Guide

Here are highlights of the new features available since 2.0.

-          ordinary and partial differential equation solvers

-          Optimization: Quadratic Programming,  Sequential Quadratic Programming, (Mixed) Integer Linear Programming, Semi-Definite Programming

-          ARIMA fit

-          LASSO and LARS

-          Gaussian quadrature/integration

-          Interpolation methods

-          Trigonometric functions and physical constants

-          Extreme Value Theory

Continuing our tradition, we will still provide trial license and academic license for eligible schools and research institutes. Moreover, we now provide another way to get a FREE SuanShu license – the contribution license. If you are able to contribute code to the SuanShu library, you can get a permanent license. For more information, see: http://numericalmethod.com/blog/suanshu/

We hope that you will find the new release of SuanShu more helpful than ever in your work. If you have any comments to help us improve, please do let us know.

Happy birthday to TianTians and Merry Christmas to all!

1.

Many people lose money playing the stock market. The strategies they use are nothing but superstitions. There is no scientific reason why, for example, buying on a breakout of the 250-day-moving average, would make money. Trading profits do not come from wishful thinking, ad-hoc decision, gambling, and hearsay, but diligent systematic study.
• Moving average as a superstitious trading strategy.

2.

Many professionals make money playing the stock market. One approach to investment decision or trading strategy is to treat it as a science. Before we make the first trade, we want to know how much money we expect to make. We want to know in what situations the strategy will make (or lose) money and how much.
• Moving average as a scientific trading strategy

3.

There are many mathematical tools and theories that we can use to quantify, analyse, and verify a trading strategy. We will show case some popular ones.
• Markov chain (a trend-following strategy)
• Cointegration (a mean-revision strategy)
• Stochastic differential equations (the best trading strategy, ever!)
• Extreme value theory (risk management, stop-loss)
• Monte Carlo simulation (what are the success factors in a trading strategy?)

One most popular way to create spurious trading model is data snooping or data mining. Suppose we want to create a model to trade AAPL daily. We download some data of, e.g., 100 days of AAPL, from Yahoo. If we work hard enough with the data, we will find a curve (model) that explains the data very well. For example, the following curve perfectly fits the data.

Suppose the prices are

Of course, most of us are judicious enough to avoid this obvious over-fitting formula. Unfortunately, some may fall into the trap of it in disguise. Let’s say we want to understand what factors contribute to the AAPL price movements or returns. (We now have 99 returns.) We come up with a list of 99 possible factors, such as PE, capitalization, dividends, etc. One very popular method to find significant factors is linear regression. So, we have

Guess how well this fits? The goodness-of-fit (R-squared) turns out be 100% – a perfect fit! It can be proved that this regression is a complete nonsense. Even if we throw in random values for those 99 factors, we will also end up with a perfect fit regression. Consequently, the coefficients and t-stats mean nothing.
Could we do a “smaller” regression on a small subset of factors, e.g., one factor at a time, and hope to identify the most significant factor? This step-wise regression turns out to be spurious as well. For a pool of large enough factors, there is big probability of finding (the most) significant factors even when the factors values are randomly generated.

Suppose we happen to regress returns on only capitalization and finds that this factor is significant. Even so, we may in fact be doing some form of data snooping. This is because there are thousands other people testing the same or different factors using the same data set, i.e., AAPL prices from Yahoo. This community, taken as a whole, is doing exactly the same step-wise regression described in the last paragraph. In summary, empirical evidence alone is not sufficient to justify a trading model.

To avoid data snooping in designing a trading strategy, Numerical Method Inc. recommends our clients a four-step procedure.

1. Hypothesis: we start with an insight, a theory, or a common sense about how the market works.
2. Modeling: translate the insight in English into mathematics (in Greek).
3. Application: in-sample calibration and out-sample backtesting.
4. Analysis: understand and explain the winning vs. losing trades.

In steps 1 and 2, we explicitly write down the model assumptions, deduce the model properties, and compute the p&l distribution. We prove that under those assumptions, the strategy will always make money (on average). Whether these assumptions are true can be verified against data using techniques such as hypothesis testing. Given the model parameters, we know exactly how much money we expect to make. This is all done before we even look at a particular data set. In other words, we avoid data snooping by using the data set only until the calibration step and after we have created a trading model.

An example of creating a trend following strategy using this procedure can be found in lecture 1 of the course “Introduction to Algorithmic Trading Strategies”.

• HKU-HKUST-Stanford Conference in Quantitative Finance. Dec 9, 2011.
• Workshop On Recent Developments Of Financial Mathematics (REDFIN2011). Dec 13, 2011.

You can find the powerpoint here.

Abstract:

There needs a technology to streamline the quantitative trading research process. Typically, quants/traders, from idea generation to strategy deployment, may take weeks if not months. This means not only loss of trading opportunity, but also a lengthy, tedious, erroneous process marred with ad-hoc decisions and primitive tools. From the organization’s perspective, comparing the paper performances of different traders is like comparing apples to oranges. The success of the firm relies on hiring the right geniuses. Our solution is a technological process that standardizes and automates most of the mechanical steps in quantitative trading research. Creating a new trading strategy should be as easy and fun as playing Legos by assembling together simpler ideas. Consequently, traders can focus their attention on what they are supposed to be best at – imagining new trading ideas/strategies.

Excerpts:

• In reality, the research process for a quantitative trading strategy, from conceptual design to actual execution, is very time consuming, e.g., months. The backtesting step, in the broadest sense, takes the longest time. There are too many details that we can include in the backtesting code. To just name a few, data cleaning and preparation, mathematics algorithms, mock market simulation, execution and slippage assumptions, parameter calibration, sensitivity analysis, and worst of all, debugging. In practice, most people will ignore many details and make unfortunate “approximation”. This is one major reason why real and paper p&l’s are different.
• Before AlgoQuant, there is no publicly available quantitative trading research platform that alleviates quants/traders from coding up those “infrastructural” components. Most of the existing tools are either lacking extensive built-in math libraries, or lacking modern programming language support, or lacking plug-and-play “trading toolboxes”.
• Technology can change the game by enhancing productivity. Imagine there is a system that automates and runs in a parallel grid of 100s of CPUs for you all those tedious and mundane tasks, data cleaning, mock market, calibration, and mathematics. You can save 80% of coding time and can focus your attention to trading ideas and analysis. Jim, using Matlab, may find a successful trading strategy in 3 months. You, equipped with the proper technology, may find 3 strategies in a month! The success of a hedge fund shall no longer rely on hiring genius.
• After we code up a strategy and choose a parameter set, there is a whole suite of analysis that we can go through and many measures that we can compute to evaluate the strategy. For instance, we can see how the strategy performs for historical data, simulated data generated from Monte Carlo simulation (parametric) or bootstrapping (non-parametric), as well as scenario data (hand crafted). We can construct the p&l distribution (it is unfortunate that historical p&l seems to be the popular performance measure; we traders do not really care about what we could make in the past but care only about our bonuses in the future; so what we really want to see is the future p&l distribution for uncertainty not historical p&l); we can do sensitivity analysis of parameters; we can compute the many performance statistics. All these are very CPU-intensive tasks. Using AlgoQuant, you simply feed your strategy into the system. AlgoQuant runs all these tasks on a parallel grid and generates a nice report for you.
• The academic community publishes very good papers on quantitative trading strategies. Unfortunately they are by-and-large unexplored. First, they are very difficult to understand because they are written for peer reviewers not laymen. Second, they are very difficult to reproduce because most authors do not publish source code. Third, they are very difficult to apply in real trading because the source code is not meant for public use, even if available.

Red Herring announced its Top 100 Asia award in recognition of the leading private companies from Asia, celebrating these startups’ innovations and technologies across their respective industries.

Red Herring’s Top 100 list has become a mark of distinction for identifying promising new companies and entrepreneurs. Red Herring editors were among the first to recognize that companies such as Facebook, Twitter, Google, Yahoo, Skype, Salesforce.com, YouTube, and eBay would change the way we live and work.

“Choosing the companies with the strongest potential was by no means a small feat,” said Alex Vieux, publisher and CEO of Red Herring. “After rigorous contemplation and discussion, we narrowed our list down from hundreds of candidates from across Asia to the Top 100 Winners. We believe Numerical Method Inc. embodies the vision, drive and innovation that define a successful entrepreneurial venture. Numerical Method Inc. should be proud of its accomplishment, as the competition was very strong.”

Red Herring’s editorial staff evaluated the companies on both quantitative and qualitative criteria, such as financial performance, technology innovation, management quality, strategy, and market penetration. This assessment of potential is complemented by a review of the track record and standing of startups relative to their sector peers, allowing Red Herring to see past the “buzz” and make the list a valuable instrument of discovery and advocacy for the most promising new business models in Asia.

Red Herring Top 100 Asia Award Ceremony