Link to Research Gate (this and other articles): https://bit.ly/3171Stocks

In this podcast we discussed consulting on quantum computing in financial services, and the challenges of a quantum computing consulting start-up.

Good morning quantum computing and portfolio optimization researchers,

So, a few days ago our models went haywire.  They started picking low BETA stocks to reduce portfolio risk (without us telling it to do that).  Who would have thought?  We are in the middle of a sector rotation from COVID 19 stocks to mainstreet stocks (small businesses = $IWM = Russell 2000) and low BETA stocks (those that provide a bit of safety when markets fall).

But, it also started to fall behind the power curve in picking deep enough minima.  It could be that picking low BETA stocks is easier than finding high BETA stocks that offset each other's risk.  

We ran our problem on the D-Wave 2000Q solver 6 last night and received good results, but no longer better than our classical models.  A few updates: we modified our chain strengths (0.75, 1.5) and our chain lengths varied (28, 38).  We failed to embed once out of 7 times (max clique of 64 fully connected vertices).  The system was fast and responsive.  We shifted our matrix to the negative at first, and received good answers, then shifted our matrix positive, and also received good answers (albeit, different ones).  Our best answers classically, and on the D-Wave, were between 7 and 9 stocks in a portfolio, and very low BETA securities (~0.4 or less).  

Why did our classical do better?  Thanks to Davood Rashmanifard, who as a Masters student in Iran wrote a thesis* on how to optimize a portfolio with a genetic algorithm, we found an optimization step in our Genetic Algorithm that was missing, and it caused a team discussion that uncovered a few more improvements we could make.  We increased the mutant:child ratio in our breeding (more noise), and updated how we determine the fitness (CQNS score) of each chromosome (stock portfolio).  We also updated our mutations to be more reasonable (flip only 1 or 2 stocks) in some cases.  We also blast out our initial population 100x before starting to breed.  Our Genetic Algorithm performs better now.  

We also modified our custom simulated annealer.  We created two versions of it, and customized them further for the task at hand.  One annealer jumps very often, even to less attractive portfolios, runs very hot to start (100 degrees), and finds a solution from a random seed.  We made this one start hotter.  It jumps to different neighborhoods easily and often, seeking out the best global minima before settling into a local minima.

Our second annealer starts with our best portfolio found (at that moment), and starts cooler, but runs more cold.  It may start at 0.001 degree and run to 1e-13 degrees.  We also suppressed its ability to jump to a less attractive portfolio...so it runs deeper in looking for local minima in the neighborhood.  In other words, it is nearsighted, and that is good for making that portfolio area the best it can be.

Finally, we continue to find ways to optimize our code in the Jupyter Notebooks to run faster and find deeper solutions.  We have found a number of promising research papers to keep learning new tricks and methods for deepening all of our methods.

Now that we have improved our stock picking algorithm and solvers, we plan to run another 3,200+ stock run and see if we start picking higher BETA stocks (like we did in October).

Here are a few pictures of what a successful run looks like on the D-Wave.  This uses the D-Wave Problem Inspector application.  Thank you D-Wave!
​
Finally, our research on Kurtosis and Skewness in portfolios has slowed down as we continue to improve our stock picking model.  Our next step, is to try and run our stock picker on the D-Wave Pegasus with only 64 stocks, and see if we get better answers.  We will also port our desktop stock picking code to our HPe deskside server to see if we can find deeper and better portfolios.  

Written by: Jeffrey Cohen, Chicago Quantum
Contact me at  +1.312.515.7333 (cell) and jeffrey@quantum-usaci.com (email).  
Client inquiries welcome.

To learn about and try our stock analysis service, please click the button below.  Your purchase funds our research.  



* https://www.academia.edu/2044223/Portfolio_Selection_and_Optimization_with_Genetic_Algorithm

Jeffrey Cohen, President, US Advanced Computing Infrastructure, Inc.
Nov 12, 2020, 5:37pm CT
Stargate log 202011121737: approaching the end of a long day of conferences and coding

We analyzed 3,231 stocks by applying our Chicago Quantum Net Score and picked an efficient portfolio of 5 stocks.  These stocks should have reduced volatility while maintaining their BETA exposure to the market.  See the end of the article for the 5 US common stock tickers we picked.

We have broken up our logic to run the big sets of stocks in one code-set (yesterday and today we validated 3,231 US Common Stocks and ran them).  Instead of a fast run, we ran it long and slow...deep analysis into the portfolios.  We had 253 days of stock data, and a search space of 10 ^ 972, so I figured it was worth the extra processing time.

BETA range post validation (3.11 and 0.0119) 
market return (1 year): 14.95%.
All in stocks:
- Chicago Quantum Ratio = 0.8468
- Sharpe Ratio = 6.69
- CQNS_Power from CQNS_True = 4.182982075474502
- Expected Return = 17.53%
- Variance = 0.00068608
- Standard Deviation = 2.62%
- CQNS score = 0*   (actually -3.25 e-19)

63 stock best solution: CQNS Score: -0.001288, or 1.29 x 10-3

So, we picked 64 stocks for round two.  The model picked 63, and two failed 2nd-level validation (we had new market data), so we were down to 61 stocks.  We looked at competing models in our stable (bespoke simulated annealer and GA random seed), and found 3 stocks that were in both of those '2nd best' portfolios, but not in the best one.  So, now we have 64 stocks.

Here are the 64 stocks we ran:
Stocks = ACIA ACTG AKAM BGIO BJ CALM CBPO CCX CVR DISCB DUC EGF EQC EQS EVN FIZZ FMY GBAB GIS GIX HMG IGI ITP JMP LN LSXMB MASI MCA MGF MHF MIY MSN MTT NCB NID NMI NUM NXC NXQ PATI PFMT PLAG PSTL PZZA QGEN REGN RELL RMG RMI SBE SBI SEB SG SWBI THS TIF TRMD TRNE TVC VCIF VCRA VIRT VZ WPM

Interesting data about this 2nd set of stocks (the first set of 'Allstars') is that they have very low BETA values: (0.648 and 0.0135)

Having a very high CQNS power (4.18) means the expected returns are less important (actually have a lower value) that typically, and therefore the volatility is relatively higher, and more important in the optimization.  We used to run CQNS power at ~ 3.0, which should pick higher-BETA stocks.  

The world looked a little brighter when we ran 'round 2' and market return was now 14.99%, up 4 basis points for the year.  This is a challenge of running the model in two parts during the trading day.

Now for the good part.  The D-Wave found a portfolio with the best CQNS score: -1.5 x 10-4.  We ran the D-Wave about 10 times.  Took about 15 minutes of elapsed time, including a short break for coffee.

The other solvers did ok, but not as well.  Our best classical solutions came from either a simulated annealer or a genetic algorithm.  Tonight we are going to beef up our genetic algorithm to find better answers.  We need that one to really dig into the matrix and give us better answers (like it has in the past before we speeded it up).

Table of methods, scores, # stocks in best answer
D-Wave QA, -1.5 x 10-4, 5
D-Wave SA, -1.0 x 10-4, 1
Custom GA, -8.8x 10-5, 6
Custom SA, -7.7 x 10-5, 2
D-Wave TABU, -7.4 x 10-5, 30
Custom Bifurcator, -5.0 x 10-5, 38
All 64 Stocks, -4.9 x 10-5, 64

Would you like to know what five (5) stocks we picked?  Here they are:
QA D-Wave -0.000150  ['AKAM', 'CCX', 'FMY', 'PZZA', 'VZ'] 5

On the D-Wave 2000Q solver 6 (Chimera) we ran Chain Strength = 1, annealing time = 50 microseconds, and 500 runs for portfolios of size (2,8) and the best one was 5 stocks.

This is a pretty big map for the Chimera.  In a recent run, 64 stocks uses 1,364 qubits as 'target variable' with a maximum chain length of 30 qubits (to hold one stock value).  Also, each run took 146k microseconds, or 146 milliseconds of QPU access time, which includes programming time and sampling time.

Please feel free to ask me questions about the work or the D-Wave quantum annealer runs.  We are all learning, researching, and growing.  As an example, I am curious if the Pegasus (new D-Wave Advantage (TM) system) would give us a better answer for 5 stocks.

Here are some graphics from the D-Wave Inspector (TM) for the quantum annealing runs.  There are very few chain breaks.  You can see in one run the energy was very bad (very positive), with a few negative values which we wanted.  This is still a probabilistic system, and our creativity is in getting it to find really good answers quickly and cheaply.

Listening to Baby Driver Movie track "Was He Slow" hoping my code will run faster.  This part won't.  It is data validation.  We start off with 3,437 stocks, and perform initial data validation.  The first run takes 24 minutes in Python with Yahoo Finance data (thank you Yahoo Finance).  We run 10 automated check on each stock ticker, and we filter out and remove any stocks with missing or invalid data.  That part takes 15+ minutes and is tedious.  We also had one BETA < 0 stock from work earlier today and removed that stock.  More are coming with negative BETA values I would guess.
Had to run data validation a second time to remove another set of ~25 stocks without valid data.  I found them after the first validation (same checks), but then the overall download failed to deliver enough data (e.g., 252 days of adjusted close per ticker).
​
BTW, it is after midnight and I really want to see what stocks come through the analysis.  We have had a very volatile week since the election.  BTW, our two latest picks, AX/MCRB and PDCE have torn the cover off the baseball...hit it out of the park, and in other words, seem to continue to outperform the benchmarks.

Today was a busy day of coding on our CQNS model.  We did a few things and found it runs faster and better.  Those things were:
1. We went back to the QUBO friendly formulation of the CQNS.  It is a little less 'true' to the economics, but it allows us to have QUBO and algebraic solving occur with the same formulations.  This is where we take the individual stocks and raise their expected returns to a power, then calculate the portfolio return.  We had switched it from taking the expected return of the portfolio, and then raising that to a power.  It is truly a minor point, only a few basis points of difference, but it makes the code run better.
2.  We scrubbed the first part of the code, the one that has data validation and the first solution of the problem.  We removed all extra code or charting or reporting from this first section.  This one is too large for quantum computing to solve (going from 3,200 stocks to a smaller number of stocks (say N).  Ran it before dinner and it screamed with speed on my 2013 iMAC with 16GB RAM.  It might be time to buy a new iMAC...or not be lazy and dust off my HPe server.
​3.  We dug a little deeper into the D-Wave Simulated Annealer and had it find N stocks (out of a smaller set of 422 stocks) by adding back all the systems parameters and optimizing them.  Let's hope that same tuning works for 3,200 stocks.  Tabu sampler tuning still did not provide us a matching portfolio (e.g., N stocks out of 422), but it came closer.  Not giving up on the MultiStart Tabu solver.
4.  We decided to keep SPY as the ETF for the S&P 500 instead of using the ^GSPC index.  It works well.
5.  We find a very small bug in our calculation of log returns.  The difference between day zero and day one throws an error.  A few weeks ago, we decided to 'fix' that error and fill the first day with a '1.0' value.  We did that by switching from .dropna to .fillna(1).  The problem is that this creates a first day doubling of every stock on the first day of the sample.  Log returns value of 1, is a 100% percentage gain.  I believe this is what caused the 68 stock solution before election day.  This had artificially reduced all the BETA scores, expected returns, and likely other unintended consequences.  This is now fixed, and we have a more normal, wider range of BETA values.  
6.  We have also put our solvers to work in different ways, and this is helping us get slightly better answers overall.  For example, we now use our bespoke simulated annealer against the best answer already found, and as a solver with a random seed.  As a pair, this does better.  We also are training our genetic algorithms to do better by adding different types of mutation in the breeding process, and continuing to run the GA against the best answer seen so far, and doubling generations and initial random seed sizes.  
7.  The one issue we have that we cannot code around is the D-Wave Advantage (TM) 1.1 solver which allows us to go from 64 to 134 stocks at one time on the quantum annealer.  If we want to run more than 64 stocks at a time in the second program...then we need it.  However, tonight the waiti time to try an embed 3,200 stocks on the D-Wave was over 25 minutes.  The embedding did not fail, it timed out...and then was going to time out for 1,000 runs.  So, we are going back to the D-Wave Systems 2000Q Chimera architecture to run our quantum algorithm.  It has wait times in the seconds, not minutes.  
8.  Therefore, we are going to bring our solvers down to N stocks in the first program, and N will be 64 stocks or less.  After that, we will run all 64 stocks through our set of classical solvers, and through a battery of runs on the D-Wave Systems 2000Q (solver #6 tonight).  This allows us to scale back up to the D-Wave Advantage and hopefully see better performance on a 64 stocks solution with less chain breaks and stress on the QUBO / qubits.

Oh, and our new research into the higher moments of stock prices is progressing.  We have a simple way to identify our outliers for kurtosis and skew.  These metrics, along with expected return and variance, can give us a better understand of each individual stock.  We are still working on a formulation for calculating these metrics for a portfolio, as opposed to for a stock, which is called cokurtosis and coskewness.

The final word...I spent many hours this week fixing our website and optimizing it for search and advertisements.  Please take a look when you have a chance, and I welcome your feedback.

Everyone please stay safe out there.  We are likely heading for a 'stay at home' order in Illinois.

Best regards,
Jeffrey Cohen
President, Chicago Quantum, US Advanced Computing Infrastructure, Inc.