Artificial Intelligence, Machine Learning, and Big Data


It seems like you can't watch the or read an article without hearing about AI, Robots, Big Data, Machine Learning etc...  Often its confusing and sometimes maybe even scary.  Are Bill Gates, Stephen Hawking, and Elon Musk right?  Will Artificial Intelligence one day threaten civilization?  I don't know.  But I do know that these three men are way smarter than I am so I wouldn't bet against it... (insert Twilight Zone music here)...  Anyway, here are some briefs notes on these topics and how Investment Managers can utilize them to enhance their returns.  Hopefully it will help you understand how to benefit from AI before Armageddon ensues. 


What is Artificial Intelligence?

Artificial Intelligence, Machine Intelligence or my preferred term, synthetic intelligence, is intelligence displayed by computers, and other types of machines, as opposed to natural intelligence which is displayed by humans and other animals.  When a machine can complete cognitive functions that humans associate with their minds such as learning. predicting, and problem solving it is considered 'Artificial Intelligence.'

There are different levels and types of intelligence.  For example, anyone who has ever had a pet dog knows that they are intelligent.  My dog Clifford can make predictions on what we are going to do based on the leash that I get when we are going to go out.  The short leather leash means we are going for a ride while the long extended leash means we are going into the woods. When I pick up the leather leash he runs to the car,  When I pick up the extended lease he runs to the woods.  Clifford has learned how to make a prediction about what we are going to do and that is a definition of intelligence.

Clifford can also consider a goal and figure out a way to get to it.  He knows that if he uses his paw to grab me and cries and barks enough, then I will eventually give in and give him some of whatever I am eating.  He does this because in the past it has worked, and now he is making an inference and predicting that it will produce the same results.  This is also a display of what is considered 'intelligence'. 

To go to the next level, humans have the ability to make predictions but it is on a higher level because our larger brains can consider many more variables than an animal can.  Humans can analyze things that have happen in the past in order to make inferences of what will happen in the future and humans can set goals and then make predictions of what are the best actions to take in order to achieve these goals.

AI is computers and software that can do things that we define as intelligence.  They can analyze the past and use the knowledge to make predictions and inferences of what will happen in the future.  They can also set goals to achieve and predict which actions to take to maximize the chances of obtaining those goals.


What is Machine Learning?

Machine Learning is closely related to computational statistics, which focuses on making predictions through the recognition of patterns.  ML is the utilization of cognitive computing to analyze data with the goal of identifying patterns, relationships, and correlations.  These factors and patterns demonstrate the relationships between behaviors and outcomes. Once these relationships have been identified they can be used to make inferences about the behavior of new cases when they present themselves.

ML and data mining often employ the same methods and overlap significantly, but ML focuses on prediction based on known properties learned from the training data, while data mining focuses on the discovery of previously unknown properties in the data.


What is Big Data?

Big data is a term for data sets that are so large or complex that they are beyond the ability of traditionally used software tools to process and analyze.  Advanced Artificial Intelligence technologies make possible the advanced methods that have the ability to extract value from data by identifying predictive patterns.

The definition of ‘Big Data’ is a moving target.  What was considered Big Data a few years ago is no longer considered Big Data.  This dynamic requires that new types of techniques and technologies must constantly be evolving.

This data comes from Big Data repositories which are often built by corporations.  It can also come from commercial vendors.  For instance, Google handles over 100 billion searches a month, Facebook has over 50 billion photos, and people sell their DNA to places like  The amounts and types of data that can be complied from cell phones is vast.


Why now?

The sciences of Artificial Intelligence and Machine Learning are not new, but recent changes and advances in the technological landscape have made their implementation and applications viable and profitable.  The current paradigm is occurring due to massive increases in Computing Power and Big Data availability.

Over the past decade, there has been an unprecedented growth of the amount of data.  For example, in 2006, 100 exabytes of data were on the internet.  Today that number exceeds 10,000 exabyte’s.

There has also been a massive reduction in costs.  A gigabyte of data that costs $300,000 in 1981 costs just ten cents today while data sets which cost $1 million to analyze 20 years ago can now be analyzed for literally a few cents. 


Examples in Society

There are number examples of AI and ML in society that are encountered by people everyday without them even realizing it.  Apples Siri is an example of voice recognition technology driven by AI.  So are those annoying robocallers that keep calling. 

According to a report published by Stanford University, “AI & Life in 2030”, the industries and areas that will be most affected are transportation, service robotics, healthcare, education, low resource communities, public safety & security, and entertainment.  

The Department of Labor predicts that 65% of children who are in school now with eventually be employed in jobs that don’t exist yet.

Commercial Banking Industry - Credit scoring was one of the first things in the financial services industry to be automated almost seventy years ago.  It used to be that the president of the local bank could decide who could or couldn't get a mortgage or a loan.  Now a sophisticated AI architecture does.  It can make better predictions than conventional methods because it can examine vast amounts of data and find predictive patterns.    

Automotive Industry - There are more than thirty companies that are working on the creation of driverless cars and other types of vehicles.  This area is a popular topic of conversation.  People generally say that they will not get into a self driving car.  It is important to understand that commercial aircraft are flown by automated programs, and the pilots are on the plane so the passengers feel comfortable.  This is why flying is much safer than driving a car.

Healthcare Industry – There are approximately 800 medicines and vaccines that are used to treat cancer.  This negatively affects doctors because there are too many options to choose from.  Microsoft is working on a machine called ‘Hanover’ which will try to study and memorize all of the academic papers in order to predict which combinations will be most beneficial for individual patients.  It has been proven that AI is better at recognizing signs of cancer on a medical scan than most experienced doctors are.  

Sports Industry – Professional sports organizations are employing AI in numerous way.  They are scouting players based on statistics.  There is a movie about this called "Moneyball".  Formula One and NASCAR cars have hundreds of sensors which generate terabytes of data in an attempt to gain an edge.  Such data points could be fuel usage, temperature, humidity, and tire pressure and so on and so on.

Science - It shouldn't be surprising that AI has had profound effects on the ability to advance science.  For example, the Human Genome Project took 10 years to decode the human genome.  With current methods it could now accomplished in one day.  The cost reduction is 100x than the reduction predicted by Moore's law.  

Gaming and Entertainment -  In 1997 Deep Blue beat Gary Kasparov at chess.  In 2011 IBM’s Watson beats two human contestants on Jeopardy, one of which was the record holder for most wins.  This task took 5 years.  Watson spent 4 years learning English and then another year reading and retaining every word in Wikipedia as well as thousands of other books.


Artificial Intelligence, Machine Learning, and Big Data as applied to the Asset Manager Industry


It isn’t some sort of magic pixie dust.  It’s just enhanced statistical techniques that allow you to build better predictive models.” 

Adam Duncan, managing director at Cambridge Associates


"The industry is on the cusp of a technological arms race where an intelligent program is poised to join your team as a machine co-pilot"

Gary Brackenridge, global head of asset management at Linedata


"It's all about seeing how far we can push the machine in terms of taking some of the decision-making on the investment side. There are so many variables that people are looking at to be aggregated into the decision-making so the more you can use a machine, the better it is"

Man Group's Pierre Lagrange.


Machines will be doing more of the grunt work of discovering opportunities,” “They can generate hypotheses, test them, and then tell humans, ‘This is interesting, go dig deeper.’ As machines add more value, it changes the nature of work humans do.”

Vasant Dhar, Founder SCT Capital Management.


"With AI, advisors will be able to focus on what really matters to clients. AI will change everything and advisors who embrace it will win. I would guess that many advisors are woefully out of date with its technology because they don't understand and are scared of it."

Larry Miles, a principal at AdvicePeriod


"its programming and mining tons of data to figure out what the best answer may be,"

Grant Easterbrook, co-founder of Dream Forward 401(k),


AI is creating a lot of new opportunities. Just as about 100 years ago electrification changed every single major industry, I think we’re in the phase where AI will change pretty much every major industry.

Andrew Ng, chief scientist at Chinese internet giant Baidu Inc.


"A couple of years ago when I was at a Ted Talk in Vancouver, I heard Jeremy Howard speak on the future of Artificial Intelligence and how computers were already on the cusp of surpassing human performance in many tasks. Shortly thereafter, I met a fund that was modelled after Graham and Dodd's value investing philosophy but was able to analyze every public filing, such as 10Qs and 10Ks since inception. This further confirmed my belief that this was the future of investing.”

Jeffrey Tarrant, CEO and Founder of MOV37 and CEO, CIO at Protégé Partners robot can keep children occupied for hours without supervision


“AI will ultimately prove to be cheaper, more efficient, and potentially more impartial in its actions than human beings,” “It just means that their jobs will change to focus on things only humans can do"

Harvard Business Review


“Some of this is tactical and operational in nature.  If you can save money, that’s great, and if you can execute trades, that is great too. All of those processes work to make the bank more efficient. It obviously saves money in a lot of different areas and serves the client quite profoundly in the near-term.”

Foster BNY Mellon


Investment managers across various styles can utilize Artificial Intelligence, Machine Learning, and Big Data to enhance performance and reduce risk.

The ability to extract value from Big Data has become a key differentiator in the ultra competitive field of asset management.  Conventional and traditional methods will rapidly fall behind and have there ability to add value will diminish rapidly.  In contrast, managers who can overcome ‘algorithm aversion’ and learn how to integrate these technologies will have an edge that will last for a number of years.   Managers should not fear being replaced by robots. 

Artificial Intelligence, Machine Learning, and Big Data should be viewed by investment managers as tools that can be used to enhance performance and reduce risk.  They also need to understand that there are numerous ways that this can be accomplished.  There is no ‘one size fits all’ application.  These technologies can benefit managers of virtually any style.  It is well known that leading quantitative managers such as AQR and Bridgewater integrate AI and ML into their quantitative and algorithmic trading strategies.   What is less well known and understood is how they can be used to gain Alpha in traditional investment models and strategies.


Traditional Value Investors can benefit from the utilization of predictive AI models to enhance their fundamental research processes. 

A traditional value manager which is  modeled on a value philosophy such as Graham and Dodd’s could use AI to study and analyze a company's  public filings, such as 10-Qs and 10-Ks, as well as those of competitors and suppliers. The AI could also analyze and make predictions based on other important factors such as news releases, TV shows, website clicks, and mentions in text messages. 

AI & ML in the context of equity valuation can evaluate parameters such as discounted cash flows, ROEs, EBITAs, news sentiment, and literally thousands of others.  Analytics using AI& ML can be more robust than traditional financial modeling.  The models can identify patterns in massive sets of 'unstructured data'.

For example, consider the stock of a retailor.  An AI model could be built that could analyze google earth images of parking lots to see how many customers are visiting the stores in addition to analyzing web traffic of the retailor as well as those of its competitors.  It could also analyze numerous other factors such as mentions on Twitter, accounting data, financial data, market technicals, news releases, macro policy and so on.  These capabilities can be utilized in order to build better predictive models to determine value than current methods are capable of..


Investment managers can gain Alpha by using AI to enhance portfolio construction optimization.

Top down and macro managers can enhance performance and gain alpha by utilizing AI to enhance portfolio weightings and rebalance frequencies.  This can be done on the sector, industry, or individual security selection.


Residential and Commercial Backed Securities Managers can utilize AI to build better portfolio valuation models.  

Traditional or conventional portfolio valuation models may evaluate 20 – 200 variables.  An AI enhancement could bring this number up into the tens of thousands and greatly increase the predictive value.  Data can be obtained based on zip codes for example, and hundreds of factors could be considered to look for patterns.  For example, obvious things such as interest and employment rates and taxes would be evaluated.  Other less obvious factors could also be evaluated such as mortality rates, average age, regional economic conditions, school graduation rates, transportation availability, employment opportunities, and so on.


In summary, Artificial Intelligence, Machine Learning, and Big Data can enhance any investment strategy that uses tactical asset allocation, statistical modeling, and/or requires the analysis of large amounts of quantitative data.   Managers need to understand that there in no one size fits all application and there may be different ways to benefit from these technologies.  Perhaps the most important dynamic for a manager to consider is that it will save time by increasing accuracies and efficiencies.  


Artificial Intelligence and Machine Learning present a clear way for active managers to compete with and ultimately outperform their benchmarks and passive managers.

Artificial Intelligence can help lagging Active Managers compete with and surpass Passive Managers.  There is an interesting dynamic to consider when analyzing passive vs active management.   The recent advancements can help active managers outperform passive managers because passive strategies by definition do not need to enhance performance. There will certainly be other ways that passive managers will benefit from the advances in AI and ML, such as in their operational structures, but they will be not using it to enhance performance because it does not align with their strategies.


The implementation of Artificial Intelligence and help managers market and develop their businesses. 

Leading firms are developing new products that utilize AI.  There are seeders and investors are seeking to invest in funds that utilize AI & ML.  According to a recent story in Bloomberg, Man Group has gone from viewing AI with skepticism to making it a cornerstone strategy.  Their assets AUM have increased +77% since early 2014.  This isn't completely due to the use of AI, but the AI is a significant part of it.   


Active managers  need to embrace Artificial Intelligence techniques and technologies now, because within a few years their chance to gain a decisive edge will diminish as more managers embrace AI and it eventually becomes standardized and ubiquitous. 


What is bad AI?

Predictive models driven by AI can and do get things wrong.  If there are biases or problems with the data being used to build the predictive models, then the models will be flawed.  Also, if the people who build the model make incorrect predictions about how it will be used, then the model will be flawed.  Another issue could be if the Data Scientist has particular prejudices he could consciously design them into the system subverting what the data is saying.  Over fitting could also be a problem.  This occurs when an algorithm searches too far to find correlations and patterns in the data.  There is randomness in all data and if it is over analyzed there may be incorrect inferences.


Philosophical and Ethical Questions that are being discussed...

Is Artificial General Intelligence possible?  Will a machine eventually be able to solve any problem that a human can using natural intelligence?

Could AI be designed or guaranteed to be benevolent?  Because AI has no need to share our human motivational tendencies, could the designers program what we know as morality?

Are intelligent machines potentially dangerous?  Can we ensure that machines behave ethically and are used ethically?

Is there an existential risk?  Could the AI take off on its own and redesign itself at an ever increasing rate?  Could it evolve to the point that humans cannot control it?  

How will there be accountability of things such as self-driving cars?  What if there is an accident?

Is AI potentially a threat to privacy?  Can AI speech recognition technologies and the ability to read and understand every email and text for a population effectively allow Governments to suppress dissent? 

Is there a threat to human dignity if AI replaces people in positions to require respect and care, such as nurses, police, and teachers?  Will there be a diminished value of the human spirit if we think of ourselves as merely biological computers?

Will AI decrease the demand for human labor by taking away more jobs than it creates?

Will Artificial Intelligence Weapons used for military combat develop to ability to make their own intelligent decisions of who and when to kill?

Could the AI infrastructure of a hedge fund figure out how to shut down and extract the capabilities of its competitors and rivals?

Could an AI system short airline stocks and make a plane crash, or create other global conflicts to profit from?

If the human nervous system obeys the laws of physics and chemistry, which we have every reason to believe that it does, then can we reproduce the behavior of the nervous system with some physical device?  

Are emotions chemical reactions that evolution has given us for survival?  Can a machine have a consciousness?  Can the have a self awareness?  Can they intentionally cause harm?  Can they be insulted or be jealous?  Fall in love?  Enjoy music?  

What are the Religious and Moral implications of this?