AI and the stock market: Can it pick and predict stocks better than humans?

Artificial intelligence (AI) is making its mark in the world of finance, particularly in the stock market. There are two main types of AI being utilised: generative AI and predictive AI. Generative AI, which powers chatbots like OpenAI’s ChatGPT, enhances natural language processing and aids in tasks such as summarising reports or detecting trading signals. It is already being used by hedge funds, banks, and financial institutions for various purposes. However, the widespread use of generative AI is not expected to replace human jobs but rather increase productivity and create a demand for professionals skilled in both finance and AI. On the other hand, predictive AI is used in quantitative trading to make predictions about market trends and stock prices. While it has the potential to provide valuable insights, it faces challenges such as overfitting and the difficulty of explaining its decision-making process. Overall, AI is playing an increasingly significant role in finance, but its impact on the stock market is still a topic of debate, with mixed results in terms of outperforming the market.

Sign up for your early morning brew of the BizNews Insider to keep you up to speed with the content that matters. The newsletter will land in your inbox at 5:30am weekdays. Register here.

AI Can Write, But Is It Any Good at Picking Stocks?

By Justina Lee

With its potential to boost productivity across multiple industries, artificial intelligence is the stock market investment theme of the year. But if you ask traders how AI is changing investing itself, their answers might underwhelm: Twitter-reading robots, data analysis tools or algorithms for routing buy and sell orders. For financial markets, the Holy Grail remains a form of AI that can tell where prices are headed more accurately than a human can — a challenge that’s far tougher than teaching a computer to summarise a research report.

1. What kind of AI are we talking about?

Broadly, there are two main kinds of AI being used in finance. The first is generative AI — the technology that underpins chatbots such as OpenAI’s ChatGPT and Google’s Bard. The second is predictive AI, an important tool in quantitative trading, where math-trained analysts sift large amounts of financial data for patterns and trends to find new trading strategies.

Read more: BNC#5 – Cy Jacobs Q&A: Our stock market is not a reflection of SA – massive opportunity out there

2. How can generative AI be used in finance?

Generative AI mimics the workings of the human brain to perform complex cognitive tasks based on simple written prompts. The systems are trained on vast quantities of pre-existing material and learn how to use that information to craft something new, such as a blurb for a new novel, a summary of a report, a poem or a legal contract. The large language models powering these chatbots can improve something known as natural language processing, which financial professionals already use to parse earnings statements, call transcripts and other documents to detect trading signals or potential investment risks in the blink of an eye. Whereas older iterations of that process relied on spotting particular words, the latest tech is better at analysing contexts, making them more accurate. Some academic studies have shown that ChatGPT can be used to decipher the market implications of “Fedspeak” and corporate news in ways that are eerily close to what a human expert can achieve.

3. Is generative AI already being used in finance?

Yes. Hedge funds are experimenting with ChatGPT for writing code, summarising research or producing client reports, and banks are deploying LLM-based chatbots to answer client questions. Bloomberg LP, the parent of Bloomberg News, has released a large language model specifically for finance, BloombergGPT.

4. Will generative AI take Wall Street jobs?

AI has already disrupted many areas of finance, especially roles involving repetitive tasks, such as support functions and back-office processing. Generative AI also has implications for higher-skilled jobs that involve collecting and analysing data and creating reports. It doesn’t necessarily mean humans being muscled aside. Many will be trained to use these AI tools to be more productive. On the flip side, there’s big demand for data scientists with AI skills and a flair for finance. At the most enthusiastic banks, about 40% of all open job roles are for AI-related hires such as data engineers and quants, as well as ethics and governance roles, according to data from consultancy Evident. JPMorgan Chase & Co. is hoovering up talent, advertising for more AI roles than any of its rivals.

5. What about predictive AI?

Whereas generative AI is focused on creating new material based on training data, the AI used in quantitative trading is trying to make predictions such as where a bond price is headed. What these tools have in common is they’re all trained by finding patterns in large amounts of data. Quant specialists say traditional quant models are based on linear relationships, such as the observation that value stocks go up over time, but can be too simplistic to capture the complexity of markets. Recent machine-learning models, on the other hand, are better at fitting in a large number of inputs and detecting complicated patterns including how different variables interact with one another.

Read more: Premium: This stock market splash has a disturbing undertow

6. Will predictive AI soon replace traders and market strategists?

A total wipeout looks unlikely. Daily swings in stock indexes are driven by so many factors that it can be hard to detect reliable signals in all the noise. What’s more, markets are subject to constant regime changes — from regulatory reform to structural changes in investment flows — that they will respond to the same signals in different ways from one period to another. So what a machine has observed on one day may no longer apply on the next, and what looks in theory like a winning investment strategy may not actually make money in live trading. Plenty of human traders fall into that trap too. But since these AI systems are akin to a black box that finds complex linkages in vast troves of data, they are especially prone to this problem of “overfitting” to what’s actually noise. Plus the more complex a machine’s trading model, the harder it is to explain its choices compared to those of traditional quant strategies devised by humans. That’s a problem in an industry where clients often demand a coherent reason for poor performance.

7. How widespread is the use of AI for investing?

Lots of money managers say they are using machine learning in their investing process. That doesn’t mean it’s telling them what to buy or sell. AI can be used in many aspects of their work — combining trading signals, calculating the risk of a big crash or deciding how best to execute a trade. It’s probably safe to say that most quant-driven funds use machine learning in some shape or form. A handful of those, including Voleon Group or Voloridge Investment Management, are known for their AI prowess. A cohort of financial technology startups are also trying to apply AI to finance, such as EquiLibre Technologies, Kavout and Axyon AI, which are mostly all deploying machine learning to figure out where markets are going.

8. So can AI beat the markets?

There’s no clear-cut way to determine this, since different traders use AI in different ways. For one perspective, the Eurekahedge index of AI hedge funds has underperformed the broader universe by about 9 percentage points over the past five years. The AI Powered Equity ETF has also lagged the S&P 500 index by about 50 percentage points over that period. On the other hand, a 2021 academic paper suggested mutual funds powered by the technology beat their human-managed peers, though not the market. In short, AI is playing a bigger role on Wall Street. But progress is likely to be slower and less flashy than some Silicon Valley executives like to make out. (Bloomberg)

The Reference Shelf

QuickTake explainers on generative AI, how large language models work, the US-China AI rivalry, a glossary of AI terms to know and the Nvidia chip driving generative AI.
Some US lawmakers see AI posing an acute risk to the US financial system, while Blythe Masters warns of a “devastating loss” if AI goes unchecked.
A 2020 paper from AQR quants on whether machines can learn finance.
An Odd Lots podcast on how finance worked before computers.

–With assistance from Sam Potter.