* . *
  • About
  • Advertise
  • Privacy & Policy
  • Contact
Friday, July 4, 2025
Earth-News
  • Home
  • Business
  • Entertainment
    Kennedy Publishing, MGA Entertainment Launch Yummiland Magazine – License Global

    Kennedy Publishing, MGA Entertainment Launch Yummiland Magazine – License Global

    MAY HER SOUL REST IN PEACE 🙏 Veteran entertainment columnist and talent manager Lolit Solis has passed away. She was 78 years old. https://tinyurl.com/6kumarkx | LatestChika.com – Facebook

    Beloved Entertainment Icon Lolit Solis Passes Away at 78 – A Life Remembered with Love and Respect 🙏

    Neil Young Plays Rare Full-Band ‘Ambulance Blues’ With The Chrome Hearts – Yahoo

    Neil Young Stuns Fans with Rare Full-Band Performance of ‘Ambulance Blues’ Alongside The Chrome Hearts

    BTS Announce Their Big Return and Yes, They Already Have Some Major Plans in the Works – Yahoo

    BTS Announce Their Big Return and Yes, They Already Have Some Major Plans in the Works – Yahoo

    Nantucket Dance Festival opens July 8 – The Inquirer and Mirror

    Nantucket Dance Festival Launches with Thrilling Performances Beginning July 8

    A Secret Society, Ritualistic Killings, and a Century-Old Curse Netflix and YRF Entertainment’s ‘Mandala Murders’ Premieres July 25 – About Netflix

    A Secret Society, Ritualistic Killings, and a Century-Old Curse: Dive into the Chilling World of ‘Mandala Murders’ Premiering July 25

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Environmental cognitive distance, R&D capability distance, and supply chain green technology innovation – Nature

    Bridging Gaps: How Environmental and R&D Differences Drive Green Technology Innovation in Supply Chains

    LG Innotek CEO Moon Hyuksoo: “Our Next-gen Substrate Technology Will Change the Industry Paradigm” – TechPowerUp

    LG Innotek CEO Moon Hyuksoo: “Our Next-Gen Substrate Technology Will Revolutionize the Industry” Revolutionizing the Future: LG Innotek’s CEO Unveils Game-Changing Next-Gen Substrate Technology

    Inspira Technologies Secures Landmark $22.5M Deal: Major Revenue Breakthrough After FDA Clearance – Stock Titan

    Inspira Technologies Secures Landmark $22.5M Deal: Major Revenue Breakthrough After FDA Clearance – Stock Titan

    Meiwu Technology Company Limited and Shenzhen Zhinuo – GlobeNewswire

    Meiwu Technology Company Limited and Shenzhen Zhinuo – GlobeNewswire

    Owls inspire new revolutionary noise reduction technology – KTEN

    Owls inspire new revolutionary noise reduction technology – KTEN

    New center coming to Mizzou will focus on energy research and technology – Columbia Missourian

    Mizzou Launches Innovative New Center Dedicated to Energy Research and Technology

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
  • Home
  • Business
  • Entertainment
    Kennedy Publishing, MGA Entertainment Launch Yummiland Magazine – License Global

    Kennedy Publishing, MGA Entertainment Launch Yummiland Magazine – License Global

    MAY HER SOUL REST IN PEACE 🙏 Veteran entertainment columnist and talent manager Lolit Solis has passed away. She was 78 years old. https://tinyurl.com/6kumarkx | LatestChika.com – Facebook

    Beloved Entertainment Icon Lolit Solis Passes Away at 78 – A Life Remembered with Love and Respect 🙏

    Neil Young Plays Rare Full-Band ‘Ambulance Blues’ With The Chrome Hearts – Yahoo

    Neil Young Stuns Fans with Rare Full-Band Performance of ‘Ambulance Blues’ Alongside The Chrome Hearts

    BTS Announce Their Big Return and Yes, They Already Have Some Major Plans in the Works – Yahoo

    BTS Announce Their Big Return and Yes, They Already Have Some Major Plans in the Works – Yahoo

    Nantucket Dance Festival opens July 8 – The Inquirer and Mirror

    Nantucket Dance Festival Launches with Thrilling Performances Beginning July 8

    A Secret Society, Ritualistic Killings, and a Century-Old Curse Netflix and YRF Entertainment’s ‘Mandala Murders’ Premieres July 25 – About Netflix

    A Secret Society, Ritualistic Killings, and a Century-Old Curse: Dive into the Chilling World of ‘Mandala Murders’ Premiering July 25

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Environmental cognitive distance, R&D capability distance, and supply chain green technology innovation – Nature

    Bridging Gaps: How Environmental and R&D Differences Drive Green Technology Innovation in Supply Chains

    LG Innotek CEO Moon Hyuksoo: “Our Next-gen Substrate Technology Will Change the Industry Paradigm” – TechPowerUp

    LG Innotek CEO Moon Hyuksoo: “Our Next-Gen Substrate Technology Will Revolutionize the Industry” Revolutionizing the Future: LG Innotek’s CEO Unveils Game-Changing Next-Gen Substrate Technology

    Inspira Technologies Secures Landmark $22.5M Deal: Major Revenue Breakthrough After FDA Clearance – Stock Titan

    Inspira Technologies Secures Landmark $22.5M Deal: Major Revenue Breakthrough After FDA Clearance – Stock Titan

    Meiwu Technology Company Limited and Shenzhen Zhinuo – GlobeNewswire

    Meiwu Technology Company Limited and Shenzhen Zhinuo – GlobeNewswire

    Owls inspire new revolutionary noise reduction technology – KTEN

    Owls inspire new revolutionary noise reduction technology – KTEN

    New center coming to Mizzou will focus on energy research and technology – Columbia Missourian

    Mizzou Launches Innovative New Center Dedicated to Energy Research and Technology

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
Earth-News
No Result
View All Result
Home Business

GPT, Other AI Models Can’t Decode SEC Filings, New Research Finds

December 20, 2023
in Business
GPT, Other AI Models Can’t Decode SEC Filings, New Research Finds
Share on FacebookShare on Twitter

New research conducted by a startup called Patronus AI shows that large language models (LLMs), similar to the one that powers ChatGPT, usually fail to decode Securities and Exchange Commission (SEC) filings.

Despite using OpenAI’s GPT-4-Turbo, the researchers only managed to get 79 per cent of answers right on Patronus AI’s new test, the company’s founders told CNBC.

With the ability to read nearly an entire filing alongside the question, GPT-4-Turbo was the best AI model configuration they tested.

Aside from refusing to answer, the so-called large language models would oftentimes “hallucinate” and come up with figures and facts that weren’t mentioned in the SEC filings.

“That type of performance rate is just absolutely unacceptable. It has to be much much higher for it to really work in an automated and production-ready way,” Patronus AI co-founder Anand Kannappan said.

Are LLMs really reliable?

Kannappan reposted an X (formerly Twitter) post by DoorDash’s Gokul Rajaram, noting “LLMs are nondeterministic”. In other words, they are likely to produce different answers for the same input.

LLMs are nondeterministic — they’re not guaranteed to produce the same output every time for the same input. That means that companies will need to do more rigorous testing to make sure they’re operating correctly, not going off-topic, and providing reliable results. This is what…

— Gokul Rajaram (@gokulr) December 19, 2023

So, it is safe to say that companies will have to be more careful when it comes to ensuring they are providing reliable results.

The latest findings further highlight some of the AI model-related challenges big companies, especially in regulated industries such as finance, face while trying to integrate this cutting-edge technology into their operations.

One of the most promising applications for chatbots has been their ability to extract crucial numbers and perform analysis on financial narratives.

Notably, SEC filings are teeming with important data, and if a ChatGPT-like bot could flawlessly summarise them or answer queries about what is in them, it could give the user a major advantage in the competitive financial industry.

Earlier this year, Bloomberg LP used the same underlying technology as OpenAI’s GPT to develop an AI model for financial data. Likewise, finance professor Alejandro Lopez-Lira showed that ChatGPT might come in handy for predicting stock movements.

Google is also working on a Gemini AI-powered program codenamed “Project Ellmann,” which will give users a “bird’s-eye” view of their lives. Moreover, McKinsey & Company suggest generative AI will radically overhaul how wealth management firms do business.

Despite the hype surrounding the newfangled technology, GPT’s entry into the industry has been pretty rough. When Microsoft launched its Bing Chat using OpenAI’s GPT, one of its primary uses was to quickly summarise an earnings press release.

However, some hawk-eyed observers realised that the numbers in Microsoft’s example were off. In fact, some of these numbers were entirely made up. In other words, Bing AI, which was recently rebranded to Copilot, made multiple factual errors.

How did AI models perform in the tests?

Patronus AI tested 4 language models including OpenAI’s GPT-4 and GPT-4-Turbo, Anthropic’s Claude 2 and Meta’s Llama 2. The company used a subset of 150 questions it had produced for the test.

The company also tested a slew of configurations and prompts, including a setting where the OpenAI models were provided the exact relevant source text in the question, which is known as “Oracle” mode.

The other tests involved instructing the models where the underlying SEC documents would be stored. Alternatively, the models were given “long context,” which is equivalent to providing an entire SEC filing alongside the question in the prompt.

GPT-4-Turbo

GPT-4-Turbo didn’t manage to pass the startup’s “closed book” test, where the model wasn’t given access to any SEC source document. Producing a correct answer only fourteen times, the model failed to answer 88 per cent of the 150 questions it was asked.

However, it performed better when given access to the underlying filings. In Oracle mode, GPT-4-Turbo answered the questions correctly 85 per cent of the time.

Llama 2

Meta’s open-source AI model had several hallucinations and went on to produce wrong answers 70 per cent of the time. It only managed to provide correct answers 19 per cent of the time when it was given access to underlying documents.

Claude 2

Anthropic’s Claude 2 performed well when the researchers included the entire relevant SEC filing along with the question. It answered 75 per cent of the questions accurately and gave wrong answers for 21 per cent of the queries it was asked.

Despite these shortcomings, Patronus AI co-founders believe language models like GPT can help people in the finance industry.

“We definitely think that the results can be pretty promising. Models will continue to get better over time. We’re very hopeful that in the long term, a lot of this can be automated,” Kannappan said.

>>> Read full article>>>
Copyright for syndicated content belongs to the linked Source : IBTimes – https://www.ibtimes.co.uk/gpt-other-ai-models-cant-decode-sec-filings-new-research-finds-1722295

Tags: businessmodelsOther
Previous Post

Liverpool FC Receive Massive Injury Boost Ahead Of Games Against West Ham, Arsenal

Next Post

Real Madrid Defender David Alaba Receives Offer For Assistance From Bayern Munich For ACL Injury

Downtown Louisville, Ky., gets new lifestyle hotel – hotelmanagement.net

Downtown Louisville Welcomes Exciting New Lifestyle Hotel

July 4, 2025
Romer: Finding common ground in a divided world – VailDaily.com

Romer: Building Bridges to Unite a Divided World

July 4, 2025
The Republican Budget Bill Is an Economy Killer – Bloomberg.com

The Republican Budget Bill Is an Economy Killer – Bloomberg.com

July 4, 2025
Kennedy Publishing, MGA Entertainment Launch Yummiland Magazine – License Global

Kennedy Publishing, MGA Entertainment Launch Yummiland Magazine – License Global

July 4, 2025
Trump says his 6th phone call with Putin this year ‘didn’t make any progress’ – PBS

Trump says his 6th phone call with Putin this year ‘didn’t make any progress’ – PBS

July 4, 2025
Environmental cognitive distance, R&D capability distance, and supply chain green technology innovation – Nature

Bridging Gaps: How Environmental and R&D Differences Drive Green Technology Innovation in Supply Chains

July 4, 2025
Penn State Ranks Inside Top Five in EA Sports College Football 26, QB Drew Allar Makes Rating Jump – Yahoo Sports

Penn State Ranks Inside Top Five in EA Sports College Football 26, QB Drew Allar Makes Rating Jump – Yahoo Sports

July 4, 2025
Church adds Mass ‘for care of creation’ to missal, pope to celebrate – usccb

Pope Introduces New Mass Dedicated to Caring for Creation

July 4, 2025
How UMich computer science students are navigating a shifting job market – The Michigan Daily

How UMich computer science students are navigating a shifting job market – The Michigan Daily

July 4, 2025
Genoa Central Junior High Student Places in the 2025 Soybean Science Challenge – TXK Today

Genoa Central Junior High Student Excels in 2025 Soybean Science Challenge

July 4, 2025

Categories

Archives

July 2025
MTWTFSS
 123456
78910111213
14151617181920
21222324252627
28293031 
« Jun    
Earth-News.info

The Earth News is an independent English-language daily published Website from all around the World News

Browse by Category

  • Business (20,132)
  • Ecology (704)
  • Economy (731)
  • Entertainment (21,619)
  • General (15,707)
  • Health (9,768)
  • Lifestyle (735)
  • News (22,149)
  • People (730)
  • Politics (738)
  • Science (15,947)
  • Sports (21,228)
  • Technology (15,714)
  • World (711)

Recent News

Downtown Louisville, Ky., gets new lifestyle hotel – hotelmanagement.net

Downtown Louisville Welcomes Exciting New Lifestyle Hotel

July 4, 2025
Romer: Finding common ground in a divided world – VailDaily.com

Romer: Building Bridges to Unite a Divided World

July 4, 2025
  • About
  • Advertise
  • Privacy & Policy
  • Contact

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

Go to mobile version