'Feels like magic!': Groq's ultrafast LPU could well be the first LLM-native processor — and its latest demo may well convince Nvidia and AMD to get out their checkbooks

(Image credit: Groq)

Groq, led by ex-Google engineer and CEO Jonathan Ross, claims to have created the first ever Language Processing Unit (LPU) which it says can deliver the fastest speeds for AI applications.

It’s a bold claim, but one that the latest demos more than back up, suggesting it could well become an absolute game-changer for AI.

Ross, who previously designed Google’s tensor processing unit (TPU), launched Groq in 2016 to create a chip capable of executing deep learning inference tasks more efficiently than existing CPUs and GPUs.

Lightning fast

The company’s Tensor Stream Processor (TSP) is likened to an assembly line, processing data tasks in a sequential, organized manner. In contrast, a GPU is akin to a static workstation, where workers come and go to apply processing steps. The TSP’s efficiency became evident with the rise of Generative AI, leading Ross to rebrand the TSP as the Language Processing Unit (LPU) to increase its recognizability.

Unlike GPUs, LPUs utilize a streamlined approach, eliminating the need for complex scheduling hardware, ensuring consistent latency and throughput. LPUs are also energy efficient, reducing the overhead of managing multiple threads and avoiding underutilization of cores. Groq’s scalable chip design allows multiple TSPs to be linked without traditional bottlenecks, simplifying hardware requirements for large-scale AI models.

The first public demo of Groq was a lightning-fast AI answers engine that generated answers with hundreds of words in less that a second. Matt Shumer posted the test on X and says more than 3/4 of the time was spent searching not generating.

The first public demo using Groq: a lightning-fast AI Answers Engine.It writes factual, cited answers with hundreds of words in less than a second.More than 3/4 of the time is spent searching, not generating!The LLM runs in a fraction of a second.https://t.co/dVUPyh3XGV https://t.co/mNV78XkoVB pic.twitter.com/QaDXixgSzpFebruary 19, 2024

While that’s impressive, watching Groq go head to head with Chat-GPT is something else.

If you want to try Groq for yourself, to get an idea of just how fast it can be for AI, go to this chat page. Use the drop down on the left to switch between the different available models.

More from TechRadar Pro

These are the best AI tools around todayMany top firms are eager to embrace AI — but their business just isn’t readyNvidia’s fastest AI chip ever is finally available for preorder

Sign up to the TechRadar Pro newsletter to get all the top news, opinion, features and guidance your business needs to succeed!

Wayne Williams is a freelancer writing news for TechRadar Pro. He has been writing about computers, technology, and the web for 30 years. In that time he wrote for most of the UK’s PC magazines, and launched, edited and published a number of them too.

>>> Read full article>>>
Copyright for syndicated content belongs to the linked Source : TechRadar – https://www.techradar.com/pro/feels-like-magic-groqs-ultrafast-lpu-could-well-be-the-first-llm-native-processor-and-its-latest-demo-may-well-convince-nvidia-and-amd-to-get-out-their-checkbooks