(Dec 3): Amazon.com Inc’s cloud unit raced to get the latest version of its artificial intelligence (AI) chip to market, renewing efforts to sell hardware capable of rivalling products from Nvidia Corp and Google.
The accelerator, called Trainium3, was recently installed in a few data centres and will be available for customers beginning on Tuesday, Dave Brown, a vice-president with Amazon Web Services (AWS), said in an interview.
“As we get into early next year we will start to scale out very, very quickly,” he said.
The chip push is a key element of Amazon’s strategy to stand out in AI. AWS is the largest seller of rented computing power and data storage. But it has struggled to replicate that dominance among leading developers of AI tools, as some companies opt instead to rely on Microsoft Corp, which has close ties to ChatGPT maker OpenAI, or Alphabet Inc’s Google.
Amazon shares had risen 1.6% to US$237.71 ($308.01) at 11.09am in New York. Nvidia shares pared gains, while AI chip rival Advanced Micro Devices Inc dropped to a session low.
Amazon hopes to entice companies looking for a bargain. Trainium chips are capable of powering the intensive calculations behind AI models more cheaply and efficiently than Nvidia’s market-leading graphics processing units, according to the company. “We have been very pleased with our ability to get the right price performance with Trainium,” Brown said.
See also: OpenAI’s Altman urges ‘code red’ ChatGPT push as rivals move in — Bloomberg
Amazon is releasing Trainium3 about a year after deploying its predecessor accelerator. That’s a sprint by chip industry standards. “The main thing we are gonna be hoping for here is just that we don’t see any kind of smoke or fire,” an AWS engineer joked when the chip was first fired up in August.
The quick turnaround also matches the pace of Nvidia, which has pledged to release a new chip every year.
There’s a catch: Amazon’s chips lack the deep software libraries that help customers get Nvidia’s graphics processing units up and running fast. Bedrock Robotics, a company that uses AI models to enable construction equipment to operate autonomously, runs its infrastructure on AWS servers. But when it comes to building models to help guide an excavator, Bedrock uses Nvidia chips, according to chief technology officer Kevin Peterson.
See also: AMD-backed Vultr to build US$1 bil chip cluster in Ohio for AI
“We need it to be performant and easy to use,” Peterson said. “That’s Nvidia.”
Many of the Trainium chips in use today are at the disposal of Anthropic, inside data centers in Indiana, Mississippi and Pennsylvania. AWS said earlier this year that it had strung more than 500,000 of them together to help the AI start-up train its latest models and aims to dedicate one million of the chips to Anthropic by the end of the year.
Amazon is betting Anthropic’s success, along with its own AI services, can entice other companies. Amazon has announced few other major customers for the chip, leaving analysts struggling to assess Trainium’s effectiveness.
Anthropic is also using Google’s tensor processing units and cut a deal earlier this year with the Alphabet unit that gives the start-up access to tens of billions of dollars worth of computing power. In an interview with Bloomberg Television, AWS CEO Matt Garman said the company’s relationship with Anthropic was “incredibly strong”. He said the start-up has an enormous demand for computing power, leading Anthropic to use a variety of providers.
Amazon made the chip announcement at re:Invent, its annual user conference, which in recent years has become a rolling advertisement for AI services where Amazon courts builders of cutting-edge tools and companies that might want to pay for access to them.
On Tuesday, Amazon also announced updates to its main line of AI models, called Nova. New Nova 2 products include a variant called Omni, which can receive text, image, speech or video inputs and respond with both text or images.
As with its chips, Amazon has tried to sell customers on the performance of its models for their price. Prior Nova models generally haven’t ranked among the industry leaders in benchmarks that track how AI models perform in answering standardised questions.
To stay ahead of Singapore and the region’s corporate and economic trends, click here for Latest Section
“The real benchmark is the real world,” Rohit Prasad, who leads much of Amazon’s model development and the company’s artificial general intelligence team, said in an interview, adding that he expected the new models to be competitive.
The company also plans to let customers bring more data to bear when customising Amazon’s models. Nova Forge, a new product, is designed to let sophisticated users grab versions of Amazon’s Nova models before their training is complete and customise them with their own data.
Reddit Inc is using Nova Forge to build a model capable of assessing whether a post on the digital message board violates the site’s safety policies. Chris Slowe, the company’s chief technology officer, says that some AI customers are tempted to use the most advanced model to solve every problem, rather than seeking one with a particular expertise.
“The fact that we can make it an expert in our specific area is where the value comes from,” he said in an interview.
Uploaded by Tham Yek Lee
