Nvidia has revealed that the H200 is a graphics processing unit designed to train advanced artificial intelligence models supported by generative artificial intelligence technology.
The new graphics processing unit represents an improvement over the previous H100 GPU, which was used by OpenAI to train the large language model GPT-4.
Major companies, startups, and government entities are competing to obtain limited quantities of the chips.
The cost of the H100 ranges from $25,000 to $40,000 and requires thousands of units to be connected together to train large linguistic models.
The increasing trend towards Nvidia’s AI-supporting graphics processing units has led to a more than 230% increase in its stock value so far in 2023.
The company expects to achieve revenues of around $16 billion in the third quarter of this year, with an increase of up to 170% compared to last year.
The key improvement in the H200 is having a 141 gigabyte next-generation HBM3 memory chip, aiding it in executing inference or using a large linguistic model after training it to generate texts, images, or predictions.
The memory bandwidth of the graphics processing unit is increased to 4.8 terabytes per second from 3.35 terabytes per second in the H100, to enhance its capability in processing intensive tasks for generative artificial intelligence.
The company indicated that there is almost a doubling in speed in the H200 chip compared to the H100 chip in producing results. This chip is expected to be provided to customers in the second quarter of 2024, competing with AMD’s MI300X GPU.
The AMD MI300X chip boasts a larger additional memory size compared to previous models, making it suitable for running huge models for inference purposes.
The H200 is compatible with the H100, meaning that companies relying on AI and using the H100 to train large linguistic models will not have to change their server systems or software to benefit from the H200.
According to Nvidia, the H200 is available in configurations of quad graphics processing units, or eight GPUs through the company’s full HGX systems, in addition to a chip called GH200, which connects the H200 GPU to an Arm-based processor.
