China’s announcement of its latest supercomputer demonstrates that the US ban on GPU sales to China has had minimal effect. What challenges has China faced, what has it developed, and why does this finally show that bans are highly ineffective?
The Uphill Battle For China
Over the past few years, the US has introduced massive restrictions against China in an attempt to cripple Chinese capabilities on bleeding edge technologies. These restrictions, also followed by the rest of the Western world, have been designed to limit China’s access to the latest GPUs, which has hindered their ability to develop new AI capabilities.
However, this restriction has gone way beyond AI development, even having restricted China’s ability to build the fastest supercomputers, relying on older generation tech. In fact, restrictions on sale of key electronic components have also managed to cause issues in the production of modern hardware, especially in the military space.
But regardless of the uphill battle that China faces, it hasn’t stopped Chinese researchers from developing their own technologies. To no surprise at all, the very pressure that these restrictions have put on China has resulted in Chinese companies looking towards open-source hardware and software solutions.
RISC-V, for example, has seen exponential growth over the past two years, going so far as to cause the Chinese government to actively push its use as the next-generation CPU architecture to use in Chinese made systems. Of course, these technologies are several generations behind what is currently available in the West, but being entirely independent allows for China to continue on with their developments, unfazed by any technological restrictions.
China Demonstrates New Massive Supercomputer With 1.54 ExaFLOPS of Performance
Recently, China’s National Supercomputing Center announced the deployment of their latest supercomputer, called “LineShine”, that is designed for both AI and HPC applications. The new system, according to reports, has a maximum theoretical BF16 AI training performance of 1.54 ExaFLOPS.
Sources close to the development of the system have reported that the new system was developed in direct response to the US banning export of GPUs to China. To achieve such capabilities, the new supercomputer utilises 40,960 custom Armv9-based LX2 processors, and has believed to be manufactured by either Huawei or a joint Chinese HPC initiative (due to extreme secrecy inside China, the true origin is not precisely known as this time). Each of these processors integrates 304 Armv9 CPU cores with Arm SVE accelerators and SME accelerators, specifically designed for running AI matrix and vector operations.
Furthermore, each processor supports multiple datatypes including FP64, FP32, BF16, FP16, and INT8, while the total number of operations per second that the device can perform is not currently known. The system as a whole is estimated to provide 1.54 ExaFLOPS of BF16 AI performance, but individual per-core or per-processor PFLOPs figures as described are not confirmed.
At the same time, the new supercomputer also utilises an unusual hybrid memory architecture that combines on-package HBM with off-package DDR5. The use of HBM provides a high-bandwidth low-latency memory solution while the DDR5 provides a large external memory space.
During a recent demonstration, the system was shown to train a 6.3 billion-parameter generative model of Earth observation data at 2.16 ExaFLOPS, proving that GPUs are not the only solution for training large models.
Why Does this Show that Bans are Ineffective?
It goes without saying that the US has been hellbent on impeding Chinese capabilities for years, and in doing so, has restricted its ability to develop new technologies. But, it seems that the US has gone too far in this direction, as it appears that the very act of trying to prevent Chinese capabilities has had the opposite effect.
Instead of backing down and simply accepting whatever restrictions the US throws at it, the combination of desperation and fear has seen both the Chinese government and its researchers pour all their resources into developing new technologies. Furthermore, the desire to catch up with the west also gives a sense of urgency to those working in the field, recognizing that they need to be able to compete with the rest of the world.
This development of a CPU only supercomputer should thus come as no surprise, considering that people will always find solutions to problems. In this case, the problem was to create a supercomputer capable of running and training AI models, and the solution was to create a processor specifically suited to that task.
So, it doesn’t matter if NVIDIA releases new CUDA technologies, or if Intel creates a new CPU, China will certainly not be looking back, and will instead look towards its own devices, solutions, and services.
