The high-end GPU market for top-of-the-line graphics processing units for real-time graphics is mostly invisible to the general public. Like all progress, the innovations in silicon designs for GPUs happen in a cycle that typically requires around 18 to 24 months to complete a full circle from the initial design to the mass production. To design such highly complex products, large amounts of money have to be invested in research and development. However, the largest portion of these investments is spent on designing the silicon, the actual heart of the GPU, which, in turn, will also benefit lower-end GPUs as well as other semiconductor components, by virtue of the new silicon architectures for increased computing power for real-time graphics processing.
Silicon architecture drives the performance ceiling
New high-end GPUs are very different from those from five years ago, especially when it comes to the silicon architecture. The biggest change of late is the increase in the number of specialized Processing Units, or Execution Clusters, within a die. These are similar to the Programmable Shaders that existed in graphics cards from the early 2000’s, but are much more complex and are designed to handle a variety of tasks in parallel.
Flagship GPU’s are now packed with specialized processing units (SPUs) or co-processors that are designed to handle specific functions. This list of functions includes, but is not limited to, traditional video encoding (now commonly found in GPU’s and offloaded from the CPU) as well as other tasks such as upscaling lower resolution video using deep learning for enhanced detail and real time AI enhanced rendering. In the case of real time ray tracing, the GPU processes large numbers of rays in a 3D scene in real time. Also, in real time, the GPU can be used to encode 8K video in HEVC using dedicated hardware for such a task. Frame generation using deep learning for enhanced detail is also supported by such a GPU.
Flagship GPUs have a huge number of transistors. The die size for a current flagship GPU is around 600 mm2 and it contains between 50 billion and 80 billion transistors. To put this in perspective, the entire CPU from 2010’s Intel Core i7-980X had fewer than 2 billion transistors. The number of transistors is less important than how they are organized into different components and how these components communicate with each other.
Memory bandwidth becomes the new battleground
Processing power means nothing without data to process.
The amount of memory bandwidth that a high-end GPU can access has increased dramatically in the last few years, exceeding 1TB/s in some cases. However, the memory bandwidth that a GPU can access is only part of the story when it comes to the memory system of a GPU. In addition to the memory bandwidth, there is also the memory compression and the caching hierarchy. To optimize the memory system of a GPU, the engineers that design it must make predictions as to what data the GPU will need to access in a few milliseconds. If the data that the GPU needs is in the cache, then there is no delay. However, if the prediction fails, then even the fastest memory will not be able to compensate for the delay.
Manufacturing precision reaches physical limits
It’s not good enough to develop the most advanced GPU silicon architecture if it can’t be made in sufficient volume and to a consistent standard using the latest manufacturing process. The flagship GPUs are currently made using 4nm and 5nm manufacturing processes, contain tens of millions of millions of transistors, and are just a few dozen atoms wide in places.
As transistors are reduced to smaller and smaller sizes on the silicon die, a number of very difficult to manage problems begin to appear. For one, even before a bit is written in a memory location, it can be altered by cosmic radiation passing through the atmosphere to the Earth’s surface. This is a well known problem in space systems where it can cause significant problems in the memory of a computer. In addition, as the size of the silicon transistor is reduced, the barrier between the source and drain of the transistor can become so small that electrons can tunnel through the barrier. This problem, which is due to the quantum mechanical nature of the electron, is becoming increasingly difficult to manage. However, the biggest problem that is encountered with decreasing the size of the transistors on the die is the problem of temperature variation. Even small variations in temperature across the surface of the die can have a profound impact on the operation of the transistors on the die.
An example of such current flagship GPUs is the NVIDIA AD102 GPU. The NVIDIA AD102 GPU chipset is an example of a current flagship GPU. It is built using a 4nm silicon production process and contains over 76 billion transistors. The die of the GPU is very large, measuring roughly 600 mm2, while the individual transistors are as small as 4nm in size, which translates to as few as 50 atoms across. Designing and manufacturing such a die is one of the most complex challenges in the semiconductor industry, and the companies that have the capability to design and manufacture such a large piece of silicon for commercial sale are truly few in number.
Software defines hardware capability
The specifications of a GPU define its potential. However, the software is used to unlock this potential. In other words, without the right software, even the fastest graphics processing unit is unable to reach its full potential. For that reason, the software for a flagship GPU is as important as the hardware itself. The drivers for flagship GPUs contain millions of lines of code and are often optimized for specific game engines.
Flagship GPUs are developed with a degree of anticipation with designers trying to read the tea leaves as to what applications software developers are likely to release in the future and designing the GPU to be as efficient as possible at processing those functions. One area that this was particularly evident was in the early adoption of ray tracing cores. The early high-end GPUs to include ray tracing cores, the NVIDIA GeForce GTX 2080 and GTX 2070 were launched in September 2018, and for the first year or so after launch the use of the cores was effectively ‘shelf ware’ as there were no game engines that were able to take full advantage of the functionality of the cores. In fact, it wasn’t until very recently, within the last 12 months or so, that a number of high-end titles have started to include native support for ray tracing, titles such as Assassin’s Creed Odyssey and Control.
Tensor Processing Units (TPUs) in Flagship GPU Chipsets – Future Hardware in Current Graphics Cards As previously mentioned, in current flagship graphics cards, Tensor Processing Units (TPUs) can be found in order to process large amounts of information in real time for machine learning (ML). So far these processing units are mainly used for image upscaling and frame generation in games. However with current development of GPUs and TPUs, there is a large potential for more complex AI-driven graphics in the future, which current flagship GPUs can already process with currently developed software.
Performance scaling hits diminishing returns
The GPU development of flagships is dwindling, for two reasons. First, the power consumption to achieve higher performance is not increasing in the same linear fashion as the processing performance. Instead, it’s increasing by a factor of three. The consequence is a massive increase in heat, for the semiconductor industry to deal with, as they continue to try and push the limits of how large silicon die can be. There are also physical limits to the size of semiconductor die that can be economically manufactured. Indeed, there are already very real limits to the size of die that can be created and marketed today.
Just like there are many physical restrictions for die size, there are also some limits to the function per wafer (FPW) of larger dies. Due to that, the GPU industry has reached a physical boundary for the single GPU die size already.
This emerging trend will provide future high-end GPUs to be composed of several individual chips, which will be packed together into a single housing and function as a single GPU. Due to current limits of silicon fabrication it will not be feasible anymore to design large single-die GPUs in a cost-efficient manner. Future GPU architectures therefore will use several smaller dies, which are stacked on top of each other in a 3D fashion, so called die-stacked or multi-chip designs. An efficient packaging solution will become as critical as the individual dies themselves. The challenge for future GPU designers will be to connect all these individual chips together at near-silicon speeds in order to minimize any performance penalty.
