Performance Tweets Series: Multi-GPU

Multi-GPU systems are much more common than you might think. Most of the time, when someone mentions mGPU, you think about high-end gaming machines with multiple graphics cards or high-end, dual-GPU boards. However mGPU is much more than that: many modern PCs have an integrated GPU and a dedicated GPU on a graphics card. With Direct3D® 11 a driver profile is often required to take advantage of multiple GPUs e.g. via Alternate Frame Rendering (AFR). Unfortunately AFR has become increasingly difficult to implement because modern games often rely on temporal algorithms – such as reprojection techniques for temporal anti-aliasing. Even with a driver profile or by integrating the Crossfire API one is typically only able to take advantage of symmetric mGPU configurations (identical GPUs with equal performance). In most case an heterogeneous setup (GPUs from different vendors or with widely varying performance) is unable to scale efficiently.
Fortunately, the new explicit APIs attack this problem head on. With Direct3D 12, explicit mGPU is now part of the core API so developers can implement their own mGPU work distribution. There’s no more driver magic trying to optimize the game using heuristics. Now, with explicit control, developers get the power to implement the best possible mGPU support: they can take advantage of new ways for work distribution like split-frame rendering (SFR), offloading post-processing to integrated GPUs, and finally, schedule just-in-time copies of data needed for temporal algorithms.

Independent adapters

Direct3D 12 exposes mGPU in two separate ways: LDA and MDA. LDA is the “linked” adapter mode, which means you have multiple identical GPUs which are linked together into one adapter with multiple nodes. MDA is the unlinked mode, which works for any set of different GPUs. MDA mode is also the same level of sharing you get on Vulkan. MDA is also the most flexible mode in terms of what GPUs can work together, but it comes as the cost of efficiency. Because the GPUs involved may use completely different image tiling formats, only few image layouts can be shared – typically, row major only. That’s a format you can’t really use for anything besides copies as it’s very slow to sample or to render to. At the same time, MDA allows you to take advantage of GPUs from different vendors, and is also the way to go if you want to support configurations like an integrated GPU paired with a discrete GPU.

Linked adapters

The alternative to MDA is LDA – and unfortunately there’s no equivalent in Vulkan for this yet. In LDA mode, only identical (or nearly identical) GPUs are linked together. A typical example are dual-GPU cards and multi-GPU setups using identical cards. As the cards in LDA mode are guaranteed to support the same features and the same tiling formats, you can exchange textures directly between them without having to resort to basic tiling formats or copies through system memory. Additionally, you can set up a shared swap chain where the driver and runtime will take care of synchronizing data between the GPUs instead of having to issue a manual copy between them. In this mode, a symmetric work-load distribution usually works best, like AFR if latency is not an issue, or SFR if you want to minimize latency at the expense of slightly lower efficiency.

In any case, the key to good efficiency on multiple GPUs it to minimize the amount of data you need to copy, and to copy as early as possible. All copies should be executed on the copy queue, as you want them to saturate PCIe bandwidth and run while the GPU continues working on the frame. In some cases, it can be beneficial to duplicate work instead of re-using data – particle simulations for example. Especially for small amounts of data – where there is significant overhead in initiating a copy – recomputing per GPU is recommended.

That’s it for today! If you have questions, feel free to comment or ping us on Twitter: @NThibieroz & @NIV_Anteru.

Tweets

• 09: Explore ways to utilize explicit MGPU: modern CPUs contain a GPU that you can offload work onto.
• 19: Understand both MGPU LDA and MDA modes and their respective strengths & weaknesses.
• 29: In MGPU LDA mode, prefer to share the swap chain between GPUs instead of copying the back buffer manually.
• 38: MGPU: Consider other work distribution schemes than just AFR (Alternate Frame Rendering).