Metal performance in OS X El Capitan: Sometimes great, often mixed [Updated]

Update: Kishonti Ltd, creators of GFXBench, have contacted us to say that a new build of GFXBench Metal for OS X will be available soon and that it may change performance. We'll run the new version of the test and update this article as soon as it's available.

Update 10/27/2015: Kishonti has contacted us to say that GFXBench Metal for OS X will be released "later this year," but won't provide further details on how we can expect performance to change in the public version. We'll continue to monitor this situation as it develops.

Original story: OS X El Capitan includes a new graphics API called Metal, the same API the company included in iOS 8 last year. Metal is roughly analogous to technologies like DirectX 12 and the not-quite-finalized Vulkan, designed to improve performance by reducing driver overhead.

We don’t have many real apps to analyze Metal with on OS X, since it's still a brand-new operating system, and the kinds of apps that would benefit (some games, GPU-dependent or GPU-accelerated professional apps like AutoCAD or Photoshop) don’t usually pick up support for this stuff on day one. We have, however, managed to get our hands on a Metal version of the GFXBench graphics benchmark, which we can compare to the OpenGL version to draw some (admittedly limited) conclusions about Metal's improvements.

We performed our testing primarily on four Macs with a variety of GPUs. We've also included results from an iPhone 6S as a reference point for how Metal performs under iOS.

• A 2012 Mac Mini with a Core i7-3615QM and an Intel HD 4000.
• A 2012 iMac with a Core i7-3770 and an Nvidia GeForce GTX 680MX.
• A 2015 Retina MacBook Pro with a Core i7-4770HQ and an Intel Iris Pro 5200.
• A 2015 Retina MacBook with a Core M-5Y31 and an Intel HD 5300.

This list is weighted pretty heavily in favor of Intel’s integrated graphics, but this reflects the makeup of Apple’s modern Mac lineup—the company has switched to Intel’s GPUs for all but its highest-end Macs.

In iOS 8 and iOS 9, comparing the Metal version of GFXBench to the OpenGL ES version shows off near-universal improvements in the high-level tests with more mixed results in the low-level tests. This was generally the case in OS X, but the wider variety of hardware meant a wider variety of test results. Many tests showed OpenGL holding even, or even beating Metal, though tests that were limited specifically by driver overhead showed some nice gains.

These results also varied across GPUs. Faster GPUs tended to show larger relative gains compared to slower ones—our best explanation is simply that faster GPUs can offload more work from the CPU, resulting in more impressive gains.

Let’s quickly explain what all of these tests are doing. Manhattan and T-Rex are both high-level tests that render a full 3D scene. Manhattan is the more demanding of the two, and it uses more complex lighting and effects. The ALU, Driver Overhead, and Texturing tests are low-level ones that attempt to measure some specific aspect of graphics performance.

For whatever reason, the OpenGL version of GFXBench consistently beats the Metal version in the T-Rex, ALU, and Texturing tests. And usually it’s not winning by a little, it’s winning by a lot. We’ve asked Apple about these results, but the immediate takeaway is that there are some things that OpenGL is going to do better than Metal can, at least for now.

Metal, on the other hand, excels where you would expect it to. The Driver Overhead test “approximates the graphics driver’s CPU load when running the Manhattan high-level test,” which is right up Metal’s alley, since one of the API’s goals is to process more on the GPU so that the CPU is free to handle physics or AI.

Gains in this test are nearly universal but vary widely from GPU to GPU, going up 35 percent on the HD 4000, 53 percent on the Iris Pro 5200, and a whopping 609 percent for the GeForce GTX 680MX. The HD 5300’s Metal tests were unfortunately still lower than the OpenGL tests, possibly a sign that its low-voltage CPU needed to throttle down under load.

The Manhattan test’s scores also generally go up when run under Metal instead of OpenGL, a sign that it benefits more from low-overhead APIs than the T-Rex test does. Again, the effect was more pronounced on faster GPUs, particularly the Nvidia model—it's improved by nearly 50 percent, compared to 8.5 percent for the Iris Pro and 6.4 percent for the HD 5300.

What conclusions can we draw from these kind-of-all-over-the-place numbers? First, Metal won’t buy you magical universal performance improvements, especially if you’re using an older or weaker GPU like the HD 4000 or HD 5300. In the best of cases, those parts showed small improvements in Metal, but more often the scores stayed around the same.

On a related note, we can say that Metal stands to gain more on computers with faster GPUs, especially those with dedicated GPUs. Our three-year-old iMac’s Nvidia GPU showed the biggest gains in the tests Metal excelled in, which makes sense. The more the GPU can do, the more it can free up the rest of the system to do other things.

And finally, Metal isn’t an API you’ll want to use to do everything. There are many cases where OpenGL still performs better despite the fact that Apple’s implementation is out of date, and there are other cases where Metal might perform a little better but may not be worth the extra programming complexity that comes with low-overhead APIs. We saw the same kind of mixed behavior when we looked at Microsoft’s DirectX 12—everything from the kind of thing you’re processing to the GPU you’re processing it on can change the results drastically, and the low-overhead API doesn’t always come out ahead.