The Right People to Interview

We interview Neil Trevett and Tom Olson, heads of OpenCL and Vulkan working groups, respectively, about the decision to merge.

Last week, we reported that OpenCL’s roadmap would be merging into Vulkan, and OpenCL would, starting at some unspecified time in the future, be based “on an extended version of the Vulkan API”. This was based on quotes from several emails between myself and the Khronos Group.

Since that post, I had the opportunity to have a phone interview with Neil Trevett, president of the Khronos Group and chairman of the OpenCL working group, and Tom Olson, chairman of the Vulkan working group. We spent a little over a half hour going over Neil’s International Workshop on OpenCL (IWOCL) presentation, discussing the decision, and answering a few lingering questions. This post will present the results of that conference call in a clean, readable way.

First and foremost, while OpenCL is planning to merge into the Vulkan API, the Khronos Group wants to make it clear that “all of the merging” is coming from the OpenCL working group. The Vulkan API roadmap is not affected by this decision. Of course, the Vulkan working group will be able to take advantage of technologies that are dropping into their lap, but those discussions have not even begun yet.

Neil: Vulkan has its mission and its roadmap, and it’s going ahead on that. OpenCL is doing all of the merging. We’re kind-of coming in to head in the Vulkan direction.

Does that mean, in the future, that there’s a bigger wealth of opportunity to figure out how we can take advantage of all this kind of mutual work? The answer is yes, but we haven’t started those discussions yet. I’m actually excited to have those discussions, and are many people, but that’s a clarity. We haven’t started yet on how Vulkan, itself, is changed (if at all) by this. So that’s kind-of the clarity that I think is important for everyone out there trying to understand what’s going on.

Tom also prepared an opening statement. It’s not as easy to abbreviate, so it’s here unabridged.

Tom: I think that’s fair. From the Vulkan point of view, the way the working group thinks about this is that Vulkan is an abstract machine, or at least there’s an abstract machine underlying it. We have a programming language for it, called SPIR-V, and we have an interface controlling it, called the API. And that machine, in its full glory… it’s a GPU, basically, and it’s got lots of graphics functionality. But you don’t have to use that. And the API and the programming language are very general. And you can build lots of things with them. So it’s great, from our point of view, that the OpenCL group, with their special expertise, can use that and leverage that. That’s terrific, and we’re fully behind it, and we’ll help them all we can. We do have our own constituency to serve, which is the high-performance game developer first and foremost, and we are going to continue to serve them as our main mission.

So we’re not changing our roadmap so much as trying to make sure we’re a good platform for other functionality to be built on.

Neil then went on to mention that the decision to merge OpenCL’s roadmap into the Vulkan API took place only a couple of weeks ago. The purpose of the press release was to reach OpenCL developers and get their feedback. According to him, they did a show of hands at the conference, with a room full of a hundred OpenCL developers, and no-one was against moving to the Vulkan API. This gives them confidence that developers will accept the decision, and that their needs will be served by it.

Next up is the why. Read on for more.

What OpenCL Likes About Vulkan

The OpenCL working group, in the eight years since OpenCL 1.0 was published, have acknowledged some issues with the specification and how it was handled. First, they notice that they were unclear with whether they wanted to create a hardcore, explicit API, or if they wanted to make an easy-to-use one. This led to portions of the specification that fell in-between to two regions, and thus satisfies neither goal. Vulkan, on the other hand, took the stance that they should provide the former because middleware can handle the usability, and that higher-level layer can do a better job if they’re not wasting energy by having a boxing match with your attempts.

Neil: We did not focus rapidly enough on getting the middleware ecosystem in place. Now it is in place, there’s hundreds of systems using OpenCL, but, again, learning from Vulkan, where we had the middleware folks around the table during the design is key.

The second issue is that the Khronos Group has found OpenCL to be “over-monolithic”. To conform to OpenCL, you need to accept and implement the whole blob, even if your customers will never use it. Vulkan is designed to allow feature sets that can be enabled and disabled based on the needs and the underlying hardware. The main example that Neil used to illustrate this point is data types and precision; the developer and the implementation know what data structure maps best between problem and hardware, so why should the specification ignore both and blindly demand something arbitrary? A DSP that is designed for 16- or 24-bit fixed-point math, for instance, shouldn’t need to also support 32-bit floating point values just to check a box that no-one besides the specification authors care about.

Neil: This ((“this” is referring to slide 15 of Neil’s IWOCL presentation)) talks to the fact that OpenCL, its strength and its challenge, is that it’s addressing lots of different types of markets and lots of different types of processors, and the learning experience is that we’ve been over-monolithic. My favorite example is that there are multiple OpenCL implementations in Taiwan. Our Taiwanese friends, at the last meeting, they were telling me that they personally know of at least six different implementations that cannot be conformant because we insist upon IEEE 32-bit floating point, which is totally irrelevant to their target markets, like vision processing and neural network inferencing. Why? Why are we doing that? We need to let our implementors dial back on the precision that meets their market, and still be conformant.

They do want to keep the number of feature sets to a minimum, though, to prevent unnecessary fragmentation. Still, if an entire, important market cannot gain conformance because they need to implement something useless, then that’s a sign that something is broken on the specification’s end.

The third issue is the adoption of Vulkan versus OpenCL. On Apple platforms, the company has been heavily pursuing the Metal API. On Android, the Khronos Group claims that Google has “confused OpenCL and RenderScript”. That’s all Neil says about the issue, but it’s public knowledge that Google refused to ship OpenCL drivers on first-party (Nexus) devices, although they don’t prevent third-parties, like NVIDIA and Samsung, from shipping those drivers themselves. Lastly, they acknowledge that NVIDIA “hasn’t quite got to ((OpenCL)) 2.0 yet”.

Neil: I think Vulkan has a clearer adoption path on many of these platforms, so I think it will be better if we adopt and slipstream into Vulkan.

Lastly, from an organization standpoint, the Khronos Group liked the success that the Vulkan working group had with collecting feedback from non-members. They want to “ramp up” an advisory panel, like they had with Vulkan, where non-members can discuss the specification online, while under NDAs and an IP framework that allows the Khronos Group to take ideas and safely use them (from a legal standpoint).

What the Khronos Group Intends

Pushing OpenCL into Vulkan is a huge decision. While, again, OpenCL is the side doing the merging and the roadmap for Vulkan hasn’t changed, it could open up a large amount of potential for both APIs in the coming years and decades.

But that’s the future.

For now, the OpenCL working group is focused on how to make this change happen in a way that serves their users. They will need to implement OpenCL 2.x-class compute features into Vulkan, and they will need to add support for other devices, like FPGAs, DSPs, and so forth. They also intend to create a compatibility layer to support existing OpenCL 1.2 and OpenCL 2.x software. There is a lot of existing code that relies upon OpenCL, and bringing it forward is a high priority.

Neil: We have the Vulkan runtime. We have all the feature-setted blocks of functionality that can even be targeted at different markets. And then we have a very rich middleware ecosystem with languages and tools and, importantly, this is my main feature check with existing OpenCL users. We think we can get OpenCL backwards-compatibility with OpenCL 1.2/2.x with a compatibility layer. And the OpenCL community seems to trust us on that. So, and then we have their applications over that, so that’s kind-of the direction that we’re going.

Neil also mentioned that OpenCL could be tweaked in the mean time.

Neil: I think there’s some small tweaks we can do, like making FP32 optional for those DSP guys; we could do that in, like, 20 minutes. I think we should do that in parallel to the larger roadmap.

Clearing Up Lingering Questions

After the presentation, I was able to ask Neil and Tom a few questions. I started with the issues that I speculated about in the previous post.

Q: Has any legal concerns contributed to this decision? AMD provided Mantle “no strings attached” while Apple retains the OpenCL trademark and, in the past, has issued a notice that they would not license their patents for WebCL.

Neil: Honestly, that hasn’t been a factor. It hasn’t come up at all in any discussions. Yeah, the WebCL thing… that happened. But Apple has not been anything but supportive for OpenCL. They have not ever raised an IP disclosure over OpenCL itself… so… you know I can’t speak for Apple, but that hasn’t been a factor so far.

Q: Has the technical side contribute to this decision? Going in the other direction, you would basically need to re-invent Vulkan.

Neil: That’s the important thing. That’s not what we’re doing. Vulkan is going to continue. We’re not trying to layer one over the other. We haven’t decided or discussed this yet. This is our next step. Now that we know that people would like to do something like this, we have to figure out what exactly it is, and how we’re going to do it.

My personal mind-model is that we can eventually work to a place where we do have a Vulkan framework and Vulkan exists as it is and, as Tom said, it’s continuing to meet its market needs. But we can enable people who are more interested in compute to toggle one bit to turn off the graphics, and toggle another bit to turn on the more advanced compute. Now there’s a million details, we might need to work through many stages to get something that clean and nice, but, no, that’s the kind of model that many people would like to work towards. So we’re not layering one over the other.

Q: Has Vulkan been eyeing SYCL?

Tom: SYCL is very much an OpenCL thing. (…) The OpenCL group discussed this ((the decision to merge into Vulkan)) at the last meeting and they said, “we’d like to do this” and we said “that sounds fine – go nuts”. The Vulkan working group, in the mean time, is working on the other stuff: supporting the current API and working on the next one as we do.

Q: So SYCL has never really been considered?

Tom: That’s right.

Neil: And, again, this hasn’t been discussed, I’m not saying that we’ve decided or talked about this… but there’s some people who are interested in using C++ as graphics shaders. And kind-of the work that OpenCL has done on C++, if the Vulkan working group decides to do that, they could move faster because of what the OpenCL working group did in the past. But, again, there’s no decisions yet.

Tom: Absolutely. I think the Vulkan working group’s view is that we’re trying to present a clean model that you can do what you want with, rather than telling people how to write their code. We want to give them the tools. And so, from our point of view, if SPIR-V has what you need to do GLSL or HLSL or C++ as your shading language, that’s good, and we won’t get in the way of that, but we’re trying to not dictate a particular solution. So we think this is great, and, if it gives developers more options, we’re very happy about that.

Q: Was NVIDIA’s decision to add parts of OpenCL 2.0 to their drivers related to this announcement? Any comments to make?

Neil: I can clarify for NVIDIA. I think this is all public. We are fully conformant on OpenCL 1.2. We’ve added some OpenCL 2.0 functionality. It’s the functionality that our customers are actually asking us for. We’ve kind-of satisfied our developer needs, but we’re not OpenCL 2.0 compliant. This, from an ecosystem point-of-view, is an issue, but NVIDIA customers are happy. I think that situation kind-of probably informed our decision. We need to be less monolithic. It’s not quite the same thing, but it’s in the same direction. OpenCL in its current monolithic form… you kind-of have to go… it’s all or nothing. Even if, in your customer segment, you know that it won’t add any value. And why do I spend engineering resources when I know none of my customers are going to use it. I would like to be OpenCL 2.0 compliant. We kind-of painted ourselves into this monolithic corner. It’s not just NVIDIA, DSPs and there’s lots of examples. That kind of thinking has been one of the influencers into why we want to go into this much finer control slice and dice feature set direction. So we don’t get ourselves into that weird situation anymore where people want to support the spec and be conformant, but they just can’t afford or have more high priority things to do than to implement stuff no-one is going to use.

Scott (Me): Like DSPs that do, for instance, 16- or 24-bit audio having to implement 32-bit floating point.

Neil: Right. Exactly. Particularly, sort of the hotspot applications are vision processing and inferencing where it’s not just even that. It’s 16-bit… perhaps you can even get away with 8-bit… perhaps you can even get away with 4-bit. The fact that we’re insisting on 32-bit floating point is a little bit out of sync with the needs of the industry there. And that’s something we want to fix.

Q: What is your biggest concern with this transition, and why?

Neil: If I was an OpenCL user, I think my biggest question would be, “Are you sure you can do backwards compatibility with that layer?” Now we think we can, it might not be 100-point-0 percent… but we think we can get close enough that applications won’t notice any holes in the emulation layer. This is obviously something we have to work through. We set that as our key goals… whatever this new generation of OpenCL looks like, we have to not leave the existing OpenCL ecosystem behind. That’s goal number one. But I think we can do it. But yeah, that would be my questions. And we’ll know pretty soon I think… if there are going to be issues or not.

And then the other one is… how do we organize (in Khronos) to make this happen? We need Vulkan to keep moving at full speed. So we need to find an organization that OpenCL can do its thing, and Vulkan can do its thing, and we end up moving faster, not slower. So we have some thinking to do on what’s best to organize that.

Bonus: Brief comments on OTOY

Around two-thirds of the way through the interview, I asked a question that I decided to leave until now.

Q: The press release mentioned that there would be some compute applications that would benefit from graphics tasks. Which market segments? Media ISVs, like Adobe and Blender Foundation? Science and research groups? Enterprise software, like financial institutions? Super-computing? And why?

Neil: The poster-boy for this, and they’ve been tweeting about it publicly… (although) I don’t want to put words in their mouth, is OTOY. (…) That’s certainly one example where, is it a compute application? Is it a rendering application? It’s both. They want to use the graphics machinery and they want more flexible compute than they can get in the graphics APIs. They’ve been encouraging us to consider how to increase the compute flexibility while still accessing the graphics pipeline. You know OTOY, they do awesome work, so it’s fun to understand what it is those guys actually need.

When I published the previous post, I started asking around to get some opinions about how interesting OpenCL merging into Vulkan would be. I tweeted to Tim Sweeney of Epic Games, because he was vocal about C++ on GPUs and circumventing graphics APIs since, like, 2004. Somehow, OTOY’s Twitter account came across this tweet, and weighed in that SYCL, single-source C++, through Vulkan would be “Huge”.

After hearing Neil bring them up, I asked OTOY if they had any further statements, and they responded.

I should note that OTOY is not a member of the Khronos Group. I’m just bringing them up because they have an interesting perspective, and their public comments seem relevant to this post.