- Academy Color Encoding System (ACES)
- What is ACES ?
- ACES color spaces
- What about ACEScg ?
- The ultimate rendering space
- ACEScg comparison
- Display Target
- ACES in one picture
- Linear confusion
- Input Display Transform (IDT)
- Why do we get an extra bounce ?
- IDT (Input Display Transform) description
- To plot the gamut
- Reference Rendering Transform + Output Display Transform (RRT + ODT)
- ACES, OCIO and CTL
- All paths lead to ACES
- Maya and ACES
- Resolve and ACES
- Guerilla Render and ACES
- Nuke and ACES
- Substance and ACES
- Asset conversion
- Color Picking
- Photoshop and ACES
- Albedo definition
We have seen in the previous chapter about Color Management prior to 2014. I am now going to describe the revolutionary system developed by hundred of professionals under the auspices of the Academy of Motion Picture Arts and Sciences.
Spoiler alert : ACES is available through OCIO (here is the link to the ACES 1.1 config), just like the spi-anim config (seen in the previous chapter). ACES has also been implemented in CTL in Resolve and GLSL / HLSL in Unreal and Unity.
More than 300 movies have been done using ACES and big VFX studios such as ILM, MPC, Double Negative and Animal Logic use it. ACES has also become a delivery standard for Netflix. The whole idea behind ACES is to set a standard to help professionals in their color management.
If you don’t feel like following by another technical chapter, I don’t blame you. You can skip to chapter 2.
Academy Color Encoding System (ACES)
Something that really hit me when I arrived at Animal Logic in 2016 was their range of colors. The artists were working on a beautiful and very saturated movie called Lego Batman. It was my first day and I saw this shot on a screen (I think Nick Cross lit this shot).
I really thought to myself : “Wow ! This looks good ! How did they get these crazy colors ?” The range really seemed wider than my previous studio. I realized later that it was due to ACES :
- Many studios render within the sRGB gamut with a linear transfer function and display in sRGB through a 1D LUT. That is not ideal as they work in the smallest gamut of all.
- Animal Logic (and many other studios such as ILM or MPC) render in ACEScg (which is similar to Rec. 2020) and display in DCI-P3 which is the industry standard for cinema theaters. ACES helps them to manage their gamuts in the best way.
What is ACES ?
ACES has been developed by the Academy of Motion Picture Arts and Sciences, some VFX Studios (MPC, Animal Logic…) and the Camera Manufacturers (Arri, Red, Sony, Canon…). The idea behind it is pretty genius.
When cameras were analog, things were simple. There were only a couple of formats : 35mm and 70mm. The Original Print, shot on film, was available for eternity.
But with the digital revolution, multiple cameras and formats have emerged. These proprietary systems, used for Digital Cinema Distribution Master (DCDM), could be outdated quite quickly. Indeed, the technology of digital cameras evolves pretty fast. Issue is when these movies got to be remastered for new supports, the DCDM were not relevant anymore.
ACES is a series of color spaces and transforms that allows you to manipulate them. It is currently the most powerful tool in Color Management. The reference color space developed by the academy is called ACES2065-1 (AP0 primaries).
It is simply the ultimate color space and here are its characteristics :
- Ultra Wide Gamut
- High Dynamic Range
With ACES2065-1 (AP0), the idea is to get a DCDM (Digital Cinema Distribution Master) for eternity. We do NOT know how movies will be watched in 50 or 100 years. ACES has been created for this specific reason : its purpose is to last in time !
ACES color spaces
The color space ACES2065-1 (AP0) includes most color spaces. This is why it will last for a very long time. Here is a list of all the color spaces available in ACES :
- ACES2065-1 (AP0) : for file interchange, DCDM and archival format.
- ACEScg (AP1) : for rendering and CG work. It is a working color space.
- ACEScc / ACEScct (AP1) : for color grading. They are working color spaces.
- ACESproxy (AP1) : for camera playback and video displays. It is a transport color space.
There is an absolutely brilliant post about the different color spaces available in ACES. Here is the most valuable information :
- ACES 2065-1 is scene linear with AP0 primaries. It remains the core of ACES and is the only interchange and archival format.
- ACEScg is scene linear with AP1 primaries (the smaller “working” color space).
- ACEScc, ACESproxy and ACEScct all have AP1 primaries and their own specified logarithmic transfer functions.
- ACEScct is very similar to ACEScc across most of the range, except that it adds a “toe” to make it more akin to traditional “log” curves (e.g. Cineon, LogC, S-Log, etc.).
Please note that the ACES2065-1 color space is not recommended for rendering. You should use ACEScg (AP1). More explanations are provided right below.I will mostly focus on ACEScg in this post since my book is about CG projects.
What about ACEScg ?
What about Computer Graphics ? How ACES can benefit our renders ? Alex Fry explains it really well in this video. Some tests have also been conducted by Steve Agland from Animal Logic and Anders Langlands to render in ACES2065-1.
An unexpected issue occurred when rendering in the ACES2065-1 color space : AP0 was so big that it gave some crazy saturated values. It is very well explained in this post.
Therefore, another color space has been created especially for CG rendering : ACEScg (AP1). It is more or less the same as Rec. 2020. I will repeat in bold and with emphasis because it is CRITICAL : you should only render in ACEScg.
What is the difference between ACEScg and Rec. 2020 ? What is the advantage to have the green primary out of the CIE diagram in ACEScg ? To encompass P3 mostly, ACEScg is a gamut close to BT.2020 but that encompasses P3.Thanks Thomas for the explanation !
The ultimate rendering space
Why would we render in one color space and display in another ? What is the point ? Remember the Rendering Space and the Display Space ? That they do NOT have to be the same. It is something that surprises a lot of people but, yes, rendering within different primaries will NOT give the SAME result.
Rendering in Linear – sRGB or ACEScg will not give the same image. Many supervisors have told me : “What is the point in rendering in a different color space if we do NOT have the monitor to show it ?” But they are mistaken. It makes complete sense to render in one color space and view it in another.
Basically, we want to use the most appropriate color space to get the best render possible : ACEScg. That is called Wide Gamut Rendering ! How do we know that this particular color space is the most appropriate ?
To perform such a test, you need a reference called the “Ground truth“. In our case, it would be a render with no bias like spectral render engine Mitsuba. Otherwise you could not compare objectively the renders.
- Rec. 709, the smallest gamut of all.
- Spectral, the ground truth using wavelengths for its calculation.
- Rec. 2020, which is pretty much equivalent to ACEScg.
Then, you subtract them from one another. The darkest it gets, the closer to Spectral ! Just brilliant ! The technical reason behind it is given in a series of article and post :
From Thomas Mansencal : On a strictly technical point of view, rendering engines are indeed colourspaces agnostic. They just chew through whatever data you throw at them without making any consideration of the colorspace the data is stored into. However the choice of colorspace and its primaries is critical to achieve a faithful rendering. […]
So basically, if you are using textures done within a sRGB colorspace, you will be rendering in this particular colorspace. Only the use of an Input Display Transform (IDT) will allow you to render in ACEScg.
From Thomas Mansencal : […] some RGB colorspaces have gamuts that are better suited for CG rendering and will get results that overall will be closer to a ground truth full spectral rendering. ACEScg / BT.2020 have been shown to produce more faithful results in that regard. […] Yes, the basis vectors are different and BT.2020 / ACEScg are producing better results, likely because the primaries are sharper along the fact that the basis vectors are rotated in way that reduces errors. A few people (I’m one of them) have written about that a few years ago about it. […] Each RGB colorspace has different basis vectors as a result of which mathematical operations such as multiplication, division and power are not equivalent.
The display is based on your project :
- Do you work in advertising for TV and Internet ? You should display in sRGB or Rec. 709.
- Are you working in Feature Film ? Well you should display in DCI-P3.
- Do you want to output for an UHDTV ? You should display in Rec. 2020.
Rec. 2020 is clearly the future but there are no monitors that are able to cover 100% of this color space. The technology is not there yet. But in ten years maybe, it will be the new norm. And the best part is that when this day comes, ACES will still be a proper solution since it includes most of color spaces.
To sum it up :
- Render in ACEScg to get the best lighting calculation possible, even if our monitors are not capable of displaying it.
- Display in Nuke or RV the result in your display target, the most suitable to the project.
- Use a monitor that covers your needs (100% coverage of DCI-P3 for feature film).
ACES in one picture
- A. IDT is the import/conversion of the images to the ACEScg color space.
- B. ACEScg is the Rendering Space.
- C. RRT + ODT are the LUTs output to any monitor or video projector.
The RRT and ODT splines and thus the ACES system tone scale (RRT+ODT) were derived through visual testing on a large test set of images […] from expert viewers. So no, the values are not arbitrary.From Scott Dyer, ACES mentor.
The idea behind ACES is to deal with any color transform you may need :
- Is your texture in sRGB from Photoshop ? Or is it in linear within the sRGB gamut ? ACES provides all the matrix and LUTS you need to jump from one color space to another in the IDT (Input Display Transform).
- Does you monitor cover Rec. 709 or DCI-P3 ? ACES provides all the LUTs to view your renders with the most appropriate display transform.
This is why I love ACES so much : you always know in which gamut you are. Another argument I was given against ACES was : “We don’t care about ACES, we render in linear.“
That is really the original sin from the VFX industry : thinking that linear was a color space. The error is still quite common, even among VFX industry veterans. And even some softwares are wrong about it !
In Nuke, this is particularly disturbing : linear is listed as a color space, same as sRGB or Rec.709. In the support pages from The Foundry, you can find an explanation : “However, Nuke’s colorspace isn’t a standard colourspace.” Guys, this should be written in BIG RED LETTERS ! Thanks to ACES, all of this confusion gets clarified !
In the version 2.0 of this book (release in September 2020) I will try to address the LMT process of ACES. Sounds like an interesting topic !
Input Display Transform (IDT)
Here are two renders of a Cornell Box in Guerilla Render. I have used pure green at 0/1/0 and pure red at 1/0/0 for both renders.
Why do we get an extra bounce ?
Thanks to ACES we have remapped the primaries and we got an extra bounce in our render. We can do the same thing in Nuke to analyze what is actually happening :
- On the left, we have a pure green constant at 0,1,0.
- We convert it to the ACEScg color space using an OCIOColorSpace node.
- Some red and blue have been added to the primary (right image).
Here is another way of explaining it :
- On the left, we have a green primary in the Rec. 709 color space.
- Using a 3D LUT to switch from sRGB to ACEScg, this color with unique XY coordinates has been modified.
- The color is not a pure green anymore in the ACEScg color space (right image).
IDT (Input Display Transform) description
ACES provides all the 3D LUTs and Matrix we need to process these transforms. This is why it is so powerful !
Most common IDT for Computer Graphics are :
- Utility – sRGB – Texture : If your texture comes from Photoshop or Internet. Only for 8-bits texture, like an albedo map.
- Utility – Linear – sRGB : If your texture is linear within the sRGB primaries and you want to convert it to ACEScg.
- Utility – Raw : If you do NOT want any transform applied on your texture, like normal maps.
Most studios nowadays work in “lazy ACES” because of the lack of OCIO in Substance. It means that we actually paint textures in a sRGB gamut and convert them on the fly to ACEScg in the render engine.
Something that took me some time to understand as well is that if your rendering space is ACEScg, in this particular case, Utility – Raw and ACEScg are the same IDT. No transforms are applied with both options.
To plot the gamut
Plotting the gamut of an image allows you to map its pixels against the CIE 1931 Chromaticity Diagram. That is a pretty brilliant concept ! And a great way to debug ! This function is available in colour-science, developed by Thomas Mansencal.
- On the first image, we have plotted a render done in Rec. 709. The pixels are clearly limited by the Rec. 709 primaries. They are compressed against the basis vectors of its gamut.
- On the second image, we have plotted a render done in ACEScg. The pixels, especially the green ones, are not limited anymore and offer a much more satisfying coverage of the gamut.
Reference Rendering Transform + Output Display Transform (RRT + ODT)
The academy recommends the use of an ODT adapted to our output. Many artists have been confused by Nuke’s Display Transform :
- Why does sRGB display transform and sRGB (ACES) do NOT match ?
- Because the ODT of ACES includes some tone mapping !
From ACEScentral, Nick Shaw explains :
The ACES Rec.709 Output Transform is a much more sophisticated display transform, which includes a colour space mapping from the ACEScg working space to Rec.709, and tone mapping to expand mid-tone contrast and compress the shadows and highlights. The aim of this is to produce an image on a Rec.709/BT.1886 display which is a good perceptual match to the original scene.
Here is the secret recipe of why ACES looks so good ! Check the highlights on the second render, they just look amazing !
Another description of the ODT tone scale can be found here. I have also done a test on the MacBeth chart to compare the Film (sRGB) from the spi-anim config with the ACES config. The results speak for themselves.
ODT technical description
What does exactly happen when we view in Rec. 709 (ACES) with an OCIO config ? To go to Rec. 709 (ACES), OCIO first transforms the color to ACES2065-1 (AP0 primaries). Then from AP0 we go to a colour space called Shaper thanks to a 1D LUT and finally to Rec.709 thanks to a 3D LUT.
ACES, OCIO and CTL
Most color pipelines nowadays are set through OCIO which is great because of its compatibility with many softwares : Maya, Guerilla, Nuke, Mari, Rv… But there is one downside using OCIO and LUTs : you loose precision. It is really well explained in this post.
The reason the CTF works better is because it is more of an “exact math” implementation of the ACES CTL code rather than baking down into a LUT-based representation.From Doug Walker, ACES mentor
Discrete and Continuous transforms
What is happening here ? The answer has been given to me by my colleague, Christophe Verspieren. He has showed me the concept of Continuous and Discrete that is happening with the baked LUTS from OCIO. It is actually pretty easy to understand. Check this image from this site :
When we go from Scene Referred to Display Referred, it implies to cover a huge dynamic range (ACES deals something like 15 stops). The discrete transform actually covers huge zones. We do not split the dynamic range into equal zones as we prefer to split in detail must current values at the expense of highlights. Therefore the display tone mapping (ODT) makes these false chromaticities really visible by decreasing the exposure.
Also even if the transformation is mathematically defined in OCIO, the fact that it runs on GPU rather than on CPU leads to a discretization of the formula : the graphic card actually creates a LUT !
Color interpolation gaps
Furthermore, these gaps (from the discretization) are filled linearly which is not necessarily the most natural way. Even if we change gamut, we still work in RGB and linear interpolations are done on the line going through A and B. Sometimes it would be better to manage color interpolation in Lab colorspace which will only be supported in OCIO 2.0.
Not every mathematical formula is available in OCIO 1.0, only OCIO 2.0 will allow to represent correctly the necessary calculation of ACES.
Between each slice we have linear interpolations. In very large areas the interpolation lacks accuracy. This results in errors of chromaticites in highlights due to the discretisation. How does this translate visually ? Let’s have a look at some renders with extreme saturation to compare different solutions.
All paths lead to ACES
How important is this ? Should we sick to OCIO or look into this CTL implementation ? I guess the best answer I have read on this topic comes (once again !) from Alex Fry :
With normally exposed images, especially from live action cameras, you’re unlikely to see any values that render differently to the OCIO config, but with CG images containing extreme intensity and saturation you will see more pleasing roll off and desaturation.December 2016 and Alex Fry had already understood so much stuff…
So it looks like CTL would be worth using especially if you are working on saturated animated feature films ! To sum it up, here are my personal ACES recommendations :
- ACES 1.0.3 : very good config but some limitations as we have in the previous examples.
- ACES 1.1 : improved version thanks to Thomas Mansencal who increased the shaper dynamic range.
- OCIO 2.0 and ACES 2.0 should bring a perfect match to AMPAS CTL as stated in the roadmap. Beta testing will start after Siggraph 2020.
- ACES CTL has been implemented through SynColor into Maya. No color shifting.
If you’re keen on going down the CTL road, you may ask : how can I maintain a color management chain without OCIO ? That is a very tricky question. I don’t have the answer yet but Alex Fry was kind enough to share with us this Pure Nuke ACES RRT & ODT.
Quick update : the Pure Nuke ACES node has been updated by Nick Shaw to work in P3D65, Rec.709, Rec.2020…Thanks Nick !
These Nuke nodes have a 100% match to a pure CTL implementation but way faster. Please note that these nodes work with ACES2065-1 footage. I will also provide more information about CTL for Digital Intermediate (DI) for version 2.0 of the book. Stay tuned !
Maya and ACES
The Color Management Module from Maya is pretty much explicit :
By default, Maya assumes an sRGB display although it is possible to use any of the other ACES display options by editing the synColorConfig file (please see the user guide).
Resolve and ACES
All ACES transforms have been implemented in Resolve with CTL for better accuracy.
I will detail better ACES in Resolve in December 2019.
Guerilla Render and ACES
In three easy steps, you can setup ACES in Guerilla Render. Just follow the guide ! We use a .bat file to load the OCIO Config and open the software :
We set the “Gamma” (should be called “Rendering Space“) as “ACES – ACEScg” and the Display Transform as “Rec.709 (ACES)”. I have a personal preference for Rec. 709 (ACES) compared to sRGB (ACES), as it is less contrast and has a softer look.
There is one final step. Here is the trick : we add a MaterialOverride” node at the end of the Render Graph to override the Input Display Transform (IDT) for the Color entries. It allows to switch their input. Thanks to Gaetan Jayle for the tip !
- All grayscale maps will be loaded as “ACEScg” in Guerilla Render which is fine because they are linear maps with no information of color. Which means we dot not need to worry about their primaries.
- All color maps will be loaded as “Utility – sRGB – Texture” which is easier for us since our output for these textures is sRGB.
Please note that the primaries of your CG render must be ACEScg (AP1).
Nuke and ACES
ACES has been integrated natively in Nuke since version 10 if I recall correctly. Nuke 12 integrates the ACES 1.1 OCIO config and the Foundry has even removed the spi-anim config !
For final delivery of your project to the Digital Intermediate, you will have to deliver ACES compliant EXR. This is the standard set by the Academy to exchange files between facilities. This is really important. Your output of 3D render will be ACEScg (AP1) but your output of Nuke has to be ACES2065-1 (AP0) with the correct metadata.
The interchange and archival files should be written as OpenEXRs conforming to SMPTE 2065-4. In Nuke, you should set the colorspace to ACES2065-1 and check the box write ACES compliant EXR to get the correct metadata.
Substance and ACES
Substance softwares are not OCIO friendly yet. But there is a workaround available to work with an ACES ODT.
For example, Substance Painter uses a Color Profile (exr) as a display LUT. This Color Profile must also contain the transfer function adapted to your display device. Basically the LUT for Substance Painter is a 2D texture. Its generation is documented on Allegorithmic website.
If you are not familiar with this method you can also generate the texture in Nuke by using the image reference provided by Allegorithmic. You just need to apply a colorspace transform from Utility – Linear -sRGB -> Output – Rec.709 using an OCIOColorSpace node.
In the slide above, I detail two methods that will give the exact same result. I just wanted to show you both ways. Thomas Mansencal provides the Nuke nodes in this post.
One of the most frequent questions I have been asked is about converting asset. How should we deal with assets created in linear – sRGB to convert them to ACEScg ? Here is an example I had to face recently. Let’s say you work on a movie with a famous character dressed in red. Generally, the client will be very picky about this red value. It has to match !
I am pretty sure someone smarter than me could automatize this process or improve it. But I haven’t found a better way for the moment. I have also used the same technique for the following example. The challenge was to maintain the same colors from a concept painted in Photoshop into an ACES render.
I totally accept the fact that this concept has some lighting information. So it can become arbitrary to pick a color in one place or an other. I tried to be as accurate as I could.
It took me a while to understand the color picking role in the ACES config. But after six months thinking about it I have finally cracked it. More explanations will be provided before Christmas 2019.
Photoshop and ACES
ACES and Photoshop are giving headaches to many people and I wanted to describe a logarithmic workaround we have used for a couple of years now. I understand it is not as Photoshop friendly as an ICC profile but I know it would be helpful to some artists and students. We will start this workflow in Nuke and then move to Photoshop.
ACES uses two logarithmic color spaces : ACEScc and ACEScct. If you want to know more about the differences between them, just follow the link.
I know it is not a perfect workflow but I think it is an easy workaround for artists who are not familiar with ICC profiles like myself.
In the version 2.0 of this book (release in September 2020) I will try to describe the ICC profile workflow of ACES and I’ll also try to use ACES2065-1 exr files in Photoshop.
Photoshop can read a csp LUT. This is a display LUT which must contain the transfer function adapted to your display device. Generally Matte-Painting will be worked in logarithmic in Photoshop and a display LUT is compulsory to display the image correctly.
You can simply generate it in Nuke thanks to the nodes CMSTestPattern, OCIOColorSpace and GenerateLut. Here is a slide to show you how :
So where do I start ? I have set Guerilla Render with an ACES OCIO config and I have some geometry ready to be surfaced and textured. I would personally begin with the albedo color.
From Substance PBR guide : The visible color of a surface is due to the wavelengths emitted by the light source. These wavelengths are absorbed by the object and reflected both specularly and diffusely. The remaining reflected wavelengths are what we see as color.
From Jeremy Selan : In a real scene, if you measure luminance values with a tool such as a spectroradiometer, one can observe a very wide range of values in a single scene. […] Very dark materials (such as charcoal) reflect a small fraction of incoming light, often in the 3-5% range. As a single number, this overall reflectivity is called “albedo”.
Albedo chart for ACES
The Luminance data comes from Nuke’s viewer and is in linear. I have ordered the albedo values by luminance as I thought it would be more convenient. The chart itself has been written as a jpg file with the following colorspace “Utility – sRGB – Texture”.
My process to generate this chart was pretty simple. I merged one chart posted on ACEScentral, one from Sebastien Lagarde and the macbeth color checker into one. Since most of these charts have sRGB values between 0 and 255, I used this converter to normalize them. I had to dig a bit to find the macbeth values in sRGB. Then in Nuke, I used an “OCIOcolorspace” node to convert from “Utility – Linear – sRGB” to “ACES – ACEScg”. And voila !
I sometimes find shaders with the 0 value in albedo. I agree that the 0 value can be an optimization since there is no value nor bsdf to evaluate. But that’s really not the way to go in PBR.
I have been wondering for a while if there was any study between albedo and saturation. I will describe here before Christmas 2019 how the Pointer’s gamut can be helpful in this matter.
Everyone should work with ACES. As you may have noticed, I am a big fan of ACES and hopefully it will be more and more used. ACES has been developed by hundreds of industry professionals and is available for free. There is no valid reason NOT to adopt it. ACES has many advantages :
- Compatibility through OCIO with many softwares.
- Free and lot of support from the community acescentral.
- To ensure a lighting calculation in the best gamut : ACEScg.
- Less guess work and quality jump with the amazing Display Transforms.
- To generate a Digital Cinema Distribution Master (DCDM) that will still be valid in many years.
In all my testing, I never had a case where ACES would not make something look better. It is such an improvement of quality : everything looks more real and behaves more correctly. We get so much closer to a photographic approach with ACES. All the hard work has been done by the Academy of Motion Picture Arts and Sciences (AMPAS). And it is up to us, CG Supervisors and Cinematographers, to spread the word !
Here are my thoughts on ACES. This is an extensive topic and some people would explain differently. In the setup I have described we actually use three different gamuts : sRGB for texturing, ACEScg for rendering and ACES2065-1 for delivery. We can now move to less technical chapters and focus on cinematography. Yay !
- Alex Fry presentation at Siggraph 2015. It clearly shows you the visual benefits of ACES and of a photographic approach of CG. There is an article as well.
- These three articles that explain ACES were helpful to me.
- These articles from Autodesk also helped me to understand the application of ACES in CG. Framestore is using ACES as well.
- You can find a list of IDT here and some explanation about LMT there.
- An interesting introduction to ACES in Resolve, Fusion and After Effects.
- If you want to read more about ACES and ACEScg. This slide is really good as well.
- If you need a reminder about wide gamut rendering, these posts will enlighten you. Steve Agland also did some interesting tests. And Thomas of course !
- If you are curious about the ODT, check these two posts or simply ACEScentral.
- If you want to get very technical about ACES, this document is made for you.
- Of course nothing impeaches you to render in spectral like this example in Octane.
- Siggraph 2017 has this interesting but quite technical paper about spectral and ACES rendering.
- ACEScg Digipro from 2015. And more articles from Duiker Research.
- The FrameIO blog about ACES.
- Unreal Engine explains very well the use of ACES. Very good explanation !
- Birds of a feather Siggraph 2019 about ACES 2.0.
- A website about color : colorpipe and two videos in French : représentation de la couleur and Academy Color Encoding Specifications.