- The CG issue
- What is cheating ?
- Light decay
- Top Light
- Warm edge
- Luminous surfaces
- IES Lights
- Technical papers
The CG issue
We are going to review some techniques related to our medium. 3d softwares give us a tremendous amount of freedom but bring as well some constraints. How should we deal with those in an efficient way ? Should we put some limits ? It is not because you can create as many lights as you want that you should. With great power comes great responsibility.
What is cheating ?
It is actually a pretty hard question to answer to. Because cheating does not have the same definition at Illumination, Animal Logic or Ilion. It really depends on the studio and your supervisor : it is cultural.
If I look into a dictionary, cheating means acting dishonestly to gain an advantage.
But what does cheating mean in CG ? Does it mean I acted dishonestly ? In relation to what ? Reality ? Is using a blocker a cheat ? Is light-linking a cheat ? Or should we give artists the most adapted tools to their medium ?
I do not like this word because it makes me feel guilty for doing my job. No I did not cheat, I did what I had to do to make it look good.
“Hitchcock […] couldn’t get clean focus on the gun and on Ingrid Bergman in the same frame. So he used a giant gun and a huge fake hand to get the effect that he wanted.”
Lord of the rings
Peter Jackson used “forced perspective” a lot on Lord of the rings (DP : Andrew Lesnie). In the pass of Caradhras, he used a ring of 18-inch circumference when Frodo has fallen and the ring is lying in the snow.
The forced perspective trick is very well explained in the making-of.
After seeing these examples, one would think that directors of photography cheat all the time on set. Really ? Did they act dishonestly to gain an advantage ? I don’t think so. I would rather use the word : enhancing. DPs manipulate and control their environment to enhance the reality.
It is NOT a cheat, it is a trick or a gag. On set DPs would generally use those words a lot. I know it may sound like a stupid detail but vocabulary in lighting is essential. And similar tricks are also used in lighting ! Even for a “basic” daylight exterior, DPs will use reflectors, black flags and many other tools on set. We should be able to control every element. It’s what DPs do.
Issue in CG is that many supervisors do not want to give control to their team. They have seen so many horrors and crazy setups that they want to lock everything. I don’t blame them. But sups have to find the right balance between control and safety. A good way to empower a team rather than controlling it is training.A team should not feel controlled by the CG police every time they add a light.
A famous Disney movie
One of the best stories I have ever heard. Get ready.
On a famous Disney movie, Rob Legato did a “trick” with a time lapse of a night sky. They went and shot a night sky out here in the Mojave desert. Rob also did a shoot with a light box and some paper with holes in it. Brought it into nuke and enhanced it, to something that was a bit more pleasing to the eye. Things like readability of the stars against a post dusk sky. He put them both in front of Jon Favreau in our review room and asked him to pick the one he liked best.
Here comes the best part !
Without telling him what he had done and what was different. And Jon picked the one he shot with a light box and his explanation was : the stars feel real to me. It’s how I remember seeing stars coming out after dusk and getting brighter as the night goes on. It reminds me of my camping trips as a kid. Rob then tells him it’s fake and everyone bursts out laughing, even Jon ! It was pretty amazing to watch and Rob didn’t say much. He didn’t have to. But capturing how people remember things and what they hold on to is an art.
And this brilliant conclusion.
Our job is about perception more than reality. Reality can be boring. But people perceive things differently and that’s what DPS compare it to when they gauge how “real “something is.Brilliant Ted talk from Rob Legato about this topic if you want to know more.
Spider-man : Into the Spider-Verse
Loved the movie like everyone else. There is probably a project of this quality and originality every ten years or so. I just wanted to share a couple of screenshots from the making-of.
A strong art direction driving every department. Impressive !
Andre Turpin, DP of Playmobil : “We use tricks as much as we can. We put lights on the walls, the ceiling and even the floor sometimes.”
Aymeric Montouchet, DP of Detroit: become human : “The only thing we should not cheat in CG is the quadratic decay of light.”
Good and bad tricks
Now we have seen that “cheating” is a terrible word, we have to wonder about the main differences between CG and live action. I totally agree that we should work like Digital Directors of Photography. But let’s not forget that each medium has its own properties, rules and limitations. I personally see two BIG differences between live-action and CG :
- Render engines’ limitations aka we are not doing spectral rendering.
- Sampling and render times aka this is not real time.
This is why I distinguish two types of tricks : good and bad. Render engines give us so many possibilities that it is very easy to do crazy stuff. We have to find a middle ground between the freedom we will give to the artists and the control we want to have on their methods.
Examples of good tricks :
- Add lights so the shot relies less on indirect lighting ? YES.
- Use a blocker to have some vignetting in a shot ? YES.
- Use light-linking to make your character pop ? YES.
Examples of bad tricks. This is very personal by the way. You may disagree and I am happy to know what you think.
- Use light-linking to compensate for poor surfacing ? NO.
- Remove the walls from shadow casters to get more light into the room ? NO.
- Start a Master Lighting with different lights for characters and sets ? NO.
- Change the decay of a light. NO. It has to be quadratic. Except Sun and Sky obviously.
- Use some ambient lighting. NO. This is just wrong.
- Have non-physical albedo values. NO. Luminance has to be between 0.03<x<0.9.
Ambient lighting has now (almost) disappeared thanks to PBR. The easiest way to describe it would be an EnvLight with no occlusion. I guess it is only used in video games.
In the introduction, I was explaining that this book is more focused on PBR productions. So if you base things off a “ground truth” or reality, yes, we can use the word “cheat“. The software is essentially building a model in which to emulate light in the real world with things like decay, bounce, global illumination…
But this word is so negative. We should never make an artist feel guilty for having created a couple of extra lights. And if you want your artists to work in a certain way, because of art direction or render times, it is totally understandable. But train them so they know.
My lead at Animal used to tell me : “Whatever works”. Is it that simple ? If you only care about your shot, yeah that would do the trick. But lighting for me is about finding viable solutions that you can easily share with the team. Lighting is about building physical environments that stand correctly. If you start to remove walls, objects, link them together, exclude some others, what are you trying to achieve ? You are just destroying the plausibility of the CG world.
We have seen in a previous chapter that the Sun could either be a Spot or a Distant light. What visual difference does it make ? Actually a big one, because of the decay. The Lighting design for stylized animation paper explains pretty well light falloff.
From Dave Walvoord : The natural falloff from light sources is another way to add depth to shots. The brightness of light diminishes in an inverse squared manner […]. Therefore, if you want a lot of falloff then a light should be placed close to the subject. This has the effect of increasing depth between objects. A light placed farther away has less falloff and will compress the space.I personally prefer the term of decay. But it is very personal. Once again vocabulary is a very important part of our job.
Great explanation and examples in this pdf.
The characters and camera are in the exact same place, and only the light is moved [in the renders above]. In the first image, the light is far away from the characters and the depth between them is compressed. In the second image the light is brought closer and there is a large amount of falloff from Hiccup in the front to Stormfly in the back. This produces the effect that the characters are farther away from each other. This technique can be used to great effect to change the depth of the shot.
A supervisor once told me not to put any lights close to the character. I guess he is right if your goal is to tie the characters together. But it actually depends on what you want to achieve. Do you want to tie your characters together or do you want to separate them ? Both options are possible.
Many artists have asked me these questions about decay : how do you manage if you want to have a very strong light on an element but it does not illuminate too far behind it ? Is there a case where you want to force a faster attenuation of light than the quadratic ?Answer is pretty straight-forward : never.
Light-linking is when a light illuminates only certain objects. It is NOT the same as shadow linking, when you remove some shadow casters from a light. A proper light-linking will keep all the shadow casters as they are.
I have tried to find an example where light-linking (in this case it is lighting in layering but that is essentially the same thing) is a bit extreme. The characters seem to be in a different world than the set.
Linking on The Star
Since our workflow allowed it, there was basically light-linking on every shot of The Star. Most of the time my job would be to clean things a bit and simplify light rigs.
I haven’t worked on this sequence but it kinda represents the limits of dramatic lighting without any lighting justification.
We did output 250 shots per week on this project.
I have thought a lot about light-linking and right or wrong. In my opinion it does not make sense to say : light-linking is crap or I hate light-linking. Every studio has its culture and its way of doing things.The real question being : is it used properly ?
Linking on Lego Batman
For example, I did plenty of light-linking on Lego Batman because the art direction was very graphic. I did not use any on The secret life of pets because of its natural look. The only conclusion I could come with is : Light-linking is based on art direction. Honestly, it would have been impossible to do Lego Batman without it.
I generally try to avoid light-linking. I only use it if I do not have ANY other options. But it does not mean that it is not a good solution. Depending on the art direction, you may have to rim your characters like crazy. In this case, YES, light-linking is the only solution.
But I would not start a Master Lighting using it by default. When I begin a sequence, I use lights that affect everything. It is only after dailies and based on the Supervisor’s comments that I will eventually use light-linking.
Extreme light-linking on Planet 51
And if you ask me, below is the most extreme light-linking I have ever done in my life. I don’t think I have ever told anyone before. Chuck the astronaut has one sun for his body and one for his face.
I was not able to get the proper look without doing it this way. But I guess it’s okay because I don’t think you can spot it. Visually it looks like one sun.
One thing to keep in mind is that the rims cannot be stronger than the source. Because of quadratic decay, the light fades with distance.
The only advice I could give you about light-linking is from my own experience. It is very personal so take it with a grain of salt. It actually took me a while to realize that. I only do additive light-linking.
I always start with the lights affecting everything. And then I would eventually add a couple of lights only on certain elements. I try to have the highest common denominator for my light rigs. It is very rare for me to remove an object from a light influence.
Volumetric rendering refers to a technique for generating a visual representation of data that is contained in a three dimensional space (volume). Examples of this are atmospheric effects such as smoke and fog […]. In marching cubes, data is represented as voxels, but in the sense that we store some floating point value in a structured grid in three dimensions. Volume rendering is contrasted with Surface rendering because the data that is being rendered is coming from a three dimensional data set rather than a two dimensional data set (a surface).Great explanation from this website. Thanks guys !
My personal preference to work on my volumetric is to render it interactively in my master layer. It allows me to check easily its density and color. If you tweak your volumetric shader with all the geometry in matte, you may get some surprises later in comp.
- I work interactively with everything in the scene.
- But I render in separated passes for beauty and volumetric.
- I generally start with my beauty lights affecting the volumetric. It is easier to set the density this way.
- Once I am happy with the density, I’ll see if I need to create special volumetric lights or not.
I have forced the rays on purpose to make this example obvious.
My former lead Victor Pajot used to tell me : volumetrics pass always look nice (because of the black background). The real challenge is when you merge them with the beauty.
This example is not ideal but hopefully you get the idea.
Volumetric rendering is a bit technical and can possibly give headaches, even to senior artists. One of the most important parameter is the density of your volume. Because it will change greatly its look and its alpha.The Arnold website about volumetrics is very well written and easy to understand if you want to read more on the subject.
Merge of volumetrics
Should we merge in ‘over‘ or ‘plus‘ the volumetric pass ? I know that ‘over‘ is the correct way and we should definitely try to respect this. BUT production realities sometimes make it impossible. If I am in rush and my volumetric pass is too dense, I’ll do what I have to do.
If you feel like your volumetric is darkening too much your beauty, you have several options :
- Decrease the volumetric density (which will have an effect on its alpha).
- Increase the volumetric shader color.
- Increase the light exposure.
- If you cannot re-render, you may grade the volumetric pass. Rule of thumb would be to grade color (RGB) and alpha (A) at the same time to keep it consistent.
On Lego Batman, we used the depth pass. A lot. I am pretty sure half the shots have borders since the depth pass was not defocused nor anti-aliased. It is better to do the fog effect in CG if possible.The art direction is so strong and edit so fast that you do not notice anything.
Color separation with volumetrics
In the previous paragraphs we have seen a technical description of the volumetrics rendering. But I really to emphasize the fact that volumetrics are also a wonderful tool for the cinematographer.
It is funny how I have changed my mind about this topic. After Pets, I would probably have said stuff like : do not create any special lights for this pass. It is really important that volumetric and beauty have a visual connection.
Grant and Craig really made me grow as an artist and helped me realize that volumetrics (dust, fog and haze) are a tool for the cinematographer. It will help you to direct the viewer’s eye, enhance drama, create depth, complement the composition or make a character pop.
I have been looking at the topic of volumetrics and its use in movies. Something I have been noticing (and that Grant uses perfectly) is the use of cold volumetrics to create depth.
Even if it may not seem technically correct : if my sun light is warm, so should be my volumetric, right ? I firmly believe that it is artistically correct to use cold volumetrics in most cases. Or better said : to use a complementary scheme between the beauty lighting and the volumetric lighting.Since the sun is warm, plenty of daylight sequences have cold volumetrics in them.
We will see examples of this trick in the following slides.
Examples of volumetrics
Lego Batman and volumetrics
First example comes from Lego Batman (Director : Chris McKay, PD : Grant Freckelton). I just love this shot. Pay attention to the use of volumetric.
This is one of my favorite tricks : see how this blue volumetric helps to read the microwave ? There is no doubt about what batman is doing. Otherwise it would just be a dark microwave over a dark wall and it would be hard to read the action. It is the most perfect dramatic volumetric light to make the microwave pop. Awesome !
The choice of the blue color is not genuine neither. We are seeing here a perfect example of a complementary scheme : the microwave area (foreground) is warm and the wall area (background) is cold. It helps splitting the character from the background and gives more depth to the image.Once again do not hesitate to have a look at Herbert’s website.
I really believe that you do not necessarily need to look for a justification in this case. Too often we associate haze and volumetrics with pollution but we can also create them for pure dramatic purpose. No one gives a damn why there is some volumetric in Bruce Wayne’s kitchen. Really.
In dailies, the lighting artist would ask : “But where does this blue volumetric come from ?” Craig Welsh, the lighting supervisor would reply : it comes from me !We would burst out laughing after that.
No need to overthink it really. Volumetrics are such a great tool.
Rango and volumetrics
Pay attention to the warm impacts from the sun and cold volumetric rays in this saloon sequence from Rango (Director : Gore Verbinski, DP : Roger Deakins) . It gives so much depth to the frame. Awesome !
Lord of the rings and volumetrics
More examples can also be found in the Golden hall of Edoras sequence.
Blade Runner and volumetrics
Some fans have watched this movie flipped horizontally and desaturated for study purpose.
Indiana Jones and volumetrics
In this chase sequence from Indiana Jones and the Kingdom of the Crystal Skull (Director : Steven Spielberg, DP : Janusz Kaminski) you can see some warm impacts from the sun on the plants and some slightly cold volumetric rays.
More volumetrics example
There is a couple of examples of cold volumetrics that I wish to add like Predator (Director : John McTiernan, DP : Donald McAlpine), Tangled (Director : Byron Howard, DP : Mohit Kallianpur) and Donkey Skin (Director : Jacques Demy, DP : Ghislain Cloquet).
More examples of cold volumetrics can also be seen in Tangled, Predator and Donkey Skin.
You may have noticed that I am a huge fan of volumetric effects, especially cold ones. But this obsession should not convert into a tic ! Volumetrics must be designed and applied thoroughly.
The top light is a trick I have actually seen in many companies. In CG we often need to add a light coming from the top to reinforce the sky and give shape. Basically it is a Top Blue Light to reinforce the sky contribution.
Planet 51 and Top Light
On Planet 51 (Director : Jorge Blanco, Lighting Supervisor : Barbara Meyers), we used the top light on probably 80% of the shots (interior and exterior) of the movie. This is probably the movie I have used it the most.
Do you notice the blue specular on the nose and hair of these characters ? It does not come from the SkyLight. Theoretically it should. But very often in CG, Env Lights kinda flatten everything.
Great culture of character lighting at Ilion.
Top Light Recipe
The Top Light gives a nice shape to the characters’ faces and possibly adds some extra specular in a very nice way. To enhance the sky, you would :
- Create a light coming from the top with very soft shadows.
- On the X axis, value can be around 50 to 90 degrees rotation.
- Make the light blue (for example, R : 0.4 / G : 0.7 / B : 1).
- The light should not be too strong, like half a stop exposure.
Try to use matte painting in your Env light. In a perfect world, Digital Matte-Painting (DMP) would deliver a 360 skydome in .exr or .hdr format that we could use in our light rigs. So the reflections match the sky used in the final compositing. The Solid Angle documentation explains pretty well the different types of lights.This is very personal. I know some companies do not work like that at all.
I often wondered why this top direction ? Is this arbitrary or is there a reason to it ? My personal answer is that we are used to light coming from the top 99% of the time. The sky obviously, this giant top light over our heads but it is the same in a restaurant, the subway or the street at night.
This is why lighting from the bottom looks so unnatural, like the Monster lighting.
Warm edge is a way to enhance the halo (or turbidity) of the sun and to create rich and interesting shadows. I have used this trick on many different projects. It generally makes the border of the shadows more saturated and gives shape to your character when he is not in direct sunlight.
Pets and the Warm Edge
It has to be pretty subtle but still noticeable. You have to find the right balance on this. Warm edge is also called double sun or soft sun trick. Can you see the soft orange halo around the shadow border ? This is what the warm edge would do. Have a look at the example in Chapter 5.
Warm sun and cold shadows were the motto.
When you create a directional key light, it represents the sun. But what about its halo ? How can we enhance this ?
Warm Edge Recipe
- Duplicate your sun or key light.
- Make the color more saturated (for example, R : 1 / G : 0.4 / B : 0.1).
- Decrease approximately the intensity by 1 or 2 stops.
- Set the shadows way softer. The softer the shadows, the bigger the warm edge will be.
- It will create a warm edge effect on your shadow.
The whole debate about the warm edge is : should the Sun Soft have the same orientation than the Sun ? I have seen both setups. Theoretically they should have the same orientation. But sometimes it really helps to slightly rotate the Sun Soft to wrap better the character.
So I would say this : start with the same orientation for both lights and if needed rotate the Sun Soft slightly. There is a double advantage to this technique : we help the render engine with some direct lighting and it allows to shape things that would normally be in shadow.
During my holidays in Spain on Planet 51, I went to this beach where during sunset the sun would disappear behind a building. I noticed that after it disappeared, there was still some soft warm light coming from its halo. We used a lot this technique on Planet 51 and it made me feel more confident about it after experiencing it in real life.
Coming at Q4 2022. Thanks for your patience !
Coming at Q4 2022. Thanks for your patience !
Caustics are a very specific type of indirect lighting. They produce patterns of light when photons are reflected or refracted by a curved surface.They can be bounced off by a mirror or concentrated by a lens. Most famous caustics are the distortion of light at the bottom of a swimming-pool or the patterns refracted by a glass of water.
Have you heard this story about a jaguar which melted in London because of a building ?
Just an example to show you how strong caustics can be.
Caustics are a natural phenomenon and some render engines do support them. But I have never worked on a movie where we would output caustics directly in CG. Reason is pretty simple : caustics are expensive. Instead, we generally re-create them with a map. There are four good reasons for that :
- It is much cheaper this way.
- We also have more control on their look.
- We can place them wherever we want.
- It adds details to the shot.
Examples of Caustics
I personally see caustics as a tool. Sometimes when I feel my light rig is well-balanced but I want to highlight an element or guide a bit more the viewer’s eye, I know that caustics are a good way to do so.
You can use caustics to enhance your shots. They are very useful.
Caustics Light Recipe
- Duplicate your sun or key light.
- Decrease approximately the intensity by 1 or 2 stops.
- Keep the sharp shadows ! That’s very important.
- Use a map in your light. It could even be an animated sequence for a pool for example.
If you have a look at the sequence below, you’ll notice some animated caustics on the wall of the pool.For some unknown reason, caustics are not present in every shot. Looks like a mistake.
Caustics are a really nice effect to achieve in CG. Their textures add details, like on a wall for example, and make the render more interesting. Most of the time we avoid flat colors on big surfaces (like a wall) using extra impacts of lights or caustics. This is probably the biggest difference between 2D and 3D. We cannot afford the lack of details in 3D. Otherwise it will just look visually poor and cheap.
I thought for a very long time that caustics were only caused by refraction rays. But not at all ! In Arnold’s website, they mention the different types of caustics : reflective, refractive and glossy. Interesting !
How can we accentuate the light for windows or screens in a plausible and physical way ? One of the issue we face with interior lighting is to get some light inside the room. Walls tend to occlude a lot and block the light. Let’s see how we can get around this issue and force some light into interiors.
Directors in Feature Animation generally like to accentuate the lighting coming from screens. It is a behavior I have seen in many animated movies. Now the question is : how would you properly simulate the light screen ? Most of the time, the simplest solution is the best.
I picked this example from Ninjago. It’s all about enhancing and flexibility.
I am skipping here over the technical details of texture mapping the light. If the screen has an animated sequence in its shader, well it is compulsory to use the same sequence for your lights. Otherwise you may have some big discrepancies.
How can we increase the influence of the sky inside the room below ? Of course we would have to take in account render times, sampling, flexibility and control.
Sorry about the sRGB image. Not a great choice !
There are several ways to do a proper setup for this and it depends on which look you want to achieve. In CG, quite often, it is difficult to bring light inside of a room. Most of the time there is too much occlusion and we have to force light in. I would say the two biggest challenges for this setup are the noise and the fact that directors in Feature Animation generally do not like when the sky is overexposed. They like the sky to be blue. Here is the solution I have come up with.
This is pretty much the setup we used on the white apartment sequence from Pets.
I know for a fact that on Moana and the Lion King they had split skies : one for lighting and one for matte-painting. I am not against it : it is flexible and is a totally valid workflow.This is pretty much a personal choice.
One interesting option supported by several render engines is the ability to convert an area light into a portal to guide skylight sampling. Once again check the Guerilla Render website about portals.
I had once a debate about : is there such a thing as Cartoon Lighting ? Or does the cartoon look come from mainly grading and saturation ? After having given it many thoughts and thanks to the mental image we have seen in the previous paragraph, I will stick to my answer : yes, there is such a thing as cartoon lighting. Mental image is cartoon lighting.
Does it mean that I light differently based on the nature of the project ? In a way, I want to say “yes“. Each movie with its own unique art direction should require a different lighting style. But in reality, since Hollywood does not like taking risks, most PBR movies end up with the same lighting design. Which makes me light the same way on different movies.
IES stands for ‘Illuminating Engineering Society‘. It is a society that brings together lighting experts (e.g. lighting designers, consultants, engineers, sales professionals, architects, researchers, lighting equipment manufacturers, etc.) to better design lighting conditions in the real world.
IES files contain measurement data about the light distribution of a lighting fixture in an ASCII format. These files are typically used by lighting fixture manufacturers to electronically transfer photometric data over the web. Basically an IES file is the digital profile of a real world light.
Playmobil and IES lights
We used IES lights for our vehicles in Playmobil (Director : Lino Di Salvo, Art Director : Rémi Salmon).
I did most of the vehicles lighting setups on this movie.
Headlights setups are actually pretty difficult to do. You generally need several lights with light-linking to get the proper effect. There is this huge debate in studios about lights in assets. I am a big fan of what we did on Lego Batman. It makes so much sense to have the lights built in the asset rather than passing a rig of light from shot to shot.
Here is a first pass on lighting techniques. There are much more techniques I would like to describe for the end of the year 2022 with proper examples and renders.
If you are interested by this topic, you can have a look at these links :
- Volumetrics definition
- Sight Sound Motion Applied Aesthetics
- Lenses description
- Wikipedia Camera Lens and Camera Lenses
- Circle of Confusion
- Renderout is a Spanish online magazine about CG.
- Surfacing theory – Pathtracing, a french video by Jean-Michel Bihorel at Creative Seeds.
- Danielle Feinberg TED Talk
- Code & Visuals Blog
- Disney Technology Publications
- Disney Research Studios
- Unity Publications
- Blue Sky Tech Blog
- Real Time Rendering Siggraph 2019
- Self Shadow Siggraph
- The PBR book
- Psychopath Renderer’s blog
- Advanced Rendering Toolkit
- Fresnel reflection by Sane Simmart
- Thomas Müller Research
- Monte Carlo Method for Volumetric
- Tamy Boubekeur’s Denoise
- Johannes Hanika’s website
- The Rendering Equation
- 3Blue1Brown, by Grant Sanderson, is some combination of math and entertainment.