- The CG issue
- What is cheating?
- Cheating examples
- Good and bad tricks
- Cheating attitude
- Light decay
- Top Light
- Warm edge
- Luminous surfaces
- IES Lights
The CG issue
We are going to review some techniques related to our medium. 3d softwares give us a tremendous amount of freedom but bring as well some constraints. How should we deal with those in an efficient way? Should we put some limits? It is not because you can create as many lights as you want that you should. With great power comes great responsibility.
What is cheating?
It is actually a pretty hard question to answer to. Because cheating does not have the same definition at Animal Logic, Weta or Illumination. It really depends on the studio and your supervisor: it is cultural.
If I look into a dictionary, cheating means acting dishonestly to gain an advantage.
But what does cheating mean in CG? Does it mean I acted dishonestly? In relation to what? Reality? Is using a blocker a cheat? Is light-linking a cheat? Or should we give artists the most adapted tools to their medium?
I do not like this word because it makes me feel guilty for doing my job. No I did not cheat, I did what I had to do to make it look good.
“Hitchcock […] couldn’t get clean focus on the gun and on Ingrid Bergman in the same frame. So he used a giant gun and a huge fake hand to get the effect that he wanted.”
Lord of the rings
Peter Jackson used “forced perspective” a lot on Lord of the rings (DP: Andrew Lesnie). In the pass of Caradhras, he used a ring of 18-inch circumference when Frodo has fallen and the ring is lying in the snow.
Copyright © New Line Cinema 2001
After seeing these examples, one would think that directors of photography cheat all the time on set. Really? Did they act dishonestly to gain an advantage? I don’t think so. I would rather use the word: enhancing. DPs manipulate and control their environment to enhance the reality.
It is NOT a cheat, it is a trick or a gag. On set DPs would generally use those words a lot. I know it may sound like a stupid detail but vocabulary in lighting is essential. And similar tricks are also used in lighting! Even for a “basic” daylight exterior, DPs will use reflectors, black flags and many other tools on set. We should be able to control every element. It’s what DPs do.
Issue in CG is that many supervisors do not want to give control to their team. They have seen so many horrors and crazy setups that they want to lock everything. I don’t blame them. But sups have to find the right balance between control and safety. A good way to empower a team rather than controlling it is training.A team should not feel controlled by the CG police every time they add a light.
A famous Disney movie
One of the best stories I have ever heard. Get ready.
On a famous Disney movie, Rob Legato did a “trick” with a time lapse of a night sky. They went and shot a night sky out here in the Mojave desert. Rob also did a shoot with a light box and some paper with holes in it. Brought it into nuke and enhanced it, to something that was a bit more pleasing to the eye. Things like readability of the stars against a post dusk sky. He put them both in front of Jon Favreau in our review room and asked him to pick the one he liked best.
Here comes the best part!
Without telling him what he had done and what was different. And Jon picked the one he shot with a light box and his explanation was: the stars feel real to me. It’s how I remember seeing stars coming out after dusk and getting brighter as the night goes on. It reminds me of my camping trips as a kid. Rob then tells him it’s fake and everyone bursts out laughing, even Jon! It was pretty amazing to watch and Rob didn’t say much. He didn’t have to. But capturing how people remember things and what they hold on to is an art.
And this brilliant conclusion.
Our job is about perception more than reality. Reality can be boring. But people perceive things differently and that’s what DPS compare it to when they gauge how “real “something is.Brilliant Ted talk from Rob Legato about this topic if you want to know more.
Spider-man: Into the Spider-Verse
Loved the movie like everyone else. There is probably a project of this quality and originality every ten years or so. I just wanted to share a couple of screenshots from the making-of.
Copyright © Sony Pictures 2018
Andre Turpin, DP of Playmobil: “We use tricks as much as we can. We put lights on the walls, the ceiling and even the floor sometimes.”
Aymeric Montouchet, DP of Detroit: become human: “The only thing we should not cheat in CG is the quadratic decay of light.”
Good and bad tricks
Now we have seen that “cheating” is a terrible word, we have to wonder about the main differences between CG and live action. I totally agree that we should work like Digital Directors of Photography. But let’s not forget that each medium has its own properties, rules and limitations. I personally see two BIG differences between live-action and CG:
- Render engines’ limitations aka we are not doing spectral rendering.
- Sampling and render times aka this is not real time.
This is why I distinguish two types of tricks: good and bad. Render engines give us so many possibilities that it is very easy to do crazy stuff. We have to find a middle ground between the freedom we will give to the artists and the control we want to have on their methods.
Examples of good tricks:
- Add lights so the shot relies less on indirect lighting? YES.
- Use a blocker to have some vignetting in a shot? YES.
- Use light-linking to make your character pop? YES.
Examples of bad tricks. This is very personal by the way. You may disagree and I am happy to know what you think.
- Use light-linking to compensate for poor surfacing? NO.
- Remove the walls from shadow casters to get more light into the room? NO.
- Start a Master Lighting with different lights for characters and sets? NO.
- Change the decay of a light. NO. It has to be quadratic. Except Sun and Sky obviously.
- Use some ambient lighting. NO. This is just wrong.
- Have non-physical albedo values. NO. Luminance has to be between 0.03<x<0.9.
Ambient lighting has now (almost) disappeared thanks to PBR. The easiest way to describe it would be an EnvLight with no occlusion. I guess it is only used in video games.
In the introduction, I was explaining that this book is more focused on PBR productions. So if you base things off a “ground truth” or reality, yes, we can use the word “cheat”. The software is essentially building a model in which to emulate light in the real world with things like decay, bounce, global illumination…
But this word is so negative. We should never make an artist feel guilty for having created a couple of extra lights. And if you want your artists to work in a certain way, because of art direction or render times, it is totally understandable. But train them so they know.
My lead at Animal used to tell me: “Whatever works”. Is it that simple? If you only care about your shot, yeah that would do the trick. But lighting for me is about finding viable solutions that you can easily share with the team. Lighting is about building physical environments that stand correctly. If you start to remove walls, objects, link them together, exclude some others, what are you trying to achieve? You are just destroying the plausibility of the CG world.
We have seen in a previous chapter that the Sun could either be a Spot or a Distant light. What visual difference does it make? Actually a big one, because of the decay. The Lighting design for stylized animation paper explains pretty well light falloff.
From Dave Walvoord: The natural falloff from light sources is another way to add depth to shots. The brightness of light diminishes in an inverse squared manner […]. Therefore, if you want a lot of falloff then a light should be placed close to the subject. This has the effect of increasing depth between objects. A light placed farther away has less falloff and will compress the space.I personally prefer the term of decay. But it is very personal. Once again vocabulary is a very important part of our job.
© 2019 DreamWorks Animation All Rights Reserved.
The characters and camera are in the exact same place, and only the light is moved [in the renders above]. In the first image, the light is far away from the characters and the depth between them is compressed. In the second image the light is brought closer and there is a large amount of falloff from Hiccup in the front to Stormfly in the back. This produces the effect that the characters are farther away from each other. This technique can be used to great effect to change the depth of the shot.
A supervisor once told me not to put any lights close to the character. I guess he is right if your goal is to tie the characters together. But it actually depends on what you want to achieve. Do you want to tie your characters together or do you want to separate them? Both options are possible.
Light-linking is when a light illuminates only certain objects. It is NOT the same as shadow linking, when you remove some shadow casters from a light. A proper light-linking will keep all the shadow casters as they are.
I have tried to find an example where light-linking (in this case it is lighting in layering but that is essentially the same thing) is a bit extreme. The characters seem to be in a different world than the set.
Linking on The Star
Copyright © Sony Pictures Entertainment 2017
I have thought a lot about light-linking and right or wrong. In my opinion it does not make sense to say: light-linking is crap or I hate light-linking. Every studio has its culture and its way of doing things.The real question being is it used properly?
Linking on Lego Batman
For example, I did plenty of light-linking on Lego Batman because the art direction was very graphic. I did not use any on The secret life of pets because of its natural look. The only conclusion I could come with is: Light-linking is based on art direction. Honestly, it would have been impossible to do Lego Batman without it.
I generally try to avoid light-linking. I only use it if I do not have ANY other options. But it does not mean that it is not a good solution. Depending on the art direction, you may have to rim your characters like crazy. In this case, YES, light-linking is the only solution.
But I would not start a Master Lighting using it by default. When I begin a sequence, I use lights that affect everything. It is only after dailies and based on the Supervisor’s comments that I will eventually use light-linking.
Extreme light-linking on Planet 51
And if you ask me, below is the most extreme light-linking I have ever done in my life. I don’t think I have ever told anyone before. Chuck the astronaut has one sun for his body and one for his face.
I was not able to get the proper look without doing it this way. But I guess it’s okay because I don’t think you can spot it. Visually it looks like one sun.
One thing to keep in mind is that the rims cannot be stronger than the source. Because of quadratic decay, the light fades with distance.
The only advice I could give you about light-linking is from my own experience. It is very personal so take it with a grain of salt. It actually took me a while to realize that. I only do additive light-linking.
I generally start with the lights affecting everything. And then I would eventually add a couple of lights on certain elements. I try to have the highest common denominator for my light rigs. It is very rare for me to remove an object from a light influence.
Volumetric rendering refers to a technique for generating a visual representation of data that is contained in a three dimensional space (volume). Examples of this are atmospheric effects such as smoke and fog […]. In marching cubes, data is represented as voxels, but in the sense that we store some floating point value in a structured grid in three dimensions. Volume rendering is contrasted with Surface rendering because the data that is being rendered is coming from a three dimensional data set rather than a two dimensional data set (a surface).
Next example comes from Lego Batman (Director: Chris McKay, PD: Grant Freckelton). I just love this shot. Pay attention to the use of volumetric.
This is one of my favorite tricks: see how this fake blue volumetric helps to read the microwave? There is no doubt about what batman is doing. Otherwise it would just be a dark microwave over a dark wall and it would be hard to read the action. It is the most perfect cheated volumetric just to make the microwave pop. Awesome!
In dailies, the lighting artist would ask: “Where does this volumetric come from?” Craig, the lighting supervisor would reply: It comes from me!We would burst out laughing after that.
Separation by color. See how the microwave area is warm and the wall area is cold? It helps splitting the character from the background and gives more depth to the image. Once again do not hesitate to have a look at Herbert’s website.
The LEGO Batman Movie: Warner Bros. Pictures and Warner Animation Group, in association with LEGO System A/S, a Lin Pictures / Lord Miller / Vertigo Entertainment production / © 20th Century Fox 2017 / © Sony Pictures 2005
It is funny how I have changed my mind about this topic. After Pets, I would probably have said stuff like: Do not create any special lights for this pass. It is really important that that volumetric and beauty have a visual connection.
Grant and Craig really helped me to grow as an artist. Volumetrics (dust, fog and haze) are a tool for the cinematographer. It will help you to direct the viewer’s eye, enhance drama, create depth and complement the composition.
Volumetric rendering is a bit technical and can possibly give headaches, even to senior artists. One of the most important parameter is the density of your volume. Because it will change greatly its look and its alpha. The Arnold website about volumetrics is very well written and easy to understand if you want to read more on the subject.
My personal preference to work on my volumetric is to render it interactively in my master layer. It allows me to check easily its density and color. If you tweak your volumetric shader with all the geometry in matte, you may get some surprises later in comp.
- I work interactively with everything in the scene.
- But I render in separated passes for beauty and volumetric.
- I generally start with my beauty lights affecting the volumetric. It is easier to set the density this way.
- Once I am happy with the density, I’ll see if I need to create special volumetric lights or not.
My lead used to tell me: volumetrics pass always look nice (because of the black background). The real challenge is when you merge them with the beauty.
Should we merge in ‘over‘ or ‘plus‘ the volumetric pass? I know that ‘over‘ is the correct way and we should definitely try to respect this. BUT production realities sometimes make it impossible. If I am in rush and my volumetric pass is too dense, I’ll do what I have to do.
If you feel like your volumetric is darkening too much your beauty, you have several options:
- Decrease the volumetric density (which will have an effect on its alpha).
- Increase the volumetric shader color.
- Increase the light exposure.
- If you cannot re-render, you may grade the volumetric pass. Rule of thumb would be to grade color (RGB) and alpha (A) at the same time to keep it consistent.
On Lego Batman, we used the depth pass. A lot. I am pretty sure half the shots have border since the depth pass was not defocused nor anti-aliased. It is better to do the fog effect in CG if possible.
Copyright © Warner Bros. Pictures 1982
The top light a trick I have actually seen in many companies. In CG we often need to add a light coming from the top to reinforce the sky and give shape. Basically it is a Top Blue Light to reinforce the sky contribution.
Planet 51 and Top Light
On Planet 51 (Director: Jorge Blanco, Lighting Supervisor: Barbara Meyers), we used the top light on probably 80% of the shots (interior and exterior) of the movie. This is probably the movie I have used it the most.
Do you notice the blue specular on the nose and hair of these characters? It does not come from the SkyLight. Theoretically it should. But very often in CG, Env Lights kinda flatten everything.
Copyright © Sony Pictures/TriStar Pictures 2009
Top Light Recipe
The Top Light gives a nice shape to the characters’ faces and possibly adds some extra specular in a very nice way. To enhance the sky, you would:
- Create a light coming from the top with very soft shadows.
- On the X axis, value can be around 50 to 90 degrees rotation.
- Make the light blue (for example, R: 0.4 / G: 0.7 / B: 1).
- The light should not be too strong, like half a stop exposure.
Try to use matte painting in your Env light. In a perfect world, Digital Matte-Painting (DMP) would deliver a 360 skydome in .exr or .hdr format that we could use in our light rigs. So the reflections match the sky used in the final compositing. The Solid Angle documentation explains pretty well the different types of lights.This is very personal. I know some companies do not work like that at all.
I often wondered why this top direction? Is this arbitrary or is there a reason to it? My personal answer is that we are used to light coming from the top 99% of the time. The sky obviously, this giant top light over our heads but it is the same in a restaurant, the subway or the street at night.
Warm edge is a way to enhance the halo (or turbidity) of the sun and to create rich and interesting shadows. I have used this trick on many different projects. It generally makes the border of the shadows more saturated and gives shape to your character when he is not in direct sunlight.
Pets and the Warm Edge
It has to be pretty subtle but still noticeable. You have to find the right balance on this. Warm edge is also called double sun or soft sun trick. Can you see the soft orange halo around the shadow border? This is what the warm edge would do. Have a look at the example in Chapter 5.
Copyright © Illumination 2016
When you create a directional key light, it represents the sun. But what about its halo? How can we enhance this?
Warm Edge Recipe
- Duplicate your sun or key light.
- Make the color more saturated (for example, R: 1 / G: 0.4 / B: 0.1).
- Decrease approximately the intensity by 1 or 2 stops.
- Set the shadows way softer. The softer the shadows, the bigger the warm edge will be.
- It will create a warm edge effect on your shadow.
The whole debate about the warm edge is: should the Sun Soft have the same orientation than the Sun? I have seen both setups. Theoretically they should have the same orientation. But sometimes it really helps to slightly rotate the Sun Soft to wrap better the character.
So I would say this: start with the same orientation for both lights and if needed rotate the Sun Soft slightly. There is a double advantage to this technique: we help the render engine with some direct lighting and it allows to shape things that would normally be in shadow.
During my holidays in Spain on Planet 51, I went to this beach where during sunset the sun would disappear behind a building. I noticed that after it disappeared, there was still some soft warm light coming from its halo. We used a lot this technique on Planet 51 and it made me feel more confident about it after experiencing it in real life.
Caustics are a very specific type of indirect lighting. They produce patterns of light when photons are reflected or refracted by a curved surface.They can be bounced off by a mirror or concentrated by a lens. Most famous caustics are the distortion of light at the bottom of a swimming-pool or the patterns refracted by a glass of water.
Have you heard this story about a jaguar which melted in London because of a building?
Caustics are a natural phenomenon and some render engines do support them. But I have never worked on a movie where we would output caustics directly in CG. Reason is pretty simple: caustics are expensive. Instead, we generally re-create them with a map. There are four good reasons for that:
- It is much cheaper this way.
- We also have more control on their look.
- We can place them wherever we want.
- It adds details to the shot.
Examples of Caustics
I personally see caustics as a tool. Sometimes when I feel my light rig is well-balanced but I want to highlight an element or guide a bit more the viewer’s eye, I know that caustics are a good way to do so.
Copyright © Illumination 2018
Caustics Light Recipe
- Duplicate your sun or key light.
- Decrease approximately the intensity by 1 or 2 stops.
- Keep the sharp shadows! That’s very important.
- Use a map in your light. It could even be an animated sequence for a pool for example.
If you have a look at the sequence below, you’ll notice some animated caustics on the wall of the pool.For some unknown reason, caustics are not present in every shot. Looks like a mistake.
Caustics are a really nice effect to achieve in CG. Their textures add details, like on a wall for example, and make the render more interesting. Most of the time we avoid flat colors on big surfaces (like a wall) using extra impacts of lights or caustics. This is probably the biggest difference between 2D and 3D. We cannot afford the lack of details in 3D. Otherwise it will just look visually poor and cheap.
I thought for a very long time that caustics were only caused by refraction rays. But not at all! In Arnold’s website, they mention the different types of caustics: reflective, refractive and glossy. Interesting!
How can we accentuate the light for windows or screens in a plausible and physical way? One of the issue we face with interior lighting is to get some light inside the room. Walls tend to occlude a lot and block the light. Let’s see how we can get around this issue and force some light into interiors.
Directors in Feature Animation generally like to accentuate the lighting coming from screens. It is a behavior I have seen in many animated movies. Now the question is: how would you properly simulate the light screen? Most of the time, the simplest solution is the best.
I am skipping here over the technical details of texture mapping the light. If the screen has an animated sequence in its shader, well it is compulsory to use the same sequence for your lights. Otherwise you may have some big discrepancies.
How can we increase the influence of the sky inside the room below? Of course we would have to take in account render times, sampling, flexibility and control.
There are several ways to do a proper setup for this and it depends on which look you want to achieve. In CG, quite often, it is difficult to bring light inside of a room. Most of the time there is too much occlusion and we have to force light in. I would say the two biggest challenges for this setup are the noise and the fact that directors in Feature Animation generally do not like when the sky is overexposed. They like the sky to be blue. Here is the solution I have come up with.
I know for a fact that on Moana and the Lion King they had split skies: one for lighting and one for matte-painting. I am not against it: it is flexible and is a totally valid workflow. That is a personal choice.
One interesting option supported by several render engines is the ability to convert this area light into a portal to guide skylight sampling. Once again check the Guerilla Render website about portals.
I had once a debate about: is there such a thing as Cartoon Lighting? Or does the cartoon look come from mainly grading and saturation? After having given it many thoughts and thanks to the mental image we have seen in the previous paragraph, I will stick to my answer: yes, there is such a thing as cartoon lighting. Mental image is cartoon lighting.
Does it mean that I light differently based on the nature of the project? In a way, I want to say “yes“. Each movie with its own unique art direction should require a different lighting style. But in reality, since Hollywood does not like taking risks, most PBR movies end up with the same lighting design. Which makes me light the same way on different movies.
IES stands for ‘Illuminating Engineering Society‘. It is a society that brings together lighting experts (e.g. lighting designers, consultants, engineers, sales professionals, architects, researchers, lighting equipment manufacturers, etc.) to better design lighting conditions in the real world.
IES files contain measurement data about the light distribution of a lighting fixture in an ASCII format. These files are typically used by lighting fixture manufacturers to electronically transfer photometric data over the web. Basically an IES file is the digital profile of a real world light.
Playmobil and IES lights
We used IES lights for our vehicles in Playmobil (Director: Lino Di Salvo, Art Director: Rémi Salmon).
© 2019 – 2.9 Film Holding – Morgen Production | © PLAYMOBIL
Headlights setups are actually pretty difficult to do. You generically need several lights with light-linking to get the proper effect. There is this huge debate in studios about lights in assets. I am a big fan of what we did on Lego Batman. It makes so much sense to have the lights built in the asset rather than passing a rig of light from shot to shot.
Here is a first pass on lighting techniques. There are much more I would like to describe in the coming months. Meanwhile if you are interested by this topic, you can have a look at these links:
- Sight Sound Motion Applied Aesthetics
- Eisenstein et la mise en scene (This book is only available in French I think)
- Megascan library
- Lenses description
- Wikipedia Camera Lens and Camera Lenses
- Circle of Confusion
- Disney Technology Publications
- Disney Research Studios
- Blue Sky Tech Blog
- Real Time Rendering Siggraph 2019
- Self Shadow Siggraph
- The PBR book
- Advanced Rendering Toolkit
- Fresnel reflection
- Thomas Müller Research
- Monte Carlo Method for Volumetric
- Tamy Boubekeur’s Denoise