14 replies to this topic

[starstutter]

Unfortunately, there seems to be no book called "Spherical harmonic lighting for the mathematically challenged". And also, everything I've seen which attemps to explain SH just throws enourmous amounts of numbers, equasions, and pictures of blobby circles at me. Excuse me for the rudeness, it just gets a bit tiring seeing answers that take longer to decode than the problems themselves.

So anyway, here's just a few questions that may help me start on my way to understanding SH. Please try to use something other than mathematical terms... I'm a great graphics programmer, but I'm really a lost cause when it comes to math:

1. How does this process differ from something like irradience volumes?

2. Does this method take most of it's resources from realtime, or precomputed and stored in memory?

3. How exactly does something pre-computed cast dynamic shadows? And are these shadows generated per-pixel, or per-vertex?

4. I seem to get mixed answers on this one. Does this method only accuratley work for indirect lighting, or can it handle direct lights as well?

I realize this is somewhat of a large question, and to whoever answers, thank you very much in advance.

[kenpex]

looool: "I'm a great graphics programmer, but I'm really a lost cause when it comes to math"

c'mon you're joking... but anyways, let me reply some of your questions:

1) they're completelly unrelated. basically, SH are a way to express functions on a sphere, in a compact way. You can use them to encode a spherical map. And you can use them to encode the occlusion that a vextex sees around it. If you multiply togheter those two encoded SH, you get a nice shading. As you're a great graphics programmer, you can surely figure out why

2) precomputed

3) you get dynamic shadows from static geometry and dinamic lights, that's because you encode in SH the occlusion every vertex sees, so it's like you've encoded all possible shadow casters in every vertex. This works only for low frequency shadows (no hard ones). Spherical wavelets are better if you need higher frequencies. If the geometry moves, then SH shadows won't work. You can use them by only storing occlusion in the same object, so what you get is that object ambient occlusion and self-shadowing, and use another method to get the shadows cast by other objects in the scene. They are per vertex BTW

4) could handle direct, but as only low frequency shadows can be cast, it's better to use it for indirect only and use another method for direct.

http://c0de517e.blogspot.com

[Shirakana]

You're right, you can't go through SH understanding without some advanced math background. As I'm currently working on GI topics, I had to study SH in details. I'll try to make it clear for you.

Actually SH is just a basis of "sphere space". As a graphic programmer, you know that any point (or data) in 3D space has 3 coordinates, and these coordinates are just coefficients expressed in function of 3 basis vectors that define your space, e.g (1,0,0) (0,1,0) (0,0,1). With these 3 basis vectors, you can express any points in space with a linear transformation of its coefficients (coordinates). For example Point P(x,y,z) = x * (1,0,0) + y * (0,1,0) + z * (0,0,1) There you can see that x,y,z are called coefficients and (1,0,0)..(0,0,1) are called basis vectors.

You can just take the same idea with SH by saying that SH are basis vectors for sphere space, i.e you can express any data on a sphere by a linear transformation of coefficients (called SH coefficients) and SH basis vectors. SH have many advantages (due to mathematical property) such as compactness and efficient computation at runtime.

1. SH is just a mathematical tool and as kenpex said, it is unrelated with the way you can use it (I think at first it was used for quantic physic calculation). I mean that you can implement irradiance volume with SH, but also with other tools for encoding your "lighting values".

2. SH are used with precomputed technics. Basically you just precompute lighting values for your scene, express it in SH coef and store them. You can then re-use efficiently this coef at runtime to get your final color value.

3 & 4. As it is used with precomputed technics, you can't get shadows with dynamic objects (but some recent works try to extend this limitation) but you can combine it with others that enables dynamic shadows. SH is a good way to encode efficiently indirect lighting (self-shadowing, inter-reflections, glossy material...) but can handle direct lighting too, as it is just a tool you can use it as you wish. But as kenpex said, I think it is better to use it for indirect illumination only

I recommend you to read "Spherical harmonics the Gritty details" from Green, which explains quite well mathematical theory lying under SH.

[Shirakana]

Can someone can confirm what I said, just to ensure that I properly understand what I've read ?

[nomad421]

Shirakana,

I believe that what you said is mostly correct. However, here is my understanding of the situation:

Quote

Actually SH is just a basis of "sphere space". As a graphic programmer, you know that any point (or data) in 3D space has 3 coordinates, and these coordinates are just coefficients expressed in function of 3 basis vectors that define your space, e.g (1,0,0) (0,1,0) (0,0,1). With these 3 basis vectors, you can express any points in space with a linear transformation of its coefficients (coordinates). For example Point P(x,y,z) = x * (1,0,0) + y * (0,1,0) + z * (0,0,1) There you can see that x,y,z are called coefficients and (1,0,0)..(0,0,1) are called basis vectors.

The SH bases are just orthogonal functions on the surface of the sphere. This is similar to the canonical bases of R^3 (i.e. (1,0,0), (0,1,0), and (0,0,1) ) but still somewhat different. Each spherical harmonic coefficient corresponds not to a single direction, but to the values of an entire function over the whole sphere. The property that allows you to represent incident irradience with SH functions is that they are orthogonal , and they are increasing in spatial frequency (they start out varying slowly and smoothly over the sphere's surface, but higher order SH functions capture higher and higher frequency lighting effects).

Quote

You can just take the same idea with SH by saying that SH are basis vectors for sphere space, i.e you can express any data on a sphere by a linear transformation of coefficients (called SH coefficients) and SH basis vectors. SH have many advantages (due to mathematical property) such as compactness and efficient computation at runtime.

Yes, the desirable properties in particular are:

• orthogonal. This allows you to reconstruct the sampling of the sphere
  you created when doing all your monte-carlo sampling stuff. Basically, when you do that, you're evaluating (numerically) an incident radiance field over the sphere and projecting this field onto the (orthogonal) SH
  functions. This orthogonality property allows you to reconstruct the
  function using a simple dot product, because each of your SH vectors is
  orthogonal (and if properly scaled, orthonormal).
  
  
• rotationally invariant. Though the matrices are not trivial, there is an analytic formula for producing a matrix that allows you to rotate the SH coefficients. This means that once you capture a particular lighting environment in terms of SH coefficients, you can rotate the environment (and equivalently the geometry inside of it), simply by multiplying the coefficients by a special matrix. This is different from other basis functions (such as wavelets (though not spherical wavelets)) which require you to re-sample the lighting environment if your model or environment are rotated.

Quote

1. SH is just a mathematical tool and as kenpex said, it is unrelated with the way you can use it (I think at first it was used for quantic physic calculation). I mean that you can implement irradiance volume with SH, but also with other tools for encoding your "lighting values".

Exactly.

Quote

2. SH are used with precomputed technics. Basically you just precompute lighting values for your scene, express it in SH coef and store them. You can then re-use efficiently this coef at runtime to get your final color value.

Exactly.

Quote

3 & 4. As it is used with precomputed technics, you can't get shadows with dynamic objects (but some recent works try to extend this limitation) but you can combine it with others that enables dynamic shadows. SH is a good way to encode efficiently indirect lighting (self-shadowing, inter-reflections, glossy material...) but can handle direct lighting too, as it is just a tool you can use it as you wish. But as kenpex said, I think it is better to use it for indirect illumination only

This is an interesting topic, and an area of active graphics research. SH, in it's most basic form, is excellent for low frequency lighting effects (assuming infinitely distant light sources). It can handle soft shadows and diffuse interreflections very well. There has been work in re-tooling spherical harmonics to do mid-range illumination, this basically works by looking at the gradient across SH vectors and doing a 1st order Taylor approximation to the incoming irradience. The key when using spherical harmonics with other lighting effects is that lighting is additive, so that whatever you compute using SH, you can simply add the effects of direct lights onto the resultant colors.

Quote

I recommend you to read "Spherical harmonics the Gritty details" from Green, which explains quite well mathematical theory lying under SH.

This is a great first resource, and where I originally learned about SH.

[Shirakana]

Thanks nomad421, actually I didn't go too deep into math details (orthogonal properties and more) as starstutters asks but reading your post confirm that I understand well SH.

[starstutter]

*To everyone, thanks very much for the replys.*

kenpex said:

c'mon you're joking...

No, seriously. About the highest level of math I can fluently understand is algebra 2. Been struggling horribly all my life with math (it actually seems to run in my family). However, I seem to be exceptionally good with logic and complex cause/effect systems.

Really though, I've had countless people tell me that I should just forget game programming because I needed "godly math skills" to pull off even the simplest thing. People were telling me this back when I was making pong and pac-man (8th grade). Now I have a complex 3D engine which I am modelling after Unreal 3 and some parts of the CryEngine (shadows mostly). About 1/2 done with it and it's turing out fantastic. Most everything was done from scratch except for a few complex math equasions, and that was a somewhat small part.

As just an example of how my mind works with these things, this is really no different than when I was trying to do water reflections. I was going all over the net seeing enourmous equasions, complex theroms, and endless matricies thrown in my face.... seriously some were like 8 pages. Then I found out that all anyone had to say was that I needed to flip the scene upside down... why someone couldn't say that? Beats me.

But I'm getting off point. So to try and sum up a better understanding in my own simple minded words:

1. SH is nothing more than a way to store information that can be easily and efficiently accessed. The way I an relating this, is that when someone said "sphere space", I started thinking about model space, world space, blah blah blah. Not implying they are related, but can they be trandformed into "sphere space" using a typical means similar to camera transformations.

2. I see, this does make sense. Maybe this is wrong, but how are these stored? Is the information kept in each vertex as part of the mesh? Or in something like a sphere map.

3. I am relating this to ambient occlusion, something that I understand very well. It is obvious how soft shadows would come out of this, and likewise with static SH, however, I still am not grasping how this would change during runtime. The only explaination I am coming up with is a multitude of occlusion views that a single vertex has, and it can select which lighting view it needs during runtime.

4. This is all ok for me. I have plenty of other shadow casting methods, including one I am kind of pioneering that involves a combination of color channels to produce super-soft shadows without any filtering in the pixel shader. They work similar to penumbra maps exept they do not need geometry.

Maybe some things above are completley wrong, but I suppose that's my question.

As for the "Gritty Details" paper. I agree it is very descriptive, but as with everything else, I just am having a horrendously hard time comprehending all the math it expects me to pick up and run with. However, after this I will try and read it again to get a better understanding. This has already been very helpful. Thanks much.

kenpex said:

Thanks nomad421, actually I didn't go too deep into math details (orthogonal properties and more) as starstutters asks but reading your post confirm that I understand well SH.

Yes, thank you. It seems like a strange thing for the more math oriented, but going into mathmatical detail about these topics just isn't helpful at all to me. It actually makes it much worse at times.

[CheshireCat]

Don't confure SH and PRT though. SH based lighting doesn't have to be precomputed, while PRT is precomputed SH technique. SH isn't simply a way to store spherical function either but it has some nice properties. You can efficiently compute convolution of two spherical functions (common operation in lighting equations), it's smooth function (not only C0 continuity), and it's rotationally invariant (if you rotate light around object the light moves smoothly).

[nomad421]

CheshireCat,

This is a good point. The GPUGems2 book has an interesting article entitled "Real-Time Computation of Dynamic Irradiance Environment Maps" on using the GPU to compute (very low order SH approximations) of environment lighting in real time. This technique is very cool because it allows you to have a dynamic environmental light source. However, there is no PRT involved here, and thus you loose some cool effects such as shadows and diffuse interreflections. People often confuse SH and PRT, but SH are just a set of functions, and they were used in Chemistry to compute properties of electron orbitals long before they were used in graphics.

[kenpex]

@Shirakana: What U said is mostly right. The analogy with the canonical base of the 3d space is useful but you have to understand that SH are not bases of a vector space but of a function space, so you want to encode spherical functions, not points on a sphere.

@starstutter: You can surely craft a fine 3d engine without having to know anything about maths. You can surely keep asking questions, browsing sites, reading stuff and code snippets and then make your engine on those. What you cannot do is to make anything NEW if you don't understand the maths, because expecially in those days, any non-trivial shading trick is heavily depentent on maths. For example, see the fog shader of crysis. Surely you can code it without knowing maths after U see the siggraph presentation by crytek guys. But they started with a famous equation for athmosperic scattering, found it too heavy, then resorted to use a reasonable approximation by computing, analitically, a line integral.

Regarding what U say on SH, I'll try to transliterate maths in something that is not maths. See the magic:

1) no as I said, SH are a basis for a space, but that space is not one with numbers and points, like (1,0,0.5). It's a space of functions. Mhm think of a texture. It can be seen as a discrete approximation of a function f(x,y)=texture_value_at_x_y. Now wrap that texture on a sphere and you have a discrete approximation of a spherical function. An expensive one. SH are a way of encoding the same function with a few numbers. And performing operations on it. It's like Jpeg for spherical textures. But as you've using only a few coefficients (7-8 numbers in practice) you can't encode too much. In fact, you can only encode very blurred images...

2) The SH coefficients are stored with each vertex. And usually, in SH based PRT, they encode the occlusion of that vertex as seen by that point, so the spherical function you're trying to encode is a sort of spherical z-buffer around that point. It's expensive to compute, it's done offline by raytracing.

3) I think you understood right. Lights can change as you encode them as SH (you start with a spherical environment map and tranform it in SH coeffs). Then for each vertex, you multiply the light SH with the vertex occlusion SH, so you get more light for the unoccluded directions and less for the occluded ones. In this scheme, the SH light map can easily be changed in runtime.

[starstutter]

kenpex said:

What you cannot do is to make anything NEW if you don't understand the maths, because expecially in those days, any non-trivial shading trick is heavily depentent on maths.

Totally agree with the second part, but with the first part, experience has taught me otherwise. I won't go into a big rant here (and I don't blame you if you don't belive me), but somewhere around %70 of the graphics technologies that I implement I kind of whip up myself, and then find out later that they already exist. Then I use the real existing model to refine mine and make a few correntions. Sometimes I even trip over new ways of dealing with certain lighting problems that will give way better-than-typical quality in specific situations.

I won't disagree that it's not possible, I've just never had that situation happen so far even with the more advanced algorithems.

So anyway, let me try and give this one more stab:

1. So I now understand the texture reference. I'm actually a pretty big fan of texture approximations in place of math. The last thing that I'm not quite getting is the motivation for doing this. Is it mainly for speed during runtime, or can the results be done in no other way? What I'm getting right now is that SH functions make for extremley fast and memory efficient references (which is certainly a good motivation).

2. Ok, that's what I was thinking in the first place. So the means of storing this is very similar to AO except... I might get smaked for saying something dumb like this but, is an entire SH map stored for *each* vertex? That sounds completley insane, but it's the image I'm gathering in my head right now. Kind of relating it to storing a cube map for each vertex, although I'm fully aware that's nothing like this proposal. I guess I've been suprised before though. As an alternative to that idea, it's stored as 1 SH map per mesh, but so far that idea wouldn't make any sense to me.

3. That's even bringing more of an image to my head that each vertex has an SH map. That just sounds like a crazy amount of memory. Maybe that's why SH is so good though, because it is efficient enough to do that?

In case I'm making people mad with my ignorance :lol: , I think I'll go try to read that paer again

[kenpex]

I'm sure that you can make great things without maths. But I equally sure that you can't do some other great things without maths. For example, I just won't be able to explain the intricacies of modern shadow mapping projections (TSM etc) without maths. But enough of that.

1) speed, size... SM are only a couple of coeffs. And you usually multiply 2 SM togheter at each vertex. You can't multiply 2 textures togheter per vertex and then compute the mean of the result to see how much light you have in that vertex

2) Yes for each vertex, and not it's not insane as there only a few coefficients. Of course is still expensive, and in fact no shipped games use the full-blown (or mhm, naive) SH approach.

3) see 2)

[starstutter]

kenpex said:

I'm sure that you can make great things without maths. But I equally sure that you can't do some other great things without maths. For example, I just won't be able to explain the intricacies of modern shadow mapping projections (TSM etc) without maths.

That I do agree with, and I will never stop trying to learn math, just will always keep struggling with it. I'd like to be good at everything, but I'm not that talented ;) . But yes you're right; enough of that.

No games shipped huh? I thought that most of the Unreal games used some form of them. Bioshock, UT3 and the like. But I guess I couldn't really claim to know.

So really, for each vertex huh? That does give me a much better understanding of the whole thing. I'm beggining to really see how these work. A very attractive approach for future lighting.

Thanks very much for your patience and all your help.

[kenpex]

Nope, I think to remember correctly, and if I do then GoW for example uses only per vertex AO (and it's computed only in respect to the object, not considering interactions with the rest of the world, to ease instancing). You could have some of the benefits of SH lighting with simple AO by using the bent normals (bent in the direction of minimal occlusion) and by using a diffuse cube texture indexed by those normals... Full SH lighting is much more expensive than that. But you don't have to use the full SH solution. You can think of other kind of cheats, like not encoding the real SH occlusion in the vertices, but compute an approximation of it with the bent AO normal and the AO coeff. That actually could be (a lot) cheaper than using the same solution, with the diffuse cube texture. But the possibilities are endless...

http://c0de517e.blogspot.com

[RodgerBorland]

starstutter said:

So really, for each vertex huh?

Best ways that I think of it like twisting a cookie cutter. With one turn you can only see so much, but if it can link with more it becomes a cascade effect and can become a dynamic one. It isn't so much of packing the harmonics into a texture, as it is using the hardware to make it on a fly from only the building blocks. If it's only per vertex, you can get a bit chunky, and no one likes chunky cookies.