Self Shadowing artifacts - Shadow Bias and Minimum Ray Dist

[David Bluecame]

Hello,

There are several bug reports about black spots in some scenes. They seem to be caused by an incorrect self-shadow calculation, probably because the Shadow Bias and Minimum Ray distance parameters are not the adequate for the scale of the objects in the scene.

I don't yet understand the internals of YafaRay nor the render algorithms, however I had some time ago a discussion with DarkTide about this. See:


So, I want to suggest this solution: to expose the Shadow Bias to the user, just in case there are black dots in that particular scene. If the user does not enable the "Custom Shadow Bias" it will keep using the old value 0.0005 and min ray distance 0.00005. If the user enables the "Custom Shadow Bias", it will use the new Shadow Bias in the scene and automatically set Min Ray Distance to 1/10th of the Shadow Bias.
http://www.yafaray.org/node/570#comment-641

I have sent two pull requests, one for Core and another for Blender-Exporter:
https://github.com/YafaRay/Core/pull/77
https://github.com/YafaRay/Blender-Exporter/pull/22

Some examples, with the default bias and black dots and with the custom shadow bias. Please let me know what you think about all that. Of course, ideally we should have some kind of automatic bias calculation, but that's beyond my knowledge right now...

I will include this new feature in my next experimental builds... I hope you find them interesting.

[Samo]

You are tha man !

Btw, has got this shadows bias anything to do with the terminator problem, because this is a know issue in YafaRay we still don't have solution for:
http://www.yafaray.org/node/135

[wizofboz]

Although I understand the motivation for doing this, I don't believe this is a proper solution. Shadow bias is a technical detail that should not be exposed to users, IMO yafaray should automatically use correct bias value based on the scene being rendered.

Look at this post in the bug report:
http://www.yafaray.org/node/335#comment-613

Bert suggests basing shadow bias on scene size, I think you should at least experiment with this approach before doing what is proposed here.

BTW what happens to your shadow bias value in the interface if shadow bias is changed at compile time in UserConfig.txt? Is this taken into account?

[David Bluecame]

Hello, wizofboz

I fully agree with you. I don't like this solution either and I see it more as a "dirty patch" for now than as a real solution. However, I wanted to give some solution, even if it's temporary, for the users having these problems, as until now there was none for them when they had such black dot problems.

I also agree that we need to calculate these parameters automatically based on the scene size somehow. I need to investigate how to get that information from the YafaRay structures or maybe from Blender? Perhaps using the scene size is not a good idea because if the camera is zooming at a small portion of the scene, I believe it should calculate the bias / ray min dist based on the camera view extents size more than the total scene size (or maybe I'm wrong, not sure about this at all!?)

This particular pull request will change the system so it does not use the UserConfig.txt, but I can change it easily so the default values come from UserConfig.txt again.

About the terminator problem, that seems to be related to a low poly object with surface declared smooth and all mixed with the shadow bias. Low poly / smooth objects are difficult in most renderers (even LuxRender has problems with low poly glass / smooth objects as they show darker areas due to normals/rays/smooth calculation interactions.

Probably a smart shadow bias / ray min dist and increasing a bit the poly density of these smooth objects should minimize the terminator problems. Of course, there could be yet unknown bugs in the code, we cannot rule that possibility out.

[Samo]

I think YafaRay has got some kind of scene bounding box parameter calculation hard coded, at least it seems to be using one for shooting photons from a sun lamp, in the sense that the smaller a scene becomes, the more dense is the flow of photons, so it is taking into account scene bounding box in some way.

[wizofboz]

David Bluecame: That's great, I was hoping you felt this way. In this case I fully support giving users a way to possibly fix their renders until a better solution can be worked out.

I'm not totally sure, but I suspect that the camera position or view does not play a big role in this, but further experimentation can determine that.

Samo: yes I think you are right, a scene bounding box is defined, looking at the code I think it might be called sceneBound (look in scene.h and scene.cc for details). If this is correct it would be easy enough to get the size of this since it is a bound_t object and there are already functions for getting the bounding box size.

[David Bluecame]

Hello,

DarkTide told me that exposing the bias parameter was not a good idea.

So, I've closed my previous pull request and I've prepared a highly experimental change to try to automatically calculate the Shadow Bias and Ray Min Dist based on the scene size.

The new pull request is in:
https://github.com/YafaRay/Core/pull/80

I will try to make another experimental build as soon as possible with this change so it can be extensively tested. In my tests it fixed automatically the black dots caused by self-shadow in all cases except in very extreme ones (very big or very small objects).

I hope it can be used in the official YafaRay, but as it affects the very fundamentals of the entire renderer it should be very extensively tested, so I expect some adjustments may be needed before using it in YafaRay.

In any case, I hope you find it interesting.

[David Bluecame]

Hello,

I've prepared the new Experimental v0.1.99-beta2 version including the Shadow Bias auto calculation. Please let me know how it works for you:

More info and download links here:
http://www.yafaray.org/community/forum/viewtopic.php?p=31078#p31078

[Samo]

hi David

Thanks for fixing this. Indeed this is a better fix, since it will benefit other packages apart from Blender. Can you share the scenes you are testing this fix against, please? I will add them to my scenes collection and also I will use them for the release notes.

[David Bluecame]

Hello, Álvaro.

I've uploaded the example scenes and images of both wrong results with the fixed bias and good (or at least better) results with auto bias calculation here. I hope this is what you wanted:
https://github.com/DavidBluecame/Blender-Exporter/releases/download/v0.1.99-beta2/Self.shadow.bias.scenes.zip

I also believe this is a better fix than before. However, as povmaniaco pointed out here (https://github.com/YafaRay/Core/pull/81) it's possibly not the best solution as there could be big scenes with small triangles.

As I wrote in my answer to him, I believe there are four choices here:

* The original system (fixed bias), but it's known to cause artifacts in some scenes and the reason why I made this change.
* Exposing the bias parameter. This is what I wanted to do, to give the users the control about it so they could solve any issues with too big or small bias. However it was decided to avoid exposing it to the user.
* This new calculation depending on scene size. Not ideal, if you have a big scene and take shots of interior details, perhaps the bias will be too big and cause artifacts. We have to see what happens now.
* A probable better option would be to do a per-triangle shadow bias/min ray dist. I already made some tests with it. However, it's more "expensive" in computation time and slows down the render. Also, there is a risk of having inconsistent shadows if the bias is different for each triangle. In any case it's the solution that I think makes more sense in the long term...

In any case, let's see what happens with this experimental version for now, please let me know if you have any issues with it.

[Samo]

can a scene be created were we test big meshes and small triangles at the same time? only testing will tell us which solution is better.

[David Bluecame]

Hello,

I've made a test scene. You can see it with example images with both the original fixed bias system and the new scene-based auto bias calculation.

https://github.com/DavidBluecame/Blender-Exporter/releases/download/v0.1.99-beta2/cubos_yafaray_shadow_bias_tests_01b.zip

In the example, I've set the scene to have 20 frames and several Keyframes with different sizes of the "floor" plane, from scale=5 to scale=10000 (non linear).

It does not look too good. You can see different results depending on the size of the floor plane (which affects the size of the scene, used as a basis for the bias calculation). With very big plane scales from 1000 to 10000, the artifacts are really bad. Not sure if such big scale differences between total scene and the objects being rendered would happen in real scenes, though...

I will experiment with other approaches like triangle-based bias calculation, but it's more computationally expensive and will slow down the rendering in general, or perhaps object-bounds based?

[Samo]

I am preparing the tower scene before showing it to the world to test shadows bias. It is very big and at the same time with lot of detail and small triangles. What you see in the last image is in fact self shadowing kicking in.

[Samo]

tower scene with self shadowing issues
https://drive.google.com/file/d/0B9Xy9o ... sp=sharing

rendered with 0.1.99 beta2 Linux64bis

[David Bluecame]

Hello, Álvaro.

Now I'm getting confused... I've just rendered your scene with Experimental v0.1.99-beta2 Linux 64bit and it looks ok to me with no visible self shadow artifacts (??)