White dots in rendered image

Project:YafaRay
Component:YafaRay Core
Category:bug report
Priority:critical
Assigned:David Bluecame
Status:closed
Description

With Path Tracing active and the following settings:

with transparent shadow = TRUE --> we get white dots in our renders

when set to FALSE renders are normal.

John by ACCA Software S.p.A

AttachmentSize
Rendering_PathTrace_TranspShadow_01.jpg224.57 KB
Rendering_PathTrace_NOTranspShadow_02.jpg206.4 KB

Comments

#1

hi John

Can you please post the YafaRay version your are using in Edificius and how do you compile it? This issue was also reported here:

http://www.yafaray.org/community/forum/viewtopic.php?f=23&t=5019

and it seems fixed with latest compilations of Yafaray 0.1.99  beta

BTW there is a workaround which is using a blend material.

#2

Hi Samo

thanks for the reply, the used version is "0.1.99 beta 3" 32-bit

downloaded directly from your site.

greetings

John  by ACCA Software S.p.A.

#3

Assigned to:Anonymous» David Bluecame

Hello,

Please could you provide the scene or at least a portion of the scene where this problem also happens?

Thanks and best regards!

#4

I can confirm bug on Blender & YafaRay 32 and 64bits for Windows, rendered with YafaRay 0199 beta3, using scene from this thread:

http://www.yafaray.org/community/forum/viewtopic.php?f=23&t=5019 (remember to unpack textures in texture panels)

White dots number and distribution changes with either path samples or path depth changes.

Bug does not show up in Blender & YafaRay 0199 beta 3 32bits and 64bits on linux mint 64bits.

AttachmentSize
0199beta332bits.jpg 10.13 KB
0199beta364bits.jpg 6.91 KB

#5

Hi Samo,

we hope that the problem will be solved for the 32-bit version.

For the moment we will not use the "path tracing" until there will be a better version !!!

thanks and see you soon ..

ACCA software development team.

#6

I think the problem comes exclusively from mapped transparency + path tracing, since path tracing has no issues rendering full or partially transparent consecutive meshes which don't use any mapping. I would look in the code for any division where denominator gets close to 0. This explains also why you can work around this issue with a blend material, where one material is fully transparent and the other is leaves color mapping, using a leave siluette as blend material texturing mask.

#7

example of workaroung using a blend material, rendered with path tracing

File:

https://www.dropbox.com/s/cmftpi0i0hf8ijc/pineworkaround.zip?dl=0

AttachmentSize
workaround.jpg 4.6 KB

#8

Thanks Samo

the solution you indicated was already of our knowledge, unfortunately in this case we can not change the type of material.

We wanted to also enter the path tracing among our options, but given the problems put away in better times. For us, this is not a serious problem but the reflection on the glass is much more important ..

with such esteem


John

ACCA Software S.p.A.

#9

Status:active» ready to commit

Hello,

This has been quite difficult, but I think I found some kind of fix for this problem. During my tests I could see it also happens in Linux 64 bits, but in less extent. In Windows 32 it's much worse.

The root cause is still unclear to me, but there is a point in the path tracer where, if the first sample operation fails for some reason, for the next samples it uses a non-initialized value in one of the parameters. This explains why the white dots appear "randomly"

The pull request is here:

https://github.com/YafaRay/Core/pull/88

 

I would appreciate that Rodrigo could take a look at it, because even when I hope it make things better, I believe a senior programmer with more experience and deeper understanding of YafaRay should take a look at this point of the program to see if this is the best way to proceed.

For now I will mark it as "ready to commit" and I will include this fix in the upcoming beta5.

I've attached some images in Linux 64 and Win XP 32bit before and after the fix.

AttachmentSize
pine path linux64 before fix.png 26.58 KB
pine path linux64 after fix.png 25.61 KB
pine win32 before fix.png 28.57 KB
pine win32 after fix.png 25.36 KB

#10

Hello,

I've quickly made a "temporary" beta4a version for Windows 32bits including this patch so you can try it. I also included some of the fixes I've applied lately, like the problem with the bump mapping artifacts and the white dots in rough glass that Alvaro reported in other issues tracker entries etc.

The test build is here:

https://github.com/DavidBluecame/YafaRay---unofficial-Builds/raw/c19584a...

Please let me know if it works better. Depending on the results of your tests, I will have to modify this fix or I will include it in the upcoming beta5 version.

#11

Hello David,
First of all we thank you for your great work!!!

From the tests, the latest beta (4a) is the best release. There are still other issues such as transparency in the volumetric effects, but we are confident that over time you manage to solve these.

I send you the post of the various tests performed:

#12

#13

2) Normal (bump) mapping problem

AttachmentSize
NormaMap_RealTime_Edificius_01.jpg 530.67 KB
NormaMap_DirectLight_average_Beta4_02.jpg 179.43 KB
NormaMap_DirectLight_average_Beta4a_03.jpg 158.96 KB

#14

3) rough glass problem

 

Thank you very much and see you soon...

John

ACCA Software S.p.A.

AttachmentSize
RoughGlass_DirectLight_average_Beta4a_01.jpg 61.77 KB
RoughGlass_DirectLight_average_Beta4_02.jpg 61.51 KB
RoughGlass_PhotonMap_average_Beta4a_03.jpg 148.98 KB
RoughGlass_PathTracing_average_Beta4a_04.jpg 226.62 KB
RoughGlass_PathTracing_average_Beta4_05.jpg 227.19 KB

#15

Thank you, John!

Please will you send me the XML with the example of the wall?  I can see that the original bump artifact is no longer there, but it was a very crude fix and I think it's not yet really correct in beta4, although the artifact is not so obvious now. I would like to do a better job fixing the issue in the bump to get a proper result.

About the problem you mentioned about an issue with transparency and volumetrics, please will you create an issue with the explanation of the problem and some examples so I can try to replicate it here?

Thanks and best regards!

#16

Hello,

this is the XML with the example of the wall.

Thanks and best regards!

AttachmentSize
Wall_Example.zip 815.22 KB

#17

great work David !

#18

Thank you, Alvaro!

John: about the artifact with the wall you sent me. After investigating a lot I believe the problem is in the actual model, as there seem to be duplicated faces.

For example, if you do this change to remove the duplicate faces in the front, rendering with beta4 (not beta4a!) would give you a correct render:

Change this:

<set_material sval="Material_0" />
<f a="0" b="1" c="2" uv_a="0" uv_b="1" uv_c="2"/>
<f a="1" b="0" c="3" uv_a="3" uv_b="4" uv_c="5"/>
<f a="2" b="1" c="0" uv_a="2" uv_b="1" uv_c="0"/>
<f a="3" b="0" c="1" uv_a="5" uv_b="4" uv_c="3"/>

Into this:

<set_material sval="Material_0" />
<f a="0" b="1" c="2" uv_a="0" uv_b="1" uv_c="2"/>
<f a="1" b="0" c="3" uv_a="3" uv_b="4" uv_c="5"/>

The very crude "patch" I included in beta4a for the bump mapping is not giving correct results. I'm working in a better way of doing it, so for rendering the wall please use beta4 and not beta4a

AttachmentSize
wall rendered ok with beta4 after removing duplicated faces.png 231.41 KB

#19

Thanks for your reply,

you're right when referring to the model but we needed to use the double face because otherwise we get an abnormal "cut" along the surface that seems to split up the texture too.

Please see the attachment for details on what happens to the surface when we don't use a double a face.

Thanks and best regards!

John

AttachmentSize
Rendering_DirectLight_yafa_0.1.99.Beta4_.jpg 47.97 KB
OD_331F.zip 8.34 KB

#20

Hello, John.

That's not nice indeed. Please will you send me the same scene in "OBJ" so I can import it in Blender?

Thanks!

#21

Hello, John.

That seems to be a self-shadow artifacts.

Since v0.1.99-beta2, I put an automatic shadow bias calculation system in place to reduce that kind of problem. However, due to the limited precision of float and the many different possible scenes you can create, the automatic calculation cannot solve all possible self shadow problems.

So, since v0.1.99-beta4 I included also new parameters to be able to enable/disable the automatic bias calculation as well as the ability to control those parameters. For example, if you add these parameters to the Render section of the XML, your example xml file renders correctly (see attached) without need for duplicated faces:

    <adv_auto_min_raydist_enabled bval="false"/>
    <adv_auto_shadow_bias_enabled bval="false"/>
    <adv_min_raydist_value fval="5e-03"/>
    <adv_shadow_bias_value fval="0.05"/>

The tricky part is to calculate the best possible min_raydst and shadow_bias values. Usually, min_raydist is 1/10 of shadow_bias. The default shadow bias is 0.0005 (5e-04) and the default min raydist is 0.00005 (5e-05). If you have self-shadow artifacts, you can try to increase both values (for example multiply them both by 10). If not enough, try multiplying them again by 10, etc.

In more difficult scenes, where you cannot get them right, you may need to tweak separately shadow bias and min raydist.

Unfortunately doing a better render without any selfshadow artifacts would require big changes in YafaRay and the problem is that those changes would cause a significant slow down in the rendering. I hope that, for now, tweaking those parameters solve your problems.

AttachmentSize
yafray.png 174.1 KB

#22

Hello,

I've made another "temporary" win32 build "v0.1.99-beta4b". You can download it from:

https://github.com/DavidBluecame/YafaRay---unofficial-Builds/raw/master/YafaRay%20v0.1.99-beta4b%20TESTING%20(2015-07-25)%20Win32.zip

The changes are:

* Better (I hope) bump map rendering. This should solve the bump map triangular artifacts Alvaro was having. However, it will not solve the problem in your "wall" model, you need to remove the duplicate faces and it should render fine. If you have self shadow issues with models without duplicated faces, try the shadow bias / min raydist tweaks as explained before. I've made quite a few changes to bump mapping so please test it as much as you can.

* New parameter to enable/disable lights. In the XML, into each "light" section you can add:

<light_enabled bval="false"/>

or

<light_enabled bval="true"/>

* New (material) parameter to enable/disable casting shadows. In the XML, into each "material" section you can add:

<cast_shadows bval="false"/>

or

<cast_shadows bval="true"/>

 

Please let me know if it works for you. If it's ok, this will likely become the next v0.1.99-beta5 and I will release it for Linux/Windows 32/64bits.

I hope it helps. Best regards.

#23

OK! thanks for your help.

I just wanted to point out that the above problem only occurs when <type sval = "orthographic" />

When sent to "Perspective" - the resulting render is correct.

However, we'll give the latest build a look as soon as possible and let you know.

Thanks and best regards!

John

#24

We're now in the testing phase and the results are quite promising.

However, as already requested in past threads we'd like to be able to show shadows received by an object, but without showing the object. Is this goint to be implemented soon?

Another issue is being able to disable shadows generated by specific lights. We see that this part of the code already implements this feature but not currently available.

 

please advise on this.

 

http://www.yafaray.org/community/forum/viewtopic.php?f=16&t=3389

 

Thanks.

#25

Hello,

I found an issue in this new "cast_shadows" option. If you have an object "A" positioned inside the shadow of another object "B", if the object "A" has "cast shadows=false", then behind "A" you will not see the shadow casted by "B".

In this case "A" should not cast shadows by itself, but the shadow coming from "B" should still be visible as a dark area.

I've sent a patch to fix this. I will include it in the next beta.

About the other two options you mentioned:

* YafaRay is not currently able to set per-object parameters. Adding that functionality would require a lot of work. So, if possible I would like to try adding the functionality you suggest as per-material, until we can develop a per-object parameter system in the future.

* Option to disable all shadows generated by a certain light: I will include DarkTide's changes in the next experimental version.

* Option to cast shadows only but not seeing the object. Please will you explain a bit more why this is needed? I believe this would be difficult to implement, but I can investigate a bit to see how difficult...

#26

Hello,

I've uploaded a new beta4c for win32 so you can test it. It's located here:

https://github.com/DavidBluecame/YafaRay---unofficial-Builds/raw/master/YafaRay%20v0.1.99-beta4c%20(2015-07-29)%20TESTING%20win32.zip

Please, this is a VERY experimental version, just for you to test.

 

The changes are, as requested:

* Fixed the object not casting shadows removing shadow from other objects

* Ability to control per-light shadow casting. In blender exporter this is in the new "advanced settings" at the very bottom of the light panel. In the xml file, you can add this to the "light" sections:

<cast_shadows bval="true"/>

or

<cast_shadows bval="false"/>

 

* Modified the per-material shadow control, removing the recently added cast_shadows parameter.

I've added instead a per-material "visibility" enum parameter that can have the following values:

'normal' (default): Normal - Normal visibility - visible casting shadows.
'no_shadows': No shadows - visible but not casting shadows.
'shadow_only': Shadows only - invisible but casting shadows.
'invisible': Invisible: totally invisible material.

This new parameter is at the bottom of the material panel, in the new advanced settings. In XML it would be something like, in the material section:

<visibility sval="shadow_only"/>

In the xml, you can NO LONGER use the per-material "cast_shadows" parameter, use the new "visibility" instead, for example:

<material name="Material.005--9223363294108090560">
    <IOR fval="1.52"/>
    <absorption r="1" g="1" b="1" a="1"/>
    <absorption_dist fval="1"/>
    <dispersion_power fval="0"/>
    <fake_shadows bval="false"/>
    <filter_color r="1" g="1" b="1" a="1"/>
    <mirror_color r="1" g="1" b="1" a="1"/>
    <transmit_filter fval="1"/>
    <type sval="glass"/>
    <visibility sval="shadow_only"/>
</material>

<material name="Material.003--9223363294108090688">
    <IOR fval="1.8"/>
    <color r="0.669211" g="0.669211" b="0.669211" a="1"/>
    <diffuse_reflect fval="1"/>
    <emit fval="0"/>
    <fresnel_effect bval="false"/>
    <mirror_color r="1" g="1" b="1" a="1"/>
    <specular_reflect fval="0"/>
    <translucency fval="0"/>
    <transmit_filter fval="1"/>
    <transparency fval="0"/>
    <type sval="shinydiffusemat"/>
    <visibility sval="normal"/>
</material>

<material name="Material.002--9223363294108090752">
    <IOR fval="1.8"/>
    <color r="0.669211" g="0.0118081" b="0.00124767" a="1"/>
    <diffuse_reflect fval="1"/>
    <emit fval="0"/>
    <fresnel_effect bval="false"/>
    <mirror_color r="1" g="1" b="1" a="1"/>
    <specular_reflect fval="0"/>
    <translucency fval="0"/>
    <transmit_filter fval="1"/>
    <transparency fval="0.815013"/>
    <type sval="shinydiffusemat"/>
    <visibility sval="invisible"/>
</material>

 

<material name="Material.004--9223363294108090624">
    <anisotropic bval="false"/>
    <as_diffuse bval="false"/>
    <color r="1" g="1" b="1" a="1"/>
    <diffuse_color r="0.669211" g="0.669211" b="0.669211" a="1"/>
    <diffuse_reflect fval="1"/>
    <exp_u fval="50"/>
    <exp_v fval="50"/>
    <exponent fval="500"/>
    <glossy_reflect fval="0"/>
    <specular_reflect fval="0"/>
    <type sval="glossy"/>
    <visibility sval="no_shadows"/>
</material>

<light name="sunLamp">
    <angle fval="0.5"/>
    <cast_shadows bval="true"/>
    <color r="0.834798" g="0.873978" b="1" a="1"/>
    <direction x="0.566393" y="0.218391" z="0.794672"/>
    <light_enabled bval="true"/>
    <power fval="0.7"/>
    <samples ival="16"/>
    <type sval="sunlight"/>
</light>

 

I've done this new beta very quickly so please test it thoroughly. I hope this is what you need.

#27

OK!...

thanks for your great support! Very much appreciated!

We'll be testing these new features and very sure results will come as desired.

Please consider that as from next Monday, 3rd August, ACCA will be closed for three weeks for the summer holidays. We'll be back on the 24th August!

Best Regards

John

#28

Hello David,

we tested the beta version 4c seems fine !!!
The Bug fixes and new features seem to work properly !!!
For now it's fine.
If there are problems I will let you know.

Talk to you soon


John

#29

That's great, John, thanks for testing it.

If we don't talk before, I wish you a very nice summer vacation!

Best regards.

#30

Just one more thing before I forget.

We need an extra flag to control shadow reception for each material.

 

That's all for now.

 

Thanks

John

#31

Hello David,

Thanks to recent changes we have solved our problem with the shadows cast by a billboard.


Thanks

John

AttachmentSize
After.png 353.14 KB
Before.png 414.02 KB

#32

Hello David,

sorry, the images are inverted.

John

#33

Hello David,
now we are operating again !!!
We will continue testing the beta 4c and if there are problems we will notify you.

Best Regards

John

#34

Hello David,

It's possible to add an extra flag  to generate a material that does not receive and does not cast shadows.

Best Regards

John

#35

Hello, John.

Right now I'm very busy preparing the next major new experimental version of YafaRay with several new features, for example this one that perhaps could be interesting for you: http://www.yafaray.org/node/574

Once I finish it (somewhere in the next 2 months I hope!) I will try to implement the feature you are asking for.

Best regards!

#36

Now, the forum is here?? Please, here is a bug tracker. Use the forum for 'normal' talk, because more people visit the forum threads..So, can follow these changes / fix's. Thanks..

#37

Hello David,

As always we thank you for your commitment to making YafaRay a great engine best.
We wait with enthusiasm the news that soon you release, expect news from you for the new version, and in the meantime we wish you a good job to you and all the staff of YafaRay !!!

Best regards
John
ACCA Software

#38

Hello povmaniac,


You're right, we started from a BUG and we were in this discussion for the news. Next time we will use specific forums.
With the report and thanks for a job well done.

John

#40

Hello,

Thank you, povmaniac, for reminding us that we were "drifting" from the bugtracker original purpose.

Just to close and clarify this issue, the original problem described in this issue will be fixed in the upcoming next major YafaRay Experimental version.

#41

Status:ready to commit» fixed

Fixed in YafaRay-E (Experimental) v1.0.0

 

For more information, see:  http://www.yafaray.org/community/forum/viewtopic.php?f=12&t=5117

 

If you still have this issue with the new version please let us know.

#42

Status:fixed» closed

Automatically closed -- issue fixed for 2 weeks with no activity.