Upcoming YafaRay v4

[David Bluecame]

I think it's still too early to say. The code is still very unstable and many basic features we used to have up to v3.x.x are not yet ready in v4.x.x and I will have to work hard to "port" them back to the new v4 system.

About speed, it's also difficult to say. As I don't have yet all the original features I cannot do proper comparisons. Also, I have not been focusing on speed but on functionality, taking care that the speed is not worse than before at least.

My expectation is that the new intersection foundation should have the potential to be faster than before, but it will heavily depend on the scene and the computer processor being used. In some processors I would expect similar speeds while in others it could be faster. I don't know yet.

Right now I'm using a "single ray" interface (different system but similar in concept to what was used up to v3.x.x), but there will be now the potential to use "packets" of rays instead, although I'm not sure yet how that might work. You suggested that possibility a long time ago (and it is in the issues tracker list). Ray packets might speed things up a bit and I will certainly explore that possibility after I stabilize the v4 code before releasing any Alphas.

[David Bluecame]

After some changes and corrections in the code, I got this in v4:

* 3 simple cubes, set as internal YafaRay Subdivision objects with displacement textures (one image and two different varieties of Voronoi)
* Internal YafaRay Subdivision: tesselation rate 250
* Shiny diffuse, with a bit of reflectance with IOR
* HDRI
* Photon mapping (without caustics, only diffuse photons) + Final Gather
* Render time: around 6 minutes in my AMD Phenom II (6 cores)

[Rio]

Amazing !!!
Thank you so much for this kind of feature ! I guess this method use less RAM than the displace modifier ?
This last result promise a lot

[David Bluecame]

Hello,

Sorry, Olivier, I didn't compare the RAM and speed, but I would be surprised if it's worse...

[Samo]

Hi David

I wonder if photon mapping can take advantage of embree structures. What do you thing about it?

[David Bluecame]

Hello, Álvaro.

To be honest I'm not sure... Embree is intended for object intersections and I'm not sure if/how can it be used to store photons or if it would improve anything. I will keep this in mind but I will leave it for a later stage.

As you already requested, it's still too soon (I've made quite some significant changes to the code and the Blender exporter today for example), but just as a "rough" comparison test I have attached the render logs for a modified teddy scene with both v3.3.0 and the development v4. You can see all details in the log.

This render is done with photon mapping and contains several thousand instances to create the "fur". One significant difference in the v4 is that we no longer need to create the old slow/single threaded KdTree! That saves a lot of time. For example in the v3 version, the KdTree generation takes 1min 30s, while in the v4 version a BVH is generated which takes almost no time to generate

This comparison was done with the same scene, which also contains some Blender subdivision modifiers. I could have done it better for v4 by removing the Blender subdivision modifiers and using the new direct subdivision meshes, but I wanted a "same scene comparison" here.

I hope you find it informative, but please remember this is still too soon and things can change a lot still!

The lighting in the teddy bear in v4 is somewhat different for some reason, although I'm not sure whether this is more correct in the v3 render or in the v4 render? (v4 looks better to me but I could be wrong)

teddy_v3_log.pdf

(678.97 KiB) Downloaded 714 times

teddy_v4_log.pdf

(686.52 KiB) Downloaded 563 times

[loh]

My understanding is that Embree has a lot of optimization for SIMD sets including all the levels of AVX. If the comparison renders are done on an AMD Phenom II then they do not take advantage of AVX.

I wonder how much more improvement there would be in the times on a more recent CPU that supports AVX2. 2m 5s to 1m 1s is already a lot faster.

[David Bluecame]

Hello,

Yes, you are right about that. However I don't think there will be a big improvement considering the way I'm implementing Embree at the moment (single rays).

Once I get the code "working" with the current single rays implementation, I will try to modify it to use ray packets/streams instead of single rays. I hope to be able to get better speed improvements (especially in AVX/AVX2) when using that functionality. However, it's not going to be easy to adapt the code to use packets/streams!!

If I can make that work, then I will do comparisons using different CPUs, including a 4th generation Intel i7 I could use.

[Samo]

Hi David

Thanks for the answer. 50% reduction in render time is quite impressive !!

My question about photon mapping comes from the idea that photon mapping also uses standard raytracing for calculating intersections with the scene, which are needed for calculating incoming direction to use in the photon map and mirror direction for bouncing, and that it uses kd-tree for representing the photon map. I wonder if that calculus can be done with embree and that if we can use BVH instead of kd-tree for representing the photon map. Take a look at the doc below

[David Bluecame]

Thank you, Álvaro, I will take a look.

About the speed improvement, in this case it was around 33% faster (from 118s to 79s) if we look at the render speed itself, without considering the time to build the KdTree in v3 (90s) which is almost 0s in v4 in this scene.

However, the actual speedup will heavily depend on the scene and processor. I would not expect such speedups in all cases. In some situations the rendering could even be a bit slower due to the new features added to the code.

Anyway I will continue stabilizing the code and getting some features back (I managed to get back yesterday the transparent shadows and wireframe, which were giving me a hard time). After I get all old features back, I get the new features finished and I stabilize (and organize better!) the code in general, I will try to adapt the code to ray packets/streams (if I can), I hope that might help a bit more with speed.

[Samo]

thanks David for the explanation, anyway I think embree will be a turning point for YafaRay

[oort]

David,
Thanks for all your work on improving YafaRay. The test renders are looking good. I have been watching this thread but have avoided asking a question, fearing the answer. Will this version be compatible with the Wings3D plugin or will that have to be modified to work with v4?

Thanks,
oort

[David Bluecame]

Hello, oort.

Thank you! Yes, as you can imagine v4 will be quite different. The v4 XML file will be different in many aspects. But don't worry too much, as I'm also planning to update the "yafaray_v3" Wings3D plugin to the v4 as well.

Initially I will just "make it work" with the same features the yafaray_v3 plugin can handle and later on I will try to add the new Core features to it (I will need your help in that stage but it's too soon now).

[David Bluecame]

I think I have finished today (I hope) porting the MeshLights to v4. I will make some changes to the way that Shiny Diffuse Emit objects and Meshlights are handled:

* Shiny Diffuse Emit objects will still show as "bright" objects as now, but they will no longer be used to generate photons or paths in path tracing. Emit is a "fake" bright object and not an actual light source, so having it interacting with photons and paths was not giving consistent results. In fact Bidirectional integrator was ignoring Emit objects for the ligthing calculations. Now all integrators will behave similarly respect to them.

* For "arbitrary surface light sources" we will use MeshLights. Meshlights in v3 and older cannot have textures and can only have one color for the entire surface. That will change in v4 where we can select the "one color system" (faster) or to use the object material properties (including textures) for meshlight. That will alow to have properly UV mapped meshlights that will generate coloured photons and light paths, etc.

Some examples below (also included the Logs in PDF with the render details):

* Meshlight (one color) in YafaRay v3.3.0:

meshlight_test_v3.pdf

(459 KiB) Downloaded 538 times

* Meshlight (one color) in YafaRay v4 pre-alpha (should be same speed or faster than v3):

meshlight_test_v4.pdf

(424.89 KiB) Downloaded 613 times

* Meshlight (using material properties and textures) in YafaRay v4 pre-alpha (slower than v3 because it has to get the material properties in each point of the light surface and also because there is more color noise that needs to be reduced in the second AA pass):

meshlight_test_v4_textured.pdf

(977.63 KiB) Downloaded 582 times

I hope you like this

[Rio]

Oh yeah very useful feature thanks for that !! ( and for all your work)
Did you notice the different shading on cylinder ( in the left down corner)
About the Teddy bear comparison, I prefer the rendering of v4, the shadows look darker and more accurate.
Good luck for your future work