View topic - Corefarm.org, a collaborative rendering farm

Corefarm.org, a collaborative rendering farm

General questions related with the YafaRay Project, 3D computer graphics and about this site.

Re: Corefarm.org, a collaborative rendering farm

Post Tue Nov 24, 2009 12:03 pm

Hi Samo!

What an impressive scene ! How long would it have taken on a standard computer ?
william.le-ferrand
 
Posts: 41
Joined: Tue Oct 20, 2009 3:07 pm

Re: Corefarm.org, a collaborative rendering farm

Post Tue Nov 24, 2009 12:06 pm

In my computer It will take days, it's an old PIV.
User avatar
Samo
 
Posts: 3105
Joined: Tue Dec 20, 2005 10:39 am
Location: Spain

Re: Corefarm.org, a collaborative rendering farm

Post Tue Nov 24, 2009 12:34 pm

;-)

It's pretty cool, some people participate very often to Corefarm. For the last job there was 6 users participating, (and I've 2 old PC at home permanently in the grid.). Hope those figures will increase !

Keep testing and thank you very much for the feedback !

William
william.le-ferrand
 
Posts: 41
Joined: Tue Oct 20, 2009 3:07 pm

Re: Corefarm.org, a collaborative rendering farm

Post Tue Nov 24, 2009 1:24 pm

I believe that if the prone crashing could be fixed, there would be more people collaborating.
User avatar
Samo
 
Posts: 3105
Joined: Tue Dec 20, 2005 10:39 am
Location: Spain

Re: Corefarm.org, a collaborative rendering farm

Post Wed Nov 25, 2009 3:38 pm

And probably more if the page could be reached...

Tried with several browser (Opera, Forefox, MS IE) no success....I can't get their page!

But the concept sounds nice.
HHS
 
Posts: 169
Joined: Sat Jan 10, 2009 11:26 pm

Re: Corefarm.org, a collaborative rendering farm

Post Wed Nov 25, 2009 4:14 pm

Sorry, somebody submitted something really weird and the whole website went down. It's restarted now ..
william.le-ferrand
 
Posts: 41
Joined: Tue Oct 20, 2009 3:07 pm

Re: Corefarm.org, a collaborative rendering farm

Post Wed Nov 25, 2009 8:48 pm

Nice- but this is only really working, when enough user participates- for the Yafray benchmark it needed 11 minutes with just one user. My PC would have been much faster!

But anyway- I like this idea- prone is downloaded! :D
HHS
 
Posts: 169
Joined: Sat Jan 10, 2009 11:26 pm

Re: Corefarm.org, a collaborative rendering farm

Post Wed Nov 25, 2009 11:25 pm

Hi!

Welcome to the corefarm community !

Well, corefarm is supposed to be rather fast, but rather on 'big jobs', because on small one too much time is lost communicating with clients compared to the time spent computing. Moreover, corefarm is asynchronous distributed computing, and this implies I can't know precisely when a client stop working. So there is a basic timeout algorithm that is definitely crap on small examples.

Hope you'll test the corefarm again - and get nice results ! (by the way, don't hesitate to post your benchmarks on corefarm's forums)

All the best,

william
william.le-ferrand
 
Posts: 41
Joined: Tue Oct 20, 2009 3:07 pm

Re: Corefarm.org, a collaborative rendering farm

Post Wed Nov 25, 2009 11:56 pm

Hi,

I am contemplating some initial tests of corefarm, and I have a couple of questions.

I run the latest 0.1.2 yafaray testing build on my system for personal use, but I take it corefarm uses stable 0.1.1? So if I were to contribute to the corefarm network, every time i do so I must first switch to 0.1.1 on my system, and then reinstall prone?

Also, are there any guidelines for maximum size of a submitted scene, like for RAM usage, render resolution, etc.?


Thanks for investing the time in the corefarm project, I think it's a great thing for yafaray.
User avatar
wizofboz
 
Posts: 235
Joined: Tue Sep 16, 2008 7:00 pm

Re: Corefarm.org, a collaborative rendering farm

Post Thu Nov 26, 2009 12:09 am

Hi Wizofboz !

Yes, Corefarm is based on yafaray 0.1.1. If you are running windows, your prone installation is totally independant of your yafaray installation (I mean, yafaray is installed where it used to, while prone installed another yafaray 0.1.1 for its own usage), so you don't have to worry about the version issue .... as long as the XML description file is compatible with yafaray 0.1.1 !! (I've strictly no idea about it, but you can test !)

If you built prone, I hope you set the path correctly; if you didn't, it's likely you had already erased your 0.1.2 installation.

Regarding the guidelines, honestly I have no idea of the maximum job the corefarm can cope with. There has been some jobs with resolutions above 7000x7000 for rather complex scenes, and there was no problem at all. One more time, the best way to find out is to test (progressively, don't submit an enormous job just to crash the farm!), and to post the benchmarks on the dedicated forum (www.corefarm.org:81/phpbb). I suppose that the limitation today is the amount of RAM available on the client side. But anyway, corefarm should be able to cope with rather large scene without complaining too much.

A practical limitation in fact is the available bandwidth on the server; if you submit a job with a lot of textures, the server may hang for a minute or two, the time to send the textures to the clients .. Corefarm is not loadbalanced, there is a single server handling all the traffic (web service + backoffice + job queues + shared memory .. ) . If the project grows up (advertise about it if you like it!), more servers will be needed, but it is another problem.

Thanks for those questions, I hope my answers are clear enough !

All the best,

William
william.le-ferrand
 
Posts: 41
Joined: Tue Oct 20, 2009 3:07 pm

Re: Corefarm.org, a collaborative rendering farm

Post Thu Nov 26, 2009 11:04 am

Hi everybody:

First of all I want to thank William for all his effort and enthusiasm with this project. I think the idea is really good and I expect really good things about all this.

The thing worries me most, is that this is a one man project, so I fear you would get sick of all in no time with people asking for new features, errors, questions, etc, etc...
My advice is I think it´s better you do things little by little rather than triying to make all by one and get bored of it in no time. I really would like to see this project madurate, because this feature is something Yafaray needs (or at least, is another good feature to use Yafaray).

And now, my complainings and requests :lol:

- I use Windows XP with SP2 and I use Prone 0.09. I have checked the instructions you gave Samo and I have everything ok, but my Prone crushes a lot. I´m not sure about this point, but I think it crashes after each time it contributes.
This is a bit annoying as I dont want to check my computer every hour to see if Prone is ok or out.
As Samo has said before, I think this is the main issue it should be fixed. All the other things can wait, but people would contribute more (at least, more time) with a more stable Prone.

- I think you should put a FAQ section in the web page, with questions as: "Does the threads configuration in the XML archive influence in the speed the archive is rendered?", "Wath happens when I get negative credits?" and others.

- A lot of people is asking for Luxrender support. There is a way similar project with Lux, so there is no reason to waste your time in something other people is working on: http://www.luxrender.net/forum/viewtopi ... =16&t=2808

I participate in a spanish 3d forum, so I´m triying to make a bit of publicity about Corefarm:
http://www.foro3d.com/f20/corefarm-gran ... 81958.html

By the way, I like a lot the design of the web page and the logo. It´s simple and neat :)

Another time, many thanks to you for all your effort and, of course, to all the Yafaray team.

Greetings!
User avatar
Arkinauta
 
Posts: 129
Joined: Thu Feb 05, 2009 11:22 am
Location: Spain

Re: Corefarm.org, a collaborative rendering farm

Post Thu Nov 26, 2009 2:52 pm

Yes, that's a great project! :D

Unfortunately I have one big problem: job rejected; missing camera

I already took a look into the .xml, the camera is specified, and I checked the name of the camera which I offered to the corefarm...

Anyone else with the same problem here? Solution?

Cheers
HHS
HHS
 
Posts: 169
Joined: Sat Jan 10, 2009 11:26 pm

Re: Corefarm.org, a collaborative rendering farm

Post Thu Nov 26, 2009 10:06 pm

Hi HSS !

Sorry, I was out all day long ..

I checked your job manually, ok, you specified a camera ... but in fact the parsing of your XML file failed because there are characters with accents into : Saülen, etc etc .. Hey Yafaray guys, this is a bug in your XML parser ;-)

Just remove those weird labels and it will work fine .. Sorry for that bug !

But on the other hand, there are now 130 computers connected ! Enjoy !

All the best,

William
william.le-ferrand
 
Posts: 41
Joined: Tue Oct 20, 2009 3:07 pm

Re: Corefarm.org, a collaborative rendering farm

Post Fri Nov 27, 2009 10:46 am

Hi Arkinauta, hi everyone,

Sorry for answering so late, but I have some fresh news : there is a new version of prone that seems to be much more stable. You can download the installer on http://www.corefarm.org/participate.

Arkinauta, thank you very much for your kind sentence. Right now I plan to maintain the corefarm as-is, with the same functionnalities. With mates, we are investigating on the support for animation rendering, but there is still some work to do on the algorithms, it is not only a matter of implementation.

Regarding your FAQ suggestion, well, it's an excellent idea ! I added it , you can find the link in the footer page. Don't hesitate to suggest improvements for this faq page !

The design was done by a friend of mine, Grégoire, you can look at his portofolio here http://www.gdegevigney.fr/

Thanks for the advertisement on the spanish 3d forum, I hope the service will be useful to more and more people over weeks ..

All the best !

William
william.le-ferrand
 
Posts: 41
Joined: Tue Oct 20, 2009 3:07 pm

Re: Corefarm.org, a collaborative rendering farm

Post Sun Nov 29, 2009 12:16 pm

Prone crashes with job 1418, on WinXP
Code: Select all
Registered material type 'mask_mat'
Registered integrator type 'SingleScatterIntegrator'
Registered integrator type 'SkyIntegrator'
Registered volumeregion type 'SkyVolume'
Registered light type 'spherelight'
Registered light type 'spotlight'
Registered light type 'sunlight'
Registered background type 'sunsky'
Registered background type 'textureback'
Registered background type 'constant'
Registered volumeregion type 'UniformVolume'
Registered volumetric handler type 'beer'
Registered volumetric handler type 'sss'
@@@ Plugins loaded from location C:\Archivos de programa\Corefarm/plugins/
@@@ Building job
@@@ Textures extracted (8)
DEBUG : size of the scene description:
@@@ Start Docunent
startEl_scene
starting <texture>
DDD entering the texture module
Size of the array : 1550047
Init source
Loading image 2168 x 3888
color depth is 24 bits.
creating gamma LUT
OK
added Texture 'Tex.005'!
DDD, ok, texture handled correctly
startEl_scene
starting <texture>
DDD entering the texture module
Size of the array : 3046155
Init source
Loading image 2168 x 3888
color depth is 24 bits.
creating gamma LUT
OK
added Texture 'Tex.006'!
DDD, ok, texture handled correctly
startEl_scene
starting <texture>
DDD entering the texture module
Size of the array : 364948
Init source
Loading image 1600 x 1058
color depth is 24 bits.
creating gamma LUT
OK
added Texture 'Tex.003'!
DDD, ok, texture handled correctly
startEl_scene
starting <texture>
DDD entering the texture module
Size of the array : 364948
Init source
Loading image 1600 x 1058
color depth is 24 bits.
creating gamma LUT
OK
added Texture 'Tex.004'!
DDD, ok, texture handled correctly
startEl_scene
starting <texture>
DDD entering the texture module
Size of the array : 551216
Init source
Loading image 1600 x 1064
color depth is 24 bits.
creating gamma LUT
OK
added Texture 'Tex.011'!
DDD, ok, texture handled correctly
startEl_scene
starting <texture>
User avatar
Samo
 
Posts: 3105
Joined: Tue Dec 20, 2005 10:39 am
Location: Spain

PreviousNext

Return to News & Discussion



Who is online

Users browsing this forum: No registered users and 3 guests

cron