View topic - Corefarm.org, a collaborative rendering farm

Corefarm.org, a collaborative rendering farm

General questions related with the YafaRay Project, 3D computer graphics and about this site.

Corefarm.org, a collaborative rendering farm

Post Tue Oct 20, 2009 3:11 pm

Dear all,

First, I'd like to thank warmly the Yafaray team for its amazing achievements!

As part of my studies I recently built a desktop grid engine called naclgrid. To test its efficiency I decided to build a distributed version of yafaray on top of it and I finally set up a web service to promote this distributed version : http://www.corefarm.org.

Corefarm.org is a kind of marketplace for CPU power : on one hand some people submit their Yafaray rendering job and on the other hand people with idle computers share their CPU to render the pictures.

It's quite easy to submit a job : all you have to do is to upload an archive (let's say a gzipped tar archive) containing the xml Yafaray export file and the textures, or even the raw files.

It's even easier to participate : you can contribute simply by opening up a web page when you leave your computer. It requires the Google native client, a plugin for Firefox, and it is the safest way to share your CPU (as the Google native client guarantees that the remote code will not compromise your system). You can also use a standalone client.

This service is in its early development stage; it has been working fine for over a month in a closed environment so it's now time for new users to try it out!

Help yourself : everything is on http://www.corefarm.org, and do not hesitate to ask any question on the dedicated Google group at http://groups.google.fr/group/corefarm-discuss ! (or directly by email at william@corefarm.org).

Any feedback will be very much appreciated!

All the best,

William Le Ferrand
william.le-ferrand
 
Posts: 41
Joined: Tue Oct 20, 2009 3:07 pm

Re: Annoucement : Corefarm.org, a collaborative rendering farm b

Post Tue Oct 20, 2009 3:44 pm

Hi Willian,

Testing it right now. Thanks for this tool, we really need it!
User avatar
Samo
 
Posts: 3105
Joined: Tue Dec 20, 2005 10:39 am
Location: Spain

Re: Annoucement : Corefarm.org, a collaborative rendering farm b

Post Tue Oct 20, 2009 3:54 pm

Hi !

Thanks, I'm glad to hear it! It's an alpha release, do not hesitate to comment on ergonomic (is the ftp upload mechanism handy?), and the overall performance! There is room for improvements, I'm just waiting for your suggestions!
william.le-ferrand
 
Posts: 41
Joined: Tue Oct 20, 2009 3:07 pm

Re: Annoucement : Corefarm.org, a collaborative rendering farm b

Post Tue Oct 20, 2009 5:01 pm

Hi Willian

After working a bit with it, I have not been able to start rendering anything. Sometimes it says there is a texture missing, and others it can not find the XML file, despite the fact that all the needed stuff has been uploaded.

Also it keeps on changing the FTP login, which is a bit annoying. I use FireFTP to upload the files, I'm not sure using a FTP client is friendly for average users. Maybe you should create a files browser and loader interface in the web page.

Also when I click on 'collaborate' Firefox freezes, so I can't do nothing with it. It's doing some rendering work, but It would be interesting if I can keep on browsing while the file is being rendered, as long as the corefarm page is kept opened.
User avatar
Samo
 
Posts: 3105
Joined: Tue Dec 20, 2005 10:39 am
Location: Spain

Re: Annoucement : Corefarm.org, a collaborative rendering farm b

Post Tue Oct 20, 2009 5:40 pm

william

Can you join us in the chat room, it's pretty active there, and there are other people doing things about distributed rendering, maybe you can get ideas to improve your tool
User avatar
Samo
 
Posts: 3105
Joined: Tue Dec 20, 2005 10:39 am
Location: Spain

Re: Annoucement : Corefarm.org, a collaborative rendering farm b

Post Tue Oct 27, 2009 2:14 am

Dear all,

First of all, many thanks to the people who submitted jobs in the last few days! There has been several issues with the distributed antialiasing, some textures errors and so on, and all those jobs were very helpful to track the bugs.

A new version of the service is now online, at http://www.corefarm.org as usual; all the known bugs related to the distributed yafaray seem to be resolved. Feel free to discover new ones ;-)

All the best,

William
william.le-ferrand
 
Posts: 41
Joined: Tue Oct 20, 2009 3:07 pm

Re: Annoucement : Corefarm.org, a collaborative rendering farm b

Post Wed Oct 28, 2009 12:23 pm

Thank you for this, I'll use it right now! I'll make a new post to put it in my blog for some brazilian friends. I really like it.
Blender Brasil team - blender.com.br
Portfolio online
victorphellipe
 
Posts: 95
Joined: Sat Jan 10, 2009 9:31 pm
Location: Brazil, Brasilia.

Re: Annoucement : Corefarm.org, a collaborative rendering farm b

Post Wed Oct 28, 2009 12:54 pm

Great! Thank you!

What do you think of the web-based participate mechanism ? Is it handy or does it suck ?

Probably a standalone client for the grid would be nice too. Perhaps somebody here good at writing GUIs would like to join the project ? I'm modifing the DR yafaray to get a DR overlay for the standard yafaray, it will be much much cleaner. Anyway, feel free to email/IRC me !

All the best,

william
william.le-ferrand
 
Posts: 41
Joined: Tue Oct 20, 2009 3:07 pm

Re: Annoucement : Corefarm.org, a collaborative rendering farm b

Post Wed Oct 28, 2009 1:24 pm

Well,
I did not like really much of this web mecanism. It's handy, but not so "comfortable"... If you made a program (like vSwarm) will be better, 'cause we can acess our account directly from our desktop, you know? Send our jobs, share our CPUs... Everything that we can do in Web, we could make directly from our desktop. I'm tweaking my scene, I'll test it soon.

I hope you can understand me, my EN sucks...
Victor.
Blender Brasil team - blender.com.br
Portfolio online
victorphellipe
 
Posts: 95
Joined: Sat Jan 10, 2009 9:31 pm
Location: Brazil, Brasilia.

Re: Corefarm.org, a collaborative rendering farm

Post Tue Nov 17, 2009 7:42 am

Community forum for such project would be nice to report stuff and discuss problems. And I also would like to have a client app more than web option (which could be cool when using other computers where you don't want to install anything).
Sandking
 
Posts: 44
Joined: Thu Apr 16, 2009 7:27 am

Re: Corefarm.org, a collaborative rendering farm

Post Tue Nov 17, 2009 8:24 am

Hi Sandking !

What a coïncidence ! I've just uploaded an *experimental* standalone client for windows (are you running windows?). It's here, feel free to test it : http://www.corefarm.org/downloads/prone ... _setup.exe

Feel free to test it and feel free to discuss on the google group : http://groups.google.fr/group/corefarm-discuss . You would prefer a forum like this one? I suppose I can try to set up one.

All the best,

William
william.le-ferrand
 
Posts: 41
Joined: Tue Oct 20, 2009 3:07 pm

Re: Corefarm.org, a collaborative rendering farm

Post Tue Nov 17, 2009 8:30 am

Hi William

I believe he means that it would be good if we have our own subforum here, I'm fine with that idea as long as the Corefarm project has got some continuity and with help from everyone we can make it to take off.

(and If you agree with the idea, maybe you have another 'view')
User avatar
Samo
 
Posts: 3105
Joined: Tue Dec 20, 2005 10:39 am
Location: Spain

Re: Corefarm.org, a collaborative rendering farm

Post Tue Nov 17, 2009 4:03 pm

I found installing google native to my browser (google chrome) too complicated and following the directions on the google page I got stuck.

So I installed the prone.exe client and started contributing. after a few minutes, and before anything rendered it. I ran it through the shell and got to
Code: Select all
@@@ building job

before crashing.

a few questions:
I started using the dev version of google chrome because supposedly it has built in Google native support. However I couldn't find where to put the corefarm files so nothing happened when I tried to contribute through the website. Anyone know where the equivalent sel_ldr.exe and such are for chrome?

Also for the prone client, it always warms me that I am logged in as anonymous. I would like to log in as me if possible but couldn't find a read me or anything to let me know how.

I am excited for this project
loh
 
Posts: 250
Joined: Sat Sep 20, 2008 3:22 am

Re: Corefarm.org, a collaborative rendering farm

Post Tue Nov 17, 2009 4:12 pm

Hi !

Thanks for your comments !

Yes, sb submitted a buggy xml file that killed all the prone connected .. i'm adding sanity checks on server side right now ! It should be ok now, and there are some job on the grid right now, so feel free to contribute !

if you want to run prone from command line, you have to run prone.exe --ui true and to connect to your manager, on www.corefarm.org to configure it (and for it to get your login), or if you are not interested in the user interface, you can run prone.exe --login your_login

Regarding the native client, it is a rather unstable technology so for the moment I will remove support for it. It can run fine in chrome if you hack a little bit, but it stalls firefox, because you can't fork system threads for the moment. I will remove support for the google native client temporary.

Hope it helps,

All the best,

william
william.le-ferrand
 
Posts: 41
Joined: Tue Oct 20, 2009 3:07 pm

Re: Corefarm.org, a collaborative rendering farm

Post Wed Nov 18, 2009 11:42 am

Hum... after some first tests I'm really impressed, the system performace is quite impressive now.
Things to improve: usability, monitoring and yaf community participation.
User avatar
Samo
 
Posts: 3105
Joined: Tue Dec 20, 2005 10:39 am
Location: Spain

Next

Return to News & Discussion



Who is online

Users browsing this forum: No registered users and 4 guests

cron