DEoptim in Parallel
[This article was first published on FOSS Trading, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.
Running DEoptim in parallel has been on the development team’s wishlist for awhile. It had not been a priority though, because none of us have personally needed it. An opportunity arose when Kris Boudt approached me about collaborating to add this functionality as part of a consultancy project for a financial services firm.Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.
We were able to add and test the functionality within a week. The latest revision of DEoptim on R-Forge has the capability to evaluate the objective function on multiple cores using foreach. Very CPU-intensive problems will see speed increases in approximately linear time (less communication overhead).
I gave a short presentation (PDF) on the parallel functionality at the Saint Louis R User Group meetup in February. A longer-running version of the code used in the presentation is on R-Forge, in the file DEoptim/sandbox/largeN_doSNOW.R (revision 86).
There are a few things to keep in mind when using the parallel functionality. I quote from the meetup presentation:
- Data communication between nodes can overwhelm gains from processing on multiple CPUs
- Be careful with non-varying objects
- Exclude them from formal function arguments
- Copy them to nodes before optimization (clusterExport)
- If mu and sigma were formal function arguments, they would be copied to each node for all 2037 function evaluations!
To leave a comment for the author, please follow the link and comment on their blog: FOSS Trading.
R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.