R parallel on multi-CPU Windows machine

Peter B Source

Will R properly recognize all the cores on a multi-CPU Windows 10 Pro 64-bit machine? We are designing a parallel computing system with Intel CPUs, using two CPUs on one mainboard. There will be a total of 32 cores between the 2 CPUs (8 physical cores per CPU, up to 16 logical cores per CPU).

Before we spend the money, I want some confirmation that my R code will recognize and access all (or nearly all) the cores (usually, I put all but 1 into a cluster). I am using R doParallel and foreach packages successfully on a Win 10 Pro 64-bit workstation with a single 4-core CPU with 8 logical cores. I can run my R code and request 7 cores with no trouble.

rparallel-processing

Answers

answered 1 year ago Andrew Brēza #1

You should be able to use all of your cores in R. R successfully recognizes all available cores on both my laptop and my data science server.

Once you have access to one of the computers you want to use, you can find out how many cores R recognizes with the detectCores() function from the parallel package.

library(parallel)
detectCores()

Taking a step back, you might want to reconsider the architecture of your system. If you have to run something so intense that it requires a battalion of multithreaded machines, you might want to think about rewriting your code to be more efficient or possibly integrate Rcpp or move to a different language.

comments powered by Disqus