All Questions
Tagged with doparallel hpc
7
questions
3
votes
1
answer
122
views
Utilizing multiple nodes for parallel computing in R
I have access to a HPC system. Let's say I have three nodes/system available. Details of each node is as follows:
scontrol show node
Arch=x86_64 CoresPerSocket=10
CPUAlloc=20 CPUTot=20 CPULoad=...
- 444
3
votes
0
answers
100
views
How to call system() within %dopar% iterations in R
How should I call external programs from sub-instances of parallelized R? The problem could occur also on other contexts, but I am using library(foreach) and library(doFuture) on slurm-based HPC. As ...
- 79
2
votes
0
answers
594
views
`doParallel` vs `future` while using `Seurat` package
Here is the story.
From Seurat vignette, FindMarkers() can be accelerated by utilizing future package, future::plan("multiprocess", workers = 4)
However, I am running a simulation that I ...
- 426
1
vote
1
answer
895
views
Do R sessions always use one CPU core unless specifically instructed otherwise?
On occasions where heavy compute is required, I've used doParallel package to dispatch work over multiple cores. Random example:
if (detectCores()-1 > 1) {
cl <- makeCluster(detectCores()-...
- 48.5k
1
vote
0
answers
258
views
mclapply in R programming- How can I make sure all my cores and memory is utilized
I am working on doing some computation using parallel::mclapply to parallelize the process. I have a High performance computing server (HPC) with 64GB memory and 28 cores CPU. Speed of the code run ...
- 427
0
votes
2
answers
3k
views
Issue with doParallel and foreach. I can register cores, but they don't seem to run
I have a large data set (about 40 million rows x 4 columns) and I want to perform a Fisher test on the data in each row.
An example of the data looks like this:
refAppleBase altAppleBase ...
- 1
3
votes
1
answer
1k
views
Detect Cores from Multiple Nodes
I have a script in R that takes advantage of the doParallel package and the parallelized foreach function. I am currently registering my cluster by using a variation of the detectCores() command, ...
- 61