Questions tagged [doparallel]
R package that is a “parallel backend” for the foreach package. It provides a mechanism needed to execute foreach loops in parallel.
doparallel
478
questions
0
votes
1
answer
160
views
Why do pdf files that are printed using R's foreach() %dopar% construct turn out corrupted and unreadable?
I have a minimum reproducible example script below that writes identical plots to two pdf files, first serially, using a standard for loop, and then in parallel, using R's foreach() %dopar% construct:
...
- 4,553
0
votes
0
answers
73
views
R: Using nested foreach to fill a multidimensional array
I am stuck with parallelizing a nested for-loop, which populates a multidimensional array. I am trying to use foreach but have little experience/problems with combining the results/populating the ...
0
votes
1
answer
240
views
How do I summarise and bind large data sets in parallel using tidyverse in R?
I have a couple of large daily datasets that I need to summarise and bind in R by month. Because the datasets are so large, I'd like to do the summarising in parallel so that it is faster. I've been ...
- 1
1
vote
1
answer
353
views
r markdown generates duplicates if run in parallel
I am generating several reports via r markdown. If I do them one by one - everything is okay. If I use %do% - also okay. If I use %dopar% - 3 options:
Sometimes it's okay.
Sometimes reports have ...
- 281
1
vote
0
answers
55
views
How can i reduce the running time when compute variogram for large SaptialPointsDataframe in R
I am a beginner in R and geostatistics. I have a large SpatialPointsDataframe (307,907 elements, 27.2MB). When I want to use this data to build a variogram, it needs a lot of time (R runs for a day ...
- 11
0
votes
0
answers
75
views
foreach..dopar runs significantly slower on Docker vs local laptop
I am currently running into a weird issue that the foreach...dopar runs like 10x faster in my local laptop (Dell laptop, Windows 10 OS) with 15 cores than when the R code was put onto Docker container ...
- 728
0
votes
0
answers
77
views
R - How to parallelize a for loop that contains a for loop which itself contains a foreach %dopar% loop?
I'm carrying out several independent (random) experiments on an igraph graph. I'd like to run them in parallel.
An experiment has several iterations (a for loop), and each iteration consists of a ...
- 11
1
vote
1
answer
692
views
How R's parallel package would handle 12th Gen Efficiency Cores? [closed]
I was planning to do a desktop build with an i7-12700, I mostly need it for high CPU usage R tasks and most of the time parallel processing comes into play.
But I wonder how the new Efficiency Cores ...
- 899
1
vote
0
answers
207
views
How to display the progress bar for parallel processes using rstudioapi::jobRunScript?
I have a R script called parallel_progress_reprex.R that executes code in parallel and tracks the progress using progressr:
library(progressr) ## use progressr for procession updates
library(doFuture) ...
- 163
1
vote
1
answer
982
views
Error receiving results from R future running on RStudio server on Slurm
I've been struggling to find relevant answers that might help solve the problem, but after hundreds of googling I felt I was out of luck. I'm wondering if someone could point out where might the ...
- 1,303
0
votes
0
answers
38
views
Differences in the output data structure between using the R lapply and parSapply functions - how to avoid them?
I'm learning Parallel Computing in R. I have managed to go from the single-threaded approach to the multi-threaded approach and my function work both with lapply (single-threaded) and parSapply (multi-...
- 289
1
vote
0
answers
89
views
Nested parallelism using foreach and cv.glmnet
I'm trying to apply Elastic Net to a training dataset with approx. 29000 samples and 10000 features (approx 2.4gb) to create a binary logistic regression model. Using cv.glmnet, I originally wanted to ...
- 11
2
votes
0
answers
594
views
`doParallel` vs `future` while using `Seurat` package
Here is the story.
From Seurat vignette, FindMarkers() can be accelerated by utilizing future package, future::plan("multiprocess", workers = 4)
However, I am running a simulation that I ...
- 426
0
votes
1
answer
551
views
R: how to store the output of a foreach loop?
I am trying to store the output of a foreach loop in R. The code I use is as follows:
tweets_all = list()
foreach(i = 1:5, .packages = c("magrittr", "foreach", "...
- 41
1
vote
0
answers
191
views
R foreach: modify .combine to create equivalent of dplyr::bind_rows and include progress bar
Take the following MWE:
df <- data.frame(id =seq(1,100,1),random = seq(2,200,2))
cl <- parallel::makeForkCluster(5) # do not change
doParallel::registerDoParallel(cl) # do not change
results &...
- 634