Questions tagged [doparallel]
R package that is a “parallel backend” for the foreach package. It provides a mechanism needed to execute foreach loops in parallel.
doparallel
478
questions
87
votes
1
answer
1k
views
Inconsistent behaviour with tm_map transformation functions when using multiple cores
Another potential title for this post could be "When parallel processing in R, does the ratio between the number of cores, loop chunk size, and object size matter?"
I have a corpus I am ...
- 20.8k
52
votes
1
answer
18k
views
the difference between doMC and doParallel in R
What's the difference between doParallel and doMC in R concerning foreach function? doParallel supports windows, unix-like, while doMC supports unix-like only. In other words, why doParallel cannot ...
- 2,349
21
votes
2
answers
4k
views
Why does foreach %dopar% get slower with each additional node?
I wrote a simple matrix multiplication to test out multithreading/parallelization capabilities of my network and I noticed that the computation was much slower than expected.
The Test is simple : ...
19
votes
3
answers
8k
views
doParallel, cluster vs cores
What is the difference between cluster and cores in registerDoParallel when using doParallel package?
Is my understanding correct that on single machine these are interchangeable and I will get same ...
- 22.3k
18
votes
2
answers
12k
views
doParallel "foreach" inconsistently inherits objects from parent environment: "Error in { : task 1 failed - "could not find function..."
I have a problem with foreach that I just can't figure out. The following code fails on two Windows computers I've tried, but succeeds on three Linux computers, all running the same versions of R and ...
- 700
15
votes
3
answers
36k
views
How can I speed up the training of my random forest?
I'm trying to train several random forests (for regression) to have them compete and see which feature selection and which parameters give the best model.
However the trainings seem to take an insane ...
- 4,239
12
votes
2
answers
15k
views
How to export multiple function or packages in foreach loop in "R"
I'm trying to decrease run time of my code by using doParallel package in R.
I'm calling a function awareRateSIR that some extra packages are used in the body of this function. I get some error like
...
- 209
11
votes
3
answers
4k
views
how to use %dopar% when only import foreach in DESCRIPTION of a package
How to avoid "could not find function "%dopar%"" in a function of a package when only imports (not depends) foreach in DESCRIPTION of a package? is there a way like foreach::%dopar% as I use foreach::...
- 2,349
9
votes
1
answer
2k
views
Run RSelenium in parallel
How would i go about running RSelenium in parallel.
The following is an example using rvest in parallel
library(RSelenium)
library(rvest)
library(magrittr)
library(foreach)
library(doParallel)
...
- 5,911
8
votes
1
answer
7k
views
R: show error and warning messages in foreach %dopar%
I'm new to using foreach() %dopar% for paralleling, and I have some problems about how it handles errors or warnings.
when I use try() with my customized error message within foreach() %dopar%, the "...
- 103
8
votes
4
answers
2k
views
Running foreach without returning any value in R
I have a function doSomething() which runs in a foreach loop and as a result saves some calculations as .csv files. Hence I have no need for a return value of foreach, in fact I don't want a return ...
- 325
8
votes
1
answer
2k
views
Parallelizing keras models in R using doParallel
I'm trying to ensemble several neural networks using keras for R. In order to do so, I would like to parallelize the training of the different networks by using a "foreach" loop.
models <- list()
...
8
votes
2
answers
2k
views
loop inside a foreach loop using doparallel
I have a function that contains a loop
myfun = function(z1.d, r, rs){
x = z1.d[,r]
or.d = order(as.vector(x), decreasing=TRUE)[rs]
zz1.d = as.vector(x)
r.l = zz1.d[or.d]
y=vector()
for (...
- 1,772
7
votes
1
answer
7k
views
Do I have to registerDoParallel() and stopCluster() every time I want to use foreach() in R?
I read you had to use stopCluster() after running parallel function: foreach() in R. However, I can get away with registerDoParallel() and then running foreach() as many times as I want without ever ...
- 113
7
votes
2
answers
501
views
Caret doparallel in Rmarkdown: Missing verbose info when using render()
I have the following simple example Rmarkdown document (test.Rmd):
---
title: "Test Knit Caret Paralell VerboseIter"
output: html_document
---
```{r setup, include=FALSE}
knitr::opts_chunk$set(echo =...
- 6,010