Scala Future or other way to execute Java void methods in parallel

Question

I have the following piece of code:

stringList map copyURLToFile // List[String]

def copyURLToFile(symbol: String) = {
  val url = new URL(base.replace("XXX", symbol))
  val file = new File(prefix + symbol + ".csv")
  org.apache.commons.io.FileUtils.copyURLToFile(url, file)
}

How do I make copyURLToFile execute in parallel?

Scala Futures need a return type, but if I make copyURLToFile of type Future[Unit], what and how do I return from the function?

Answer 1

If you want a very easy way to parallelize this, then just map over the list using the parallel collections library.

stringList.par.map(copyURLToFile).toList

Answer 2

You need to return something, how are you going to know which files where downloaded?

Change your code to be:

def copyURLToFile(symbol: String): (String,String) = {
  val url = new URL(base.replace("XXX", symbol))
  val file = new File(prefix + symbol + ".csv")
  org.apache.commons.io.FileUtils.copyURLToFile(url, file)
  return (symbol, file)
}

And then the resulting collection will contain the symbols an the paths to the file where they were downloaded.