Merge collection operations

Question

Look at the code below:

val a = List(1,2,3,4,5)
a.filter(_ >= 3).map(_ * 9).drop(1).take(2)

My understanding is that it this iterates through the list once for each operation, generating code similar to this:

for(i <- a) {
    // filter
}
for (i <- afiltered) {
   // map
}
for (i <- afilteredandmapped) {
   // drop
}
..etc

Is there anyway to combine these operations, having chaining operations that only iterates once through the list? Generating such as below:

for (i <- a) {
    // filter
    // map
    // drop
    // take
}

Answer 1

Scala 2.8 collections introduced the notion of a Views, which are a lazy counterpart to the strict collections. It allows you to avoid the intermediate allocation of collections when applying multiple transformations on them.

You use them via calling .view on the collection, and when you want to materialize them you can do it via .force or toList/.toSeq:

val result = a
              .view
              .filter(_ >= 3)
              .map(_ * 9)
              .drop(1)
              .take(2)
              .force

Answer 2

Scala collections are eagerly evaluated. The simplest and most reliable way to prevent creation of intermediate collections is to apply the transformations to an iterator instead of to the collection itself:

val a = List(1,2,3,4,5)
a.filter(_ >= 3).map(_ * 9).drop(1).take(2)

Creates 3 intermediate lists in addition to the result

val a = List(1,2,3,4,5)
a.iterator.filter(_ >= 3).map(_ * 9).drop(1).take(2).toList

Creates only the result list.

This is very similar to the solution by Yuval Itzachow. But views are a bit more complex since they attempt to preserve the type of the collection (there is a corresponding view for each collection), so the iterator approach is preferable if you don't mind stating what you want the result to be toList vs. force.

Answer 3

You can combine the filter and map in a collect

val a = List(1,2,3,4,5)
a.collect {case x if x >= 3 => x * 9}.drop(1).take(2)

if you want the drop and take to be lazy, you'll need views, iterators or streams. A stream version, since other answers cover iterators and views;

val xs = a.toStream
xs.filter{x=>println("filtering " + x); x >= 3}
  .map{x=>println("mapping " + x); x * 9}
  .drop(1)
  .take(2)
  .toList
//> filtering 1
//| filtering 2
//| filtering 3
//| mapping 3
//| filtering 4
//| mapping 4
//| filtering 5
//| mapping5
//| res1: List[Int] = List(36, 45)

Answer 4

For Comprehension are made for these cases where the operations are combined on your behalf and an efficient code is generated for you.

(for (x <- a if x >= 3 ) yield (x * 9)).drop(1).take(2)

In fact, for comprehension is translated to the combination of withFilter and map/flatmap.

The withFilter was specifically designed when working with chains of methods to enable lazy evaluation. A filter takes a collection and produce a new collection on the spot. However, withFilter does nothing and passes the data unfiltered through to the next operation. Hence if the next operation is a map the result is realized and the whole filter + map combination is evaluated together.