Filtering out empty Optionals

Question

I want to convert my list of optional strings to a list of strings by getting rid of the empty Optionals.

Is there a shorter version for achieving this than the following code (except statically importing the methods of Collectors)?

List<Optional<String>> stringsMaybe = Arrays.asList(Optional.of("Hi"),
                                      Optional.empty(), Optional.of(" there!"));

List<String> strings = stringsMaybe
            .stream()
            .filter(Optional::isPresent)
            .collect(Collectors.mapping(Optional::get, Collectors.toList()));

Answer 1

It's more idiomatic to use .map on the stream instead of Collectors.mapping:

stringsMaybe.stream()
    .filter(Optional::isPresent)
    .map(Optional::get)
    .collect(toList());

Without introducing a helper method or a custom collector, that's the shortest and clearest way to do this.

Since Java 9, Optional offers a stream method, enabling you to do .flatMap(Optional::stream) instead of .filter(...).map(...).

Answer 2

Avoid using Optional.isPresent

Every time you find yourself writing a check for value presents via Optional.isPresent() or Optional.isEmpty() pause for a second because there's probably a more expressive and concise way to achieve what you're trying to do.

^{Forget that these methods exist in the Optional API for good, your code would be cleaner without them}

flatMap(Optional::stream) is succinct and clear. Hence, better than going through a step process of checking for value presence and unpacking the optional.

^{One might not consider flatMap(Optional::stream) to be very clear, but it's only a matter of familiarity and practice. After a certain experience, it'll not bear much cognitive load.}

But yet, utilizing flatMap() has a performance overhead that you need to know about.

Stream.flatMap() has a Cost

This operation performs a one-to-many transformation by generating a new Stream out of every element in the initial stream.

Note, that every optional in the stream regardless of the value presence will spawn a new nested stream. Because Stream is a complex stateful object which can not be reused, once a stream is consumed its method close() is being invoked, changing the stream state. For that reason, Stream.empty() always returns a new stream instance.

^{If you're curious, I encourage you to take a look at the source code and see for yourself that Stream.empty() (internally used by Optional.stream()) delegates the call to StreamSupport.stream() passing Spliterators.<T>emptySpliterator() as an argument, and you're getting a new Stream instance wrapping an empty spliterator as its source.}

I.e. if there are, let's say 90,000 optionals in the initial stream, then during stream execution flatMap will create and consume 90,000 nested streams which are either empty or contain a single element.

Java 16 Stream.mapMulti()

This method was specifically introduced for such scenarios:

API Note:

This method is similar to flatMap in that it applies a one-to-many transformation to the elements of the stream and flattens the result elements into a new stream. This method is preferable to flatMap in the following circumstances:

• When replacing each stream element with a small (possibly zero) number of elements. Using this method avoids the overhead of creating a new Stream instance for every group of result elements, as required by flatMap.
• When it is easier to use an imperative approach for generating result elements than it is to return them in the form of a Stream.

Both mapMulti and flatMap have the same purpose, but their internal mechanics are different.

Method mapMulti() has a parameter of type BiConsumer. This BiConsumer in turn expects two arguments: an element of the initial stream, and a Consumer of the resulting type (basically it represents the downstream operation, every new element offered to this consumer becomes a part of the resulting stream).

Here's how empty optionals can be dealt with using mapMulti() with no overhead of creating nested streams:

List<String> strings = stringsMaybe.stream()
    .<String>mapMulti(Optional::ifPresent)
    .toList();

^{So-called type-witness <String> in front of mapMulti is required because the Java compiler is unable to infer properly the type of the resulting flattened stream.}

The method reference Optional::ifPresent in this case is an equivalent of the lambda expression:

(optional, downstream) -> optional.ifPresent(downstream)

mapMulti vs flatMap

_{A word of caution against cargo cult programming}

mapMulti is a perfect match for this particular problem because this operation was designed for cases when each stream element is being replaced with a small number of elements in the resulting stream (see the quote above). But it doesn't mean that every occurrence of flatMap should be replaced with mapMulti.

Choice between these operations should be motivated by the firm understanding of their pros and cons and specifics of the problem at hand.

flatMap incorporates elements from each nested stream produced by its mapper function in a lazy way, only if needed. So, if a mapper returns a stream with a large number of elements, or even an infinite stream, it's not an issue (if there are short-circuiting operations such as limit or findFirst applied downstream to ensure that the stream will complete).

On the other hand, mapMulti executes all logic of its BiConsumer (it's impossible to do otherwise), and feeds each replacement element to the next operation. Currently (the latest LTS version at the time of writing is Java 21), this operation is implemented in such a way that all the replacement elements will be processed by the operation following the mapMulti potentially ignoring short-circuiting operations applied further in the stream.

Unfortunately, documentation does not emphasize this aspect of behavior well enough, only telling to utilize mapMulti when replacement elements are few, possibly zero. And with zero or one element, like with Optional, no abnormalities will be observed.

Here's an example where mapMulti doesn't exhibit fully lazy behavior:

var numbers = List.of(1, 2, 3, 4, 5, 6, 7, 8, 9, 10);
        
Stream.of("A", "B", "C")
    .peek(s -> System.out.println("Processing element " + s))
    .<Integer>mapMulti((s, d) -> numbers.forEach(d))
    .map(i -> i * i)
    .peek(System.out::println)
    .findFirst()
    .ifPresent(i -> System.out.println("Only this element was needed: " + i));

Output:

Processing element A
1
4
9
16
25
36
49
64
81
100
Only this element was needed: 1

As can be seen, only the first element A from the stream source gets processed (everything is lazy so far). However, when mapMulti comes into play, laziness gets lost. It produces 10 elements out of the initial element A and subsequent operations map and peek are executed for each of these ten elements. Ultimately, findFirst discards 9 of them, performing operations on these extra elements was not needed.

But if we replace mapMulti with flatMap, then map and peek are executed only for one element and stream terminates without performing unnecessary computations (see this demo).

Historical note: before Java 10 flatMap operation was also not fully lazy.

To conclude this topic, these are the potential drawbacks of using mapMulti operation that you need to be aware of:

• due to type inference limitations, Java compiler often fails to infer the type of stream it returns, so you need to resort either to using a type witness or a lambda with explicitly specified type arguments which make code more noisy;

• in some cases, it tends to gravitate towards imperative coding style;

• behavior of this operation is not fully lazy when it produces several elements, as was demonstrated above.

Don't store Optionals in a Collection

Maybe you created List<Optional<String>> only to illustrate the example, but the point remains.

It's akin to a null-infested collection because all empty optionals are as meaningless as null-elements, they are only consuming space.

Remainder: it's not a good practice to ascribe a special meaning to null values in your business logic. The same holds true for an empty optional. When you need to perform a certain action on an empty optional (like retrying or throwing an exception), consider doing it right on the spot where the optional is obtained instead of propagating it.

Here's a quote from the answer by Stuart Marks, Java and OpenJDK developer:

I'm sure somebody could come up with some contrived cases where they really want to store an Optional in a field or a collection, but in general, it is best to avoid doing this.

A Collection containing Suppliers of optional which can be evaluated on demand in a lazy fashion would more practical.

Don't move operations that can be done in the Stream into a Collector

Collector mapping() intended to be used as a downstream collector (of another collector). Not as a substitution of the stream operation map().

Typically, a chain of collectors starts with a collector representing such a type of accumulation operation, which is not present in the Stream. For instance, groupingBy(), partitioningBy(), teeing(), etc. And then as the second collector in the chain, you might make use of mapping(), filtering(), flatMapping(), maxBy(), reducing(), counting(), etc. which have direct analogs in the Stream.

If you're doing otherwise, promoting a stream operation into a collector, you're making the code more difficult to comprehend.

Expressiveness is the most powerful weapon of the Functional programming, don't relinquish it.

Or to put it simpler, try to keep your collectors short.

Answer 3

The question starts with a wrong premise. You should not ever have a list of optionals. The problematic code precedes the one you provided and should be fixed instead.

Please read this explanation from Brian Goetz, who works as a Java language architect at Oracle: "we did have a clear intention when adding this feature, and it was not to be a general purpose Maybe type, as much as many people would have liked us to do so."

Answer 4

If you are using rxJava you can do something like:

Flux<Optional<String>> maybeStrings = Flux.just(
    Optional.of("Hi"),
    Optional.empty(),
    Optional.of(" there!"));

Flux<String> strings = maybeStrings.handle((maybeString, synchronousSink) -> 
    maybeString.ifPresent(synchronousSink::next));