I have the impression that the underlying problem is the expected value itself, not the integral (on which the expected value is based, of course). But since the question asks about the integral, I really don't see why people consider the Lebesgue integral inherently more difficult than the Riemann integral.
For all the examples below, consider $\mu$ to be a measure on (some $\sigma$-algebra of) $X$.
The opposition vertical vs horizontal is false
It's not like we cannot introduce the Lebesgue integral by vertical slices (see Wiki). The basic definition I was given in my second year analysis course was that the integral of a simple function $f = \sum_{i=1}^n a_i 1_{A_i}$ is given by $\int_X f d\mu = \sum_{i=1}^n a_i \mu(A_i)$. One can define integrals for larger classes of functions by using a limiting process.
Note that the integral $\int_X f d\mu$ above is nothing else than looking at vertical slices $A_i \times [0,a_i]$ and summing up their measures (OK, one would have to introduce the product measure $\mu \otimes \lambda^1$ on $X \times \mathbb{R}$, but that's not the point). The only difference between this and the Riemann integral is that the latter requires $A_i$ to be intervals in $\mathbb{R}$. Quite arbitrary, right? This takes us to the next point.
Riemann integration is built upon additional (and a posteriori, irrelevant) structure
Riemann integration requires us to look at intervals in $\mathbb{R}$, while Lebesgue integration only sees the $\sigma$-algebra of measurable sets, disregarding any (possible) existing metric or topological structure on $X$ (although for most applications we do care about measuring Borel sets, and about some metric compatibility, too). Mathematicians around 1900 needed some time to arrive at this generalization and simplification, so why not embrace their wisdom?
One application that a non-math major student could appreciate concerns summing up infinite series. Did you notice how confusing it is to study conditionally convergent series, the Riemann rearrangement theorem in particular? And how absolutely convergent series are better behaved, for some mysterious reasons?
In a way, it's an artifact of how we introduce infinite summation:
\begin{equation}
\label{eq:standard}
\sum_{n=1}^\infty a_n = S
\quad \Longleftrightarrow \quad
\forall_{\varepsilon > 0} \, \exists_{N > 0} \, \forall_{n>N} \, |a_1+\ldots+a_n - S| < \varepsilon.
\tag{$\star$}
\end{equation}
The order of the sequence $(a_n)$ is encoded into the definition - a fact which most students (I, for one) miss at first encounter. Hence all the problems with rearrangements.
The alternative Lebesgue summation approach is as follows. If $a_n \ge 0$, then let
$$
\sum a := \sup \left\{ \sum_{j \in J} a_{j} : J \subseteq \mathbb{N} \text{ is finite} \right\}.
$$
This supremum may well be infinite, but it always exists. And of course, the result does not depend on the order of $(a_n)$ in any way. One can check that $\sum a$ coincides with the sum in \eqref{eq:standard}.
For a general sequence $(a_n)$, consider its positive and negative part:
$$
a^+_n := \max(a_n,0),
\qquad
a^-_n := \max(-a_n,0).
$$
This way, $a_n = a^+_n - a^-_n$. If both $(a^+_n)$ and $(a^-_n)$ have finite sums, we put
$$
\sum a := \sum a^+ - \sum a^-.
$$
Again, the order doesn't matter. Note that the assumption $\sum a^+, \sum a^- < \infty$ is equivalent to $\sum |a| < \infty$, which is the same as saying that $\sum_{n=1}^\infty |a_n|$ is absolutely convergent.
The best thing is, the definition above is not an ad hoc construction. It's simply the result of applying Lebesgue integration theory to $\mathbb{N}$ with the counting measure (no measurability nuisance involved!).
The Lebesgue integral can be built on top of the Riemann integral
There's a number of reasons why horizontal is in fact better that vertical. BCLC's answer links to the money analogy Lebesgue himself made when explaining his integration theory. Jon Bannon's answer highlights the role of superlevel sets $S_{\lambda} := \{ x \in X : f(x) > \lambda \}$. Let me take it a bit further.
The horizontal slice between two values $\lambda < \mu$, i.e. the set $\{ (x,t) : \lambda < t \le \min(\mu, f(x)) \}$, has measure between $(\mu-\lambda) \mu(S_\lambda)$ and $(\mu-\lambda) \mu(S_\mu)$. If you sum up the contribution of all slices, and take the limit (as size of slices tends to zero), the integral turns out to be
\begin{equation}
\label{eq:horizontal}
\int_X f d \mu = \int_0^\infty \mu(S_\lambda) d\lambda,
\tag{$\star\star$}
\end{equation}
at least for $f \ge 0$. The cheapest way to have the Lebesgue integral is by taking \eqref{eq:horizontal} as the definition, where the left side is Lebesgue and the right side is Riemann. The necessary ingredients are:
- determining $\mu(S_\lambda)$ (you need to have a measure $\mu$ to do this);
- making sure that $\lambda \mapsto \mu(S_\lambda)$ is a Riemann-integrable function (it is, because it's monotone);
- defining the Riemann integral $\int_0^\infty \mu(S_\lambda) d\lambda$.
The are reasons why more powerful tools are better than less powerful tools
For historical and pedagogical reasons, it's nice to start with something less powerful, but conceptually simpler. And many times it's perfectly OK to stop there. But students can benefit from knowing that the integration theory doesn't stop on Riemann, and for a good reason.
As already discussed in the other answers, Lebesgue's theory is useful for Fourier series and Fourier transform in general (due to limit theorems, I guess), calculus of variations (for the same reason, basically), probability theory (thanks to the general setting of measure spaces) and geometric applications (while one can extend Riemann integration to manifolds somehow, it's better to think about abstract measures).
All right, most students won't need it personally, but at least they should know that the world has made progress since the XIX century. And if they're OK with using smartphones or WolframAlpha as calculators - just because there's a powerful and easy-to-use tool at hand - they might be OK with Lebesgue's integral as well.
