I was wondering how one can construct a Borel set that doesn't have full measure on any interval of the real line but does have positive measure everywhere.
To be precise, if $\mu$ denotes Lebesgue measure, how would one construct a Borel set $A \subset \mathbb{R}$ such that $$0 < \mu(A \cap I) < \mu(I)$$ for every interval $I$ in $\mathbb{R}$?
Moreover, would such a set necessarily have to contain infinite measure?
If you got this from Rudin (it is Exercise 8, Ch. 2 in his Real & Complex Analysis), here is his personal answer (excerpted from Amer. Math Monthly, Vol. 90, No.1 (Jan 1983) pp. 41-42). He works with the unit interval $[0,1]$, but of course this can be extended to $\mathbb R$ by doing the same thing in each interval (and by scaling these replications appropriately you can get the final set with finite measure). Anyways, here's how it goes:
"Let $I=[0,1]$, and let CTDP mean compact totally disconnected subset of $I$, having positive measure. Let $\langle I_n\rangle$ be an enumeration of all segments in $I$ whose endpoints are rational.
Construct sequences $\langle A_n\rangle,\langle B_n\rangle$ of CTDP's as follows: Start with disjoint CTDP's $A_1$ and $B_1$ in $I_1$. Once $A_1,B_1,\dots,A_{n-1},B_{n-1}$ are chosen, their union $C_n$ is CTD, hence $I_n\setminus C_n$ contains a nonempty segment $J$ and $J$ contains a pair $A_n,B_n$ of disjoint CTDP's. Continue in this way, and put $$ A=\bigcup_{n=1}^{\infty}A_n. $$
If $V\subset I$ is open and nonempty, then $I_n\subset V$ for some $n$, hence $A_n\subset V$ and $B_n\subset V$. Thus $$ 0<m(A_n)\leq m(A\cap V)<m(A\cap V)+m(B_n)\leq m(V); $$ the last inequality holds because $A$ and $B_n$ are disjoint. Done.
The purpose of publishing this is to show that the highly computational construction of such a set in [another article] is much more complicated than necessary."
Edit: In his excellent comment below, @ccc managed to isolate the necessary components of my solution, and after incorporating his observation it has been greatly simplified. (Actually, after trimming the fat, I've realized that it is actually not entirely dissimilar from Rudin's.) Here it is:
Let $\{r_n\}$ be an enumeration of the rationals, let $V_1$ be a segment of finite length centered at $r_1$, and let $V_n$ be a segment of length $m(V_{n-1})/3$ centered at $r_n$. Set $$ W_n=V_n-\bigcup_{k=1}^{\infty}V_{n+k}, $$ and observe that \begin{equation} m(W_n)\geq m(V_n)-\sum_{k=1}^{\infty}m(V_{n+k})=m(V_n)-m(V_n)\sum_{k=1}^{\infty}3^{-k}=\frac{m(V_n)}{2}. \end{equation} In particular, $m(W_n)>0$.
For each $n$, choose a Borel set $A_n\subset W_n$ with $0<m(A_n)<m(W_n)$. Finally, put $A=\bigcup_{n=1}^{\infty}A_n$. Because $A_n\subset W_n$ and the $W_n$ are disjoint, $m(A\cap W_n)=m(A_n)$. That is to say, $$ 0<m(A\cap W_n)<m(W_n) $$ for every $n$. But every interval contains a $W_n$, so $A$ meets the criteria, and has finite measure (specifically, $m(A)\leq\sum_n m(V_n)=2 m(V_1)<\infty$).
As a curiosity, here's my own "unnecessarily computational" way (though it's not quite as lengthy as that in the article Rudin was referring to), which I can't resist including because I slaved over it when I first came across this problem, before finding Rudin's solution:
Let $\{r_n\}$ be an enumeration of the rationals, and put $$ V_n=\left(r_n-3^{-n-1},r_n+3^{-n-1}\right),\qquad W_n=V_n-\bigcup_{k=1}^{\infty}V_{n+k}. $$ Observe that \begin{equation} m(W_n)>m(V_n)-\sum_{k=1}^{\infty}m(V_{n+k})=m(V_n)-m(V_n)\sum_{k=1}^{\infty}3^{-k}=\frac{m(V_n)}{2}.\qquad\qquad(1)\label{8.1} \end{equation} (We have strict inequality because there exist rationals $r_i$, with $i>n$, in the complement of $V_n$.)
For each $n$, let $K_n$ be a Borel set in $V_n$ with measure $m(K_n)=m(V_n)/2$. Finally, put $$A_n=W_n\cap K_n,\qquad A=\bigcup_{n=1}^{\infty}A_n.$$
To prove that $A$ has the desired property, it is enough to verify that the inequalities $$0<m(A\cap V_n)<m(V_n)\qquad\qquad(3)\label{8.3}$$ hold for every $n$. (This is because every interval contains a $V_n$.) For the left inequality, it is enough to prove that $m(A_n\cap V_n)=m(A_n)=m(W_n\cap K_n)>0$. This follows from the relations $$m(W_n\cup K_n)\leq m(V_n)<m(W_n)+m(K_n)=m(W_n\cup K_n)+m(W_n\cap K_n),$$ the second inequality being a consequence of (1) and the fact that $m(K_n)=m(V_n)/2$.
For the right inequality of (3), observe that $V_n\subset W_i^c$ for $i<n$, and that therefore $$ m(A\cap V_n)=m\left(\bigcup_{k=0}^{\infty}A_{n+k}\cap V_n\right)\leq\sum_{k=0}^{\infty}m(K_{n+k}\cap V_n) $$ $$ <\sum_{k=0}^{\infty}m(K_{n+k})=\sum_{k=0}^{\infty}\frac{m(V_{n+k})}{2}=\sum_{k=0}^{\infty}\frac{m(V_n)}{2^{k+1}}=m(V_n). $$ The strict inequality above follows from three observations: (i) $m(K_i)>0$ for every $i$; (ii) $K_i\subset V_i$; and (iii) there exist neighborhoods $V_i$, with $i>n$, that are contained entirely in the complement of $V_n$.
So $A$ meets the criteria (and also has finite measure).
Nick S. has already posted a solution and mentioned a 1983 paper by Rudin, but I thought the following additional comments could be of interest. Essentially the same construction that Rudin gives can be found in Oxtoby's book "Measure and Category" (p. 37 of the 1980 2nd edition). Also, note that these constructions give $F_{\sigma}$ examples which, from a descriptive set theoretic point of view, are about as simple as you can get. (It's like asking for a countable ordinal bound on something and you're able to come up with 1 as the bound.)
If we relax the assumption from "Borel set" to "Lebesgue measurable set" then, surprisingly, MOST measurable sets in the sense of Baire category have this property. For simplicity, let's restrict ourselves to measurable subsets of the interval $[0,1]$. Let $M[0,1]$ be the collection of Lebesgue measurable subsets of $[0,1]$ modulo the equivalence relation $\sim$ defined by $E \sim F \Leftrightarrow \lambda(E \Delta F) = 0$, where $\lambda$ is Lebesgue measure. That is, two measurable sets are considered equivalent iff their symmetric difference has Lebesgue measure zero. Note that the property you're asking about (both the set and its complement have positive measure intersection with every open subinterval of $[0,1]$) is well defined on these equivalence classes, so we can safely work with representatives of these equivalence classes.
The set $M[0,1]$ can be made into a metric space by defining $d(E,F) = \lambda (E \Delta F)$. Under this metric $M[0,1]$ is a complete metric space. One way to show this is to note that $d(E,F) =\int_{0}^{1}\left| \chi_{E}-\chi _{F}\right| d\lambda$ and then make use of theorems involving Lebesgue integration (e.g. show that $M[0,1]$ is a closed subset of the complete metric space $\mathcal{L}^{1}[0,1]$). See, for instance, p. 137 in Adriaan C. Zaanen's 1967 book "Integration" (or pp. 80-81 of Zaanen's 1961 "An Introduction to the Theory of Integration"). For a direct proof that doesn't rely on Lebesgue integration, see p. 44 in Oxtoby's book "Measure and Category", pp. 87-88 in Malempati M. Rao's 2004 book "Measure Theory and Integration", or pp. 214-215 (Problem #13, which has an extensive hint) in Angus E. Taylor's 1965 book "General Theory of Functions and Integration".
In the complete metric space $M[0,1]$ (which also happens to be separable), the collection of elements (each of these elements is essentially a subset of $[0,1]$) that have the property you're looking for (both the set and its complement have positive measure intersection with every open subinterval of $[0,1]$) has a first Baire category complement. This is Exercise 10:1.12 (p. 411) in Bruckner/Bruckner/Thomson's 1997 text "Real Analysis" (freely available on the internet), Exercise 2 (pp. 78-79) in Kharazishvili's 2000 book "Strange Functions in Real Analysis", and it's buried within the proof of Theorem 1 (p. 886) of Kirk's paper "Sets which split families of measurable sets" [American Mathematical Monthly 79 (1972), 884-886].
(Added the next day) I thought I'd mention a few ways that such sets have been applied. Probably the most common application is to easily obtain an absolutely continuous function that is monotone in no interval. If $E$ is such a set in $[0,1]$ and $E' = [0,1] - E$, then $\int_{0}^{x}\left| \chi_{E}-\chi _{E'}\right| d\lambda$ is such a function. See, for example: [1] Charles Vernon Coffman, Abstract #5, American Math. Monthly 72 (1965), p. 941; [2] Andrew M. Bruckner, "Current trends in differentiation theory", Real Analysis Exchange 5 (1979-80), 90-60 (see pp. 12-13); [3] Wise/Hall's 1993 "Counterexamples in Probability and Real Analysis" (see Example 2.26 on p. 63). Another application is to construct a Lebesgue measurable function (Baire class 2, in fact) $g$ such that there is no Baire class 1 function $f$ that is almost everywhere equal to $g$. (Recall that every Lebesgue measurable function is almost everywhere equal to a Baire class 2 function.) For this ($g = \chi_{E}$ for any set $E$ as above will work), see: [4] Hahn/Rosenthal's 1948 "Set Functions" (see middle of p. 147); [5] Stromberg's 1981 "Introduction to Classical Real Analysis" (see Exercise 13c on p. 309). Finally, here are some miscellaneous other applications I found in some notes of mine last night: [6] Goffman, "A generalization of the Riemann integral", Proc. AMS 3 (1952), 543-547 (see about 2/3 down on p. 544); [7] MR 36 #5892 (review of a paper by A. Settari); [8] Gardiner/Pau, "Approximation on the boundary ...", Illinois J. Math. 47 (2003), 1115-1136 (see Lemma 3 on p. 1130).
This question has a cute answer -- if you know about Markov chains.
Consider the nearest-neighbour Markov chain $(X_n)$ on $\mathbf{Z}$ with the following transition probabilities: from $0$ it goes to $\pm 1$ with probability $1/2$, from $n\neq 0$ it moves one step towards $0$ with probability $1/4$ and one step towards $\infty$ (if $n>0$) or $-\infty$ (if $n<0$) with probability $3/4$.
It is simple to check (e.g. via the strong law of large numbers) that $|X_n|$ goes to $+\infty$ almost surely, so by symmetry $$ \mathbf{P}(\lim X_n = + \infty | X_0=0) = \mathbf{P}(\lim X_n = - \infty | X_0=0) = \frac{1}{2}. $$
Now define a Borel set $A \subset [0,1]$ as follows: use the digits in the binary expansion of $x \in [0,1]$ (which are i.i.d. Bernoulli variables which parameter $1/2$) to simulate the Markov chain $(X_n)$ starting from the origin, and let $A$ be set of $x$ for which $X_n \to + \infty$. Since the Markov chain is irreducible, $A$ has non-full non-zero measure in every subinterval of $[0,1]$.
For the OP last question: since $$ \lim_{k \to - \infty} \mathbf{P} \left(\lim X_n = +\infty | X_0= k\right) =0 ,$$ one gets a variant of the example with $m(A)$ arbitrary small by starting the Markov chain from $k$. Using such constructions in every interval $[n,n+1]$ produces a set $B \subset \mathbf{R}$ with finite measure whose intersection with every interval has positive measure.
Let $I = [0,1]$. We construct a partition of $I$ into $A_0$ and $A_1$ using binary digits of numbers in $x$. Recall that with respect to Lebesgue measure, the binary digits are like infinite coin toss experiment. Also recall that while a number can have more than one binary representations, such numbers form a measure zero set, so we can safely forget existence of such numbers.
Let $ c_1 = 1, c_2 = 2 < c_3 < c_4 < c_5 < \cdots $ be an increasing sequence of integers, such that $c_{n+1} - c_n$ grows fast enough. (How fast? To be determined later.) Let $J_n = \{k \in \mathbb N: c_n \le k < c_{n+1} \}$.
We say $x\in I$ is $n$-happy if k'th (binary) digit of x for each k in $J_n$ is 0. We say $x$ is $n$-angry if k'th digit of x for each k in $J_n$ is 1 . We say $x$ is $n$-emotional if it is $n$-happy or $n$-angry. If you imagine $x$ as an outcome of tossing a coin infinite times, then you can think of "1-emotional", "2-emotional", "3-emotional", .... as a sequence of events with rapidly decreasing probability.
Given an $x\in I$, the set $N(x)$ of all $n$ for which $x$ is $n$-emotional is a non-empty set because $1 \in N(x)$. If $c_{n+1} - c_n$ grows fast enough (linear growth is enough), then $N(x)$ is a finite set for almost all $x$, due to Borel-Cantelli lemma. Let $n(x) = \sup N(x)$. Now the function $x \mapsto n(x) $ is finite a.e. This allows us to partition $I$ into $A_0$ consisting of all $x$ that are $n(x)$-happy and $A_1$ consisting of all $x$ that are $n(x)$-angry.
To show that $A_0$ intersects every subinterval of $I$ in positive density, let $I'$ be an arbitrary subinterval of $I$. Then there exist $n$ and digits $x_1, \cdots x_{c_n-1}$ such that $I'$ contains the interval $I''$ consisting of all numbers in $I$ whose first $c_n -1$ digits are precisely $x_1, \cdots x_{c_n-1}$. The $n$-lucky numbers in $I''$ form yet another interval, say $I'''$. Given a random choice of $x \in I'''$, it's more likely for $x$ to be in $A_0$ than to be in $A_1$ because of $n$-luckiness of $x$. Therefore $A_0$ intersects $I'''$ in more than half density, and thus intersects $I'$ in positive density.
Remark 1. There is no measurable subset of $I$ which intersects every subinterval in exactly half density. Terence Tao uses this to construct a non-measurable set in here.
Remark 2. Lebesgue density theorem
There are several variations to this. Here is the outline to one approach that yields a set of first category $A$, that has finite measure, and which satisfies the condition \begin{align} 0<m(I\cap A)< m(I) \end{align} for any nontrivial interval in $\mathbb{R}$. This I found in George, C., Exercices et problèmes d'intégration, Bordas, Paris. 1980. Chapter 1.
Notice that:
Suppose that a compact nowhere dense set $K_n\subset K_0$, $n\geq1$, with $m(K_n)=\sum^n_{j=1}2^{-j}=1-2^{-n}$ has been constructed. Repeating the construction in step 2 with $K_n$ in place of $K_1$, we obtain a set $E_{n+1}\subset K_0\setminus K_n$ which is the countable union of disjoint compact nowhere dense sets, each of which is contained in an open connected component of $K_0\setminus K_n$, and such that $m(E_{n+1})=2^{-1}\big(1-(1-2^{-n})\big)=2^{-n-1}$. Set $K_{n+1}=K_n\cup E_{n+1}$. As before, $K_{n+1}$ is closed and nowhere dense.
The set $E=\bigcup_nE_n=\bigcup_nK_n$ is a set set of first category (countable union of nowhere dense sets) and $m(E)= 1$. For each $m\in\mathbb{Z}_+=\{0\}\cup\mathbb{N}$ define $$F_m=\bigcup_{n\geq m}E_{2n+1},\qquad G_m=E\setminus F_m$$ Each $F_m$ has the desired property in $K_0=[0,1]$. To see this, notice that since $E=F_m\cup G_m$, for each interval $I\subset K_0$ of positive length, either $m(I\cap F_m)>0$ or $m(I\cap G_m)>0$. Suppose $m(I\cap F_m)>0$. Then there are points $x,y\in I\cap F_m$ with $x<y$. Choose $N\in\mathbb{N}$ such that $[x,y]\subset \bigcup^M_{j=m}E_{2j+1}\subset K_{2M+1}$. Since $K_{2M+1}$ is nowhere dense, $[x,y]$ contains one of the connected components $J$ (which is an open interval) of $K_0\setminus K_{2M+1}$. It follows that $$m(I\cap G_m)\geq m(J\cap G_m)\geq m(J\cap E_{2M+2})>0$$ A similar argument shows that if $m(I\cap G_m)>0$, then $m(I\cap F_m)>0$. As a consequence $$0<m(I\cap F_m)<m(I\cap F_m)+m(I\cap G_m)=m(I)$$
Let $A=\bigcup_{n\in\mathbb{Z}}F_{|n|}+n$, where $B+x=\{b+x: b\in B\}$). The set $A$ is of first category -being the countable union of sets of first category- and has the property that for any interval $I$ in $\mathbb{R}$ that has positive length, $$0<m(I\cap A)<m(I)$$
Also, $m(A)<\infty$. To see this, notice that \begin{align} m(F_m)=\sum^\infty_{n=m}m(E_{2n+1})=\sum^\infty_{n=m}2^{-(2n+1)}=\frac12\sum^\infty_{n=m}4^{-n}=\frac23 4^{-m} \end{align} Hence \begin{align} m(A)=\frac23+\frac43\sum^\infty_{n=1}4^{-n}=\frac23+\frac13\frac{1}{1-(1/4)}=\frac23+\frac49=\frac{10}{9}<\infty \end{align}
First work in a bounded interval $I$, such as $[0, 1]$, then by taking the union of the translates we get the desired set.
Let $K \subset I$ be a compact set of positive measure such that ${m(K \cap I) < |I|}$ for every interval ${I}$ of positive length, as shown here: Example of positive measure compact set that contains no interval. We can further ensure that $m(K) > 1/2|I|$ by taking an open dense subset $D \subset I$ to be sufficiently small, and let $K := I \setminus D$. Denote $K$ by $E_0$. The complement $I \setminus E_0$ is open, and thus can be written as an at most countable union of disjoint non-empty open intervals $H_1 = I \setminus E_0 = \bigcup_{i} V_{i,1}$, meaning the "holes" in the first iteration. Let $K_{i,1} \subset V_{i,1}$ be scaled copies of $K$, that "fills" the holes, and define $E_1 := \bigcup_{i} K_{i,1}$. For each $i$, the complement $V_{i,1} \setminus K_{i,1}$ is open, and thus a countable union of disjoint open intervals, fill the holes $H_2 = I \setminus E_0 \setminus E_1 = \bigcup_{i} V_{i,2}$ so produced again by $K_{i,2} \subset V_{i,2}$, and define $E_2 := \bigcup_i K_{i,2}$, etc.
Finally, let $E = \bigcup_{n=1}^\infty E_n$. Note that by our construction, $0 < m(E_n) < \frac{1}{2^n}|I|$ for each $n > 0$, so $m(E) = \sum_{n=1}^\infty m(E_n) <\sum_{n=1}^\infty \frac{1}{2^n}|I| = |I|$. For any subinterval $I' \subset I$, $m(I'\setminus K) > 0$. Since $I' \setminus K = I' \cap D$ is dense in $I'$, we have either $I' \supset V_{i,1}$ or $I' \subset V_{i,1}$ for some $i$. In both cases, there exists an $N > 0$ such that $V_{i,N} \subset I'$ for some $N > 0$. So $0 < m(E \cap I') \leq m(I' \setminus V_{i,N}) + m(E \cap V_{i,N}) <m(I' \setminus V_{i,N}) + |V_{i,N}| = |I'|$. As we fill $V_{i,N}$ iteratively just like we fill $I$ by $E$.
This approach is like dropping pebbles in an empty jar, first we fill the jar with large pebbles, then with smaller ones, then yet smaller ones and so on. By the end of this process, any portion of the jar will contain some pebbles and some holes not filled by any pebbles.
Well, suppose you enumerated the rational intervals $I_n$, and for each $I_n$, let $J_n = I_n \setminus \cup_{i=1}^{n-1}I_i$. If $J_n$ has measure 0, let $A_n = \varnothing$. Otherwise, let $A_n$ be a subset of $I_n$ such that $\mu(A_n) < \max\{\mu(I_n)/2, \epsilon\cdot 2^{-n}\}$. Then let $A$ be the union of the sets $A_n$. Doesn't that do it?
