I noticed on Math Stack Exchange a terrific thread which highlighted a number of very visually interesting math concepts. I would be curious to see graphics/gifs which anyone has that very clearly illustrate a statistics concept (particularly those that might serve as motivation for students just starting to learn statistics).
I am thinking of things along the lines of how videos of a Galton board make the CLT instantly relatable.
I like images illustrating how different patterns can have similar correlation. The ones below are from Wikipedia articles on correlation and dependence
and Anscombe's quartet with correlations of about $0.816$
A phenomenon that appears when a key variable is omitted from the analysis of a relationship between one or more independent variables and a dependent variable. For instance, this shows the more bedrooms houses have, the lower the home price:
(source: ba762researchmethods at sites.google.com)
which seems counter-intuitive, and is easily resolved by plotting all the data points that make up the average for each area, on the same graph. Here, the greater number of bedrooms correctly indicate pricier homes when also observing the neighborhood variable:
(source: ba762researchmethods at sites.google.com)
If you'd like to read more about the above example and get a far better explanation than I was able to provide, click here.
One of the most interesting concepts that are very important today and very easy to visualize is "overfitting". The green classifier below presents a clear example of overfitting [Edit: "the green classifier is given by the very wiggly line separating red and blue data points" - Nick Cox].
From Wikipedia:
How does a 2D dataset where the mean of X is 54 with a SD 17, and for Y 48 and 27, respectively, and the correlation between the two is -0.06?
Introducing the Anscombosaurus:
And its companion, the Datasaurus Dozen:
I think spurious correlations also deserve their own post. I.e. correlation does not equal causation. Perhaps one of the things used most often when trying to bend the truth using statistics. Tyler Vigen has a famous website with lots of examples. To illustrate - see the plot below where the number of polio cases and the ice cream sales are clearly correlated. But to assume that polio causes ice cream sales or the other way around is clearly nonsensical.
P.S: Relevant xkcd 1 and relevant xkcd 2
An $\color{orangered}{\text{unbiased estimator}}$ is on average correct. A $\color{steelblue}{\text{biased estimator}}$ is on average not correct.
Why then, would you ever want to use a biased estimator (e.g. ridge regression)?
The answer is that introducing bias can reduce variance.
In the picture, for a given sample, the $\color{orangered}{\text{unbiased estimator}}$, has a $68\%$ chance to be within $1$ arbitrary unit of the true parameter, while the $\color{steelblue}{\text{biased estimator}}$ has a much larger $84\%$ chance.
If the bias you have introduced reduces the variance of the estimator sufficiently, your one sample has a better chance of yielding an estimate close to the population parameter.
"On average correct" sounds great, but does not give any guarantees of how far individual estimates can deviate from the population parameter. If you would draw many samples, the $\color{steelblue}{\text{biased estimator}}$ would on average be wrong by $0.5$ arbitrary units. However, we rarely have many samples from the same population to observe this 'average estimate', so we would rather have a good chance of being close to the true parameter.
When first understanding estimators and their error, it's useful to understand two sources of error: bias and variance. The below image does a great job illustrating this while highlighting tradeoffs between these two sources of error.
The bullseye is the true value the estimator is trying to estimate and each dot represents and estimate of that value. Ideally you have low bias and low variance, but the other dart boards represent less than ideal estimators.
Principal component Analysis (PCA) PCA is a method for dimension reduction. It projects the original variables in the direction that maximizes the variance.
In our figure, the red points come from a bivariate normal distribution. The vectors are the eigenvectors and the size of these vectors are proportional to the values of the respective eigenvalues. Principal component analysis provides new directions that are orthogonal and point to the directions of high variance.
The concept of eigenvectors and eigenvalues which are the basis for principal component analysis (PCA), as explained on wikipedia:
In essence, an eigenvector $v$ of a linear transformation $T$ is a nonzero vector that, when $T$ is applied to it, does not change direction. Applying $T$ to the eigenvector only scales the eigenvector by the scalar value $\lambda$, called an eigenvalue. This condition can be written as the equation: $T(v) = \lambda v$.
The above statement is very elegantly explained using this gif:
Vectors denoted in blue $\begin{bmatrix}1 \\1 \\ \end{bmatrix}$ and magenta $\begin{bmatrix}1 \\-1 \\ \end{bmatrix}$ are eigenvectors for the linear transformation, $T = \begin{bmatrix}2 & 1 \\1 & 2 \\ \end{bmatrix}$. The points that lie on the line through the origin, parallel to the eigenvectors, remain on the line after the transformation. The vectors in red are not eigenvectors, therefore their direction is altered by the transformation. Blue vectors are scaled by a factor of 3 -- which is the eigenvalue for the blue eigenvector, whereas the magenta vectors are not scaled, since their eigenvalue is 1.
Link to Wikipedia article.
Trade-off bias variance is another very important concept in Statistics/Machine Learning.
The data points in blue come from $y(x)=\sin(x)+\epsilon$, where $\epsilon$ has a normal distribution. The red curves are estimated using different samples. The figure "Large Variance and Small Bias" presents the original model, which is Radial basis function network with 24 gaussian bases.
The figure "Small Variance and Large Bias" presents the same model regularized.
Note that in the figure "Small Variance and Large Bias" the red curves are very close to each other (small variance). The same does not happen in the figure "Large Variance and Small Bias" (large variance).
Small Variance and Large Bias
Large Variance and Small Bias
Here is very basic one, but in my opinion very powerful because it's not only a visual explanation of a concept but also asks for visualising or imagining a real object depicting the concept:
Neophytes sometimes have a hard time understanding very basic concepts like mean, median and mode.
So, for helping them to better grasp the idea of mean:
Take this skewed distribution and do a 3D print of it, in plastic, or carve it in wood, so now you have a real object in your hands. Try to balance it using just one finger... the mean is the only point where you can do that.
The figure below shows the importance of defining preciselly the objectives and assumptions of a clustering problem (and a general statistical problem). Different models may provide very different results:
Sources: ScikitLearn
Okay, so this one is less about illustrating a basic concept, but it is very interesting both visually and in terms of applications. I think showing people what they can ultimately accomplish with what they are learning is a great form of motivation, so you can pitch it as an example of developing and applying statistical models, which depends on all the more fundamental statistical concepts they are learning. With that, I present to you...
Species Distribution Modelling
It's actually a very broad topic with a lot of nuance in terms of types of data, data collection, model setup, assumptions, applications, interpretations, etc. But very simply put, you take sample information about where a species occurs, then use those locations to sample potentially relevant environmental variables (e.g., climate data, soil data, habitat data, elevation, light pollution, noise pollution, etc), develop a model using the data (e.g., GLM, point process model, etc), then use that model to predict across a landscape using your environmental variables. Depending on how the model was setup, what's predicted might be potential suitable habitat, likely areas of occurrence, species distribution, etc. You can also change the environmental variables to see how they impact these results. People have used SDMs to find previously unknown populations of a species, they've used them to discover new species, with historical climate data they've used them to predict backwards in time where a species used to occur and how it got to where it is today (even all the way back through glaciation periods), and with things like future climate predictions and habitat loss, they are used to predict how human activities will affect the species in the future. These are just a few examples, and if I have time later I'll find and link interesting papers. In the meantime here's a quick image I found illustrating the basics:
