I just rigorously learned that OLS is a special case of MLE. It surprises me because the popular and "reliable" sources such as researchgate and this do not mention this most important connection between MLE and OLS!
I am not sure if there are any simple regression or estimation method that does not belong to MLE.
Least squares is indeed maximum likelihood if the errors are iid normal, but if they aren't iid normal, least squares is not maximum likelihood. For example if my errors were logistic, least squares wouldn't be a terrible idea but it wouldn't be maximum likelihood.
Lots of estimators are not maximum likelihood estimators; while maximum likelihood estimators typically have a number of useful and attractive properties they're not the only game in town (and indeed not even always a great idea).
A few examples of other estimation methods would include
method of moments (this involves equating enough sample and population moments to solve for parameter estimates; sometimes this turns out to be maximum likelihood but usually it doesn't)
For example, equating first and second moments to estimate the parameters of a gamma distribution or a uniform distribution; not maximum likelihood in either case.
method of quantiles (equating sufficient sample and population quantiles to solve for parameter estimates; occasionally this is maximum likelihood but usually it isn't),
minimizing some other measure of lack of fit than $-\log\mathcal{L}$ (e.g. minimum chi-square, minimum K-S distance).
With fitting linear regression type models, you could for example look at robust regression (some of which do correspond to ML methods for some particular error distribution but many of which do not).
In the case of simple linear regression, I show an example of two methods of fitting lines that are not maximum likelihood here - there estimating slope by setting to 0 some other measure of correlation (i.e. other than the usual Pearson) between residuals and the predictor.
Another example would be the Tukey's resistant line/Tukey's three group line (e.g. see ?line in R). There are many other possibilities, though many of them don't generalize readily to the multiple regression situation.
All MLE is minimax but not all minimax is MLE. Some examples of minimax estimators that do not maximize a likelihood are ROC regression, conditional logistic regression, Cox proportional hazards models, nearest neighbor, quasilikelihood, the list goes on and on. Hodge's "superefficient" estimator beats maximum likelihood as a more efficient UMVUE (unbiased minimum variance) estimator of the mean in a normal sample but it is NOT minimax
Bayesian approaches do not involve maximizing a likelihood function, but rather integrating over a posterior distribution. Note that the underlying model may be exactly identical (i.e., linear regression, generalized linear regression), but we also need to provide a prior distribution which captures our uncertainty in the parameters before seeing the data. The posterior distribution is simply the normalized distribution of the prior times the likelihood.
I believe that most statisticians these days generally agree that a Bayesian approach is generally superior to an MLE approach for parameter estimation. However, when one has a lot of data, it may not be so much better that it's both the extra computational costs (integrating is harder than optimizing!) and extra effort of coming up with a prior distribution. In fact, one can show that asymptotically, the MLE + normal approximation approaches the posterior distribution under certain conditions.
$$ Y_i = \alpha + \beta x_i + \varepsilon_i $$
Suppose you have the Gauss–Markov assumptions:
One cannot do MLE because there's no parametrized family of distributions. But one can still do ordinary least squares.
And among all linear combinations of the $y$-values with non-random observable coefficients, that are unbaised estimators of $\alpha$ and $\beta,$ the least-squares estimators have the smallest variance.
An answer to the question "What regression/estimation is not a MLE?", a simple and robust alternative to Least-Squares (LS) is reportedly Least-Absolute Deviation (LAD).
To quote a source:
"The least absolute deviations method (LAD) is one of the principal alternatives to the least-squares method when one seeks to estimate regression parameters. The goal of the LAD regression is to provide a robust estimator."
Interestingly, per a reference, to quote "The least absolute deviations estimate also arises as the maximum likelihood estimate if the errors have a Laplace distribution." Here is a link that discusses some interesting applications of the Laplace (like as a Bayesian prior, and for extreme events).
Historically, the LAD procedure was introduced 50 years before the least-squares method (1757) by Roger Joseph Boscovich, who employed it to reconcile incoherent measures relating to the shape of the earth.
An illustrative difference is in the very simple case of Y = Constant, where the LS returns the sample mean, while the LAD selects the sample median! So in contexts with one or two extreme values, which for whatever reason (like heteroscedasticity), that may arise, LS could display a major shift in the true slope estimate, especially when there is one very low and/or a high observation, as a noted weakness. Wikipedia on robust regression makes a supporting comment:
"In particular, least squares estimates for regression models are highly sensitive to outliers."
With respect to applications, this can be particularly important, for example, in chemistry-based data analysis to predict a so-called reaction's Rate Law (which is based on the slope estimate).
