All Questions
Tagged with nonparametric bayesian
60
questions
0
votes
0
answers
33
views
Is bootstrapping inherently Frequentist? If so, how do we do a Bayesian non-parametric two-sample test?
I normally use frequentist statistics but I now want to use Bayesian statistics as I want to carry out a two-sample (randomised control trial) test that includes prior information. I have an existing ...
- 913
4
votes
1
answer
52
views
In what ways is Gaussian Process Regression both parametric and non-parametric?
Gaussian Process Regression is considered a "non-parametric" model. However, the term "non-parametric" is often used imprecisely to mean different things, leading to questions ...
- 847
0
votes
0
answers
30
views
Fisher information or Bayesian Uncertainty for non-parametric distributions
This question sounds ridiculous, let me clarify motivation:
Fisher information & Bayesian inference uncertainty seemed very cool to me because they can effectively tell you "how ...
- 411
0
votes
0
answers
26
views
BART with non-parametric heteroscedastic noise?
Is there a variant of BART that robustly captures noise that is both heteroscedastic and non-parametric (or has an a-priori unknown parametric form)?
For example, a BART that could fit this test data:
...
- 467
0
votes
0
answers
65
views
Bayesian analysis of non-normally distributed variable
I would like to use an Bayesian approach to compare a continuous non-normally distributed variable taking values between -1 to 1 between two populations. The measurements are not paired.
Overall my ...
- 101
1
vote
0
answers
61
views
How can I combined Bayesian and non-parametric techniques?
I'd like to combine Bayesian and non-parametric (e.g. XGBoost) models, with the goal of getting a probability distribution over my target variable rather than a point estimate. I have a prior, and I ...
- 851
2
votes
4
answers
235
views
good intermediate-level textbook for undergraduate applied statistics in data science?
I will be teaching an applied statistics course for the first time and the main audience will be 2nd and 3rd year undergraduates, mostly data science majors. They will have an intro statistics course ...
1
vote
0
answers
61
views
trace class of prior covariance operator in Bayesian inference problem
I'm interested in certain Bayesian inference problems where the vector space $Q$ where the parameters $\theta$ live is infinite-dimensional.
These show up all the time in the geophysical sciences -- ...
- 143
1
vote
1
answer
136
views
Deciding the Number of Clusters : Standard Methods vs. Non-Parametric Methods
I was watching this video over here (https://www.youtube.com/watch?v=UBiaLq5V7mE) that discussed a Non-Parametric based Bayesian approach for deciding the number of clusters in a dataset.
Essentially, ...
2
votes
0
answers
73
views
MCMC fitting of Dirichlet Process or Polya Tree prior to residuals in (simple linear regression)/(2-independent-samples) problem
Consider a simple location-shift semi-parametric model with two mutually-independent samples (in what follows, $F$ is a cumulative distribution function (CDF) on $\mathbb{ R }$, the $C_i$ and $T_j$ ...
- 69
2
votes
0
answers
140
views
MCMC fitting of a Dirichlet Process or Polya Tree prior to the residuals in a (simple linear regression)/(2-independent-samples) problem
Consider a simple location-shift semi-parametric model with two mutually-independent samples (here $F$ is a cumulative distribution function (CDF) on $\mathbb{ R }$, the $C_i$ and $T_j$ are real-...
- 69
2
votes
1
answer
854
views
KNN as a crude prototype of Gaussian Process Regression?
I've heard it said before that K-Means-Clustering is a prototypical method for Expectation-Maximization algorithm. Where KM Clustering returns a hard cluster assignment, EM returns soft assignments, ...
- 3,382
2
votes
0
answers
41
views
Unexpected zero on posterior density of Dirichlet process mixture
I was reading this notebook from the PyMC3 documentation about Dirichlet Process Mixtures and, on the last figure, the estimated density reaches almost zero for a particular value, despite the ...
- 2,680
2
votes
0
answers
73
views
distance for abc - nonparametric likelihood
When fitting models using abc, data is simulated using parameters drawn from the prior. The distance between the simulated data and the observed data is calculated, and typically if less than a ...
- 33
0
votes
2
answers
1k
views
Is there a Bayesian Non-Parametric one-way ANOVA?
The rough idea is that I am trying to compare linguistic properties (e.g. readability) between pieces of texts from two authors essentially. For this, I thought using an ANOVA would be appropriate. ...
0
votes
0
answers
27
views
Question about possible typo in a tutorial about the stick-breaking model of the Dirichlet distribution
I am reading a tutorial on the Dirichlet distribution: http://mayagupta.org/publications/FrigyikKapilaGuptaIntroToDirichlet.pdf
and I think there is a typo in Step 2 of the stick-breaking model of ...
1
vote
0
answers
31
views
Is data modeled by dirichlet process mixture exchangeable?
Consider DPM model:
$$
\begin{aligned} X_{i} | \phi_{i} & \sim F\left(x;\phi_{i}\right) \\ \phi_{1}, \phi_{2}, \cdots | & P \stackrel{iid}{\sim} P \\ P & \sim D P(\alpha G_0) \end{aligned}
...
- 241
1
vote
0
answers
28
views
Estimation hardness results in Bayesian inference?
Frequentist statistics has a series of fundamental hardness results that are encountered by beginning statistics students. In non-parametric statistics, a famous hardness result for the normal means ...
- 53
2
votes
0
answers
46
views
Directly applying residual bootstrap to the predictions vs. inferring the parameters?
My friend has a procedure where he does the following:
Given a dataset $(x_1,y_1),\ldots,(x_n, y_n)$ Fit $f$ according to $\hat{y_i} = f(x_i) + \epsilon_i$ where $f$ is the regression function.
...
- 203
11
votes
1
answer
514
views
Do Stochastic Processes such as the Gaussian Process/Dirichlet Process have densities? If not, how can Bayes rule be applied to them?
The Dirichlet Pocess and Gaussian Process are often referred to as "distributions over functions" or "distributions over distributions". In that case, can I meaningfully talk about the density of a ...
3
votes
2
answers
233
views
Simulating the Posterior Density of a Transformed Parameters
I am reviewing an example (p. 180-181, Example 11.3 and 11.4) from All of Statistics by Larry Wasserman. The example intends to illustrate that the posterior can be found analytically and can be ...
- 159
0
votes
2
answers
72
views
Likelihood term in Bayesian inferencing versus the general definition
In general we say that the likelihood function is defined as some $L(\theta|x)$, so that it is a function over some parameters: $\theta$ given some data: $x$. That is, $\theta$ is free to vary whilst $...
- 545
2
votes
0
answers
133
views
Smooth regression algorithms that produce zero training error
I am looking to fit three regression functions $f_1, f_2, f_3:\mathbb{R}^2 \to \mathbb{R}$. For example, let's say $X_1$ is time, $X_2$ is geographic latitude, $f_1$ is the temperature, $f_2$ is the ...
- 201
4
votes
1
answer
530
views
Is parametric Bayesian inference a special case of nonparametric Bayesian inference?
I'm thinking about univariate density estimation.
Original Question
In parametric inference, you assume the data are generated from a density that can be summarized by finitely-many parameters. You ...
- 1,425
4
votes
1
answer
958
views
Is there a loss function when estimating a model using MCMC?
I am trying to understand how fitting a model using MCMC works. Is there a loss function that is optimized?
Or is it simply a case of more draws from the distribution amount to a more complete ...
- 12.1k
1
vote
0
answers
64
views
Bayesian posterior from pairwise comparison of observations
Say I have $n$ observations of group $A$ and $m$ observations of group $B$ and a function $f: A\times B \rightarrow C$ mapping a pair of observations to one of $k$ categories.
I am interested in the ...
1
vote
0
answers
690
views
Bayesian Wilcoxon test
I have a pre-post dataset with 2 observations per subject (propotion data -bounded between 0 and 1-).
I have analyzed the data with a classical dependent t-test under the NHST paradigm. However, as ...
- 157
0
votes
2
answers
266
views
Book Bayesian Nonparametrics [duplicate]
What is the best recommended book on Bayesian Non parametric approaches ? Specifically something which also tackles regression problems such as Gaussian processes.
1
vote
0
answers
74
views
Clustering and Dirichlet process' parameter
I am reading a paper in which they describe a bayesian model in which the prior $a_i$ is defined as a Dirichlet Process (DP). They say: "We use a DP to find the optimal $a_i$ via clustering".
Later on ...
- 11
8
votes
2
answers
665
views
What is a mixture of finite mixtures?
A mixture of finite mixture models seem to be an interesting Bayesian (?) approach to solving clustering with an unknown $k$ number of components. It seems though, unlike the mixture model with a ...
- 3,056
3
votes
1
answer
111
views
Robbins estimate Empirical Bayes
From the compound sampling model where:
$Y_i | \theta_i \sim Poi(\theta_i)$
The marginal distribution of $\theta_i$ is $G$, non-parametric.
We get that the Bayes estimate of $\theta_i$ under ...
- 347
2
votes
1
answer
195
views
Reference for poor sampler mixing in large bayesian models
I keep seeing this in various presentations, but never saw a reference for it. Although it makes an intuitive sense why samplers potentially can face mixing issue when operating on large space of ...
- 1,502
1
vote
0
answers
31
views
Estimating Gamma PDF parameters from data with negative increments
Say we have collected data, and from a physical perspective we know that the collected data should increase positively with time. However the data looks more like this:
This data shown in the figure ...
- 565
1
vote
1
answer
18
views
Measuring quality of random items - probability that quality exceeds a without any assumptions
Say I draw $n$ random items and measure their quality in the interval $[0,1]$. Now I would like to know: If I draw another item, what is the probability that this item has a quality larger than $0.5$? ...
- 103
2
votes
1
answer
895
views
Combining triangular distributions
Vose (in Risk analysis a quantitative guide, 2008) argues that it is preferable to use non-parametric distributions when eliciting knowledge about an unknown distribution from experts. The argument is ...
- 33
31
votes
2
answers
10k
views
Is it true that Bayesian methods don't overfit?
Is it true that Bayesian methods don't overfit? (I saw some papers and tutorials making this claim)
For example, if we apply a Gaussian Process to MNIST (handwritten digit classification), but only ...
- 1,337
6
votes
1
answer
2k
views
What does the base distribution of the Dirichlet Process mean?
So far I only really understand the Dirichlet Process through its various metaphors. For the Polya Urn scheme, my understanding is that the "base distribution" is the original distribution of colors ...
- 1,002
8
votes
2
answers
2k
views
Bayesian nonparametric answer to deep learning?
As I understand it, deep neural networks are performing "representation learning" by layering features together. This allows learning very high dimensional structures in the features. Of course, it's ...
- 1,002
1
vote
0
answers
53
views
Nonparametric density estimation, individual probablities
Consider the problem of doing nonparametric density estimation using kernel density estimator in the common form
$k(\frac{\textbf{x} - \mathbf{x_{j}}}{h})$,
$k(\textbf{u}) = \begin{cases}
1 & \...
- 121
0
votes
1
answer
267
views
Understanding Gaussian Process and their Priors
I am very interested to understand the motivation behind why are we using these priors let's say in the context of regression. I know that the kernel depicts the distance between the points or let's ...
- 3
8
votes
1
answer
1k
views
Nonparametric nonlinear regression with prediction uncertainty (besides Gaussian Processes)
What are state-of-the-art alternatives to Gaussian Processes (GP) for nonparametric nonlinear regression with prediction uncertainty, when the size of the training set starts becoming prohibitive for ...
- 5,226
8
votes
1
answer
276
views
Dirichlet process mixture MCMC
I'm reading Markov Chain Sampling Methods for Dirichlet Process Mixture Models by Radford M. Neal. Equation (3.6) states that
$$
\text{If } c=c_{j} \text{ for some } j\neq i: P\left(c_{i}=c\;|\;c_{-i}...
- 1,142
1
vote
0
answers
444
views
Need for iid in MLE
I am studying about parametric estimation in supervised learning using maximum likelihood estimation. Here is what I learned:
Separate our training data according to class; i.e., we have c data sets ...
- 235
4
votes
0
answers
44
views
German tank variant: estimate resolution of camera given cropped photo sizes
Make whatever assumptions you like, but I like the flavor of nonparametric techniques.
I have a list of the $x_i$ by $y_i$ resolutions of a number of photos, all cropped from photos taken at the same ...
- 2,121
4
votes
0
answers
408
views
Is this how a Bayesian bootstrap works?
I am a bit new to the whole nonparametric and Bayesian idea, so tell me if this is correct: to estimate, say, the mean of a dataset's population we do the following:
We define a function $f(x)$ that ...
- 2,121
1
vote
1
answer
513
views
Likely mean of a multinomial distribution with dirichlet prior
I am working to create a Bayesian non-parametric estimate of the mean of a distribution given a distribution of observations. Ultimately I'd like to get to a credibility interval of the likely mean of ...
- 1,229
2
votes
1
answer
177
views
What does "CRP is a marginalized version of PYP" mean?
I've been reading this phrase and I don't know what it means "CRP is a marginalized version of PYP". What are the parameters/latent-variables we are marginalizing out to drive CRP from PYP?
- 1,502
5
votes
1
answer
857
views
Gaussian Process and Expectation Propagation time complexity?
What's the time complexity of training a Gaussian process and its Expectation Propagation approximation?
(Before studying them, I'd like to understand if they are even feasible for my application)
- 1,337
5
votes
4
answers
251
views
Probablistic counterpart for kNN
We know that the Gaussian Mixture Model is a probabilistic counterpart of k-means algorithm. Is there a probabilistic counterpart for kNN? (which is similar to k-means, but supervised.)
- 1,586
2
votes
1
answer
504
views
Why semi/nonparametric models?
Increasing the flexibility of models makes it prone to overfitting. On the other hand, it looks to me that, if the space function classes $\mathcal{F}$ is too big, it is hard to prove bounds on ...
- 1,586