All Questions
Tagged with dirichlet-distribution latent-dirichlet-alloc
9
questions
1
vote
0
answers
64
views
Why does latent dirichlet allocation (LDA) fail when dealing with large and heavy-tailed vocabularies?
I'm reading the 2019 paper Topic Modeling in Embedding Spaces which claims that the embedded topic model improves on these limitations of LDA. But why does LDA have these limitations—why does it fail ...
1
vote
1
answer
49
views
In Latent Dirichlet allocation, is the following formula the probability of observing a single document, or an entire corpus?
This is the formula in question:
Source: https://en.wikipedia.org/wiki/Latent_Dirichlet_allocation
- 13
1
vote
1
answer
502
views
Inference on Dirichlet hyper-parameter
I'm working on a Gibbs sampler for a (somewhat custom version of) Latent Dirichlet Allocation model.
In short, I have data that comes from a $K$-dimensional Dirichlet-Multinomial distribution, i.e.
$$...
- 153
0
votes
1
answer
158
views
LDA alpha equivalent in structural topic model
I'm using an implementation of the structural topic model (stm), written in R using the stm package.
I want to reduce the number of topics that are prevalent in ...
- 25
0
votes
1
answer
230
views
Definition of distribution conditioned on both a categorical and Dirichlet prior
If we have a conditional categorical distribution, with unknown parameters, we can represent with a table, as in the example below:
\begin{align*}
&z \quad P(z|\theta)\\
&0 \quad \theta_0\\
&...
- 191
0
votes
0
answers
101
views
Recovering $\theta$ in Dirichlet-Multinomial (Polya) distribution
I'm working on Latent Dirichlet Allocation with Collapsed Gibbs Sampling.
LDA has two Dirichlet-Multinomial distribution and one of them is a document-topic distribution that determines the ...
- 245
12
votes
0
answers
2k
views
Is sparsity of topics a necessary condition for latent Dirichlet allocation (LDA) to work
I have been playing with the hyper-parameters of the latent Dirichlet allocation (LDA) model and am wondering how sparsity of topic priors play a role in inference.
I have not performed these ...
- 3,592
4
votes
2
answers
768
views
Topic Models: Latent Dirichlet Allocations
I am trying to figure out the details of LDA and have been stuck for a while now. While reading the paper by Blei, I came across this -
Latent Dirichlet allocation (LDA) is a generative ...
- 1,087
3
votes
2
answers
3k
views
Latent Dirichlet Allocation (LDA): What exactly is inferred?
I am working my way through LDA and I think I got they main idea of it. Please correct me if I am wrong. Given the Plate notation:
The variables $\alpha$ and $\beta$ are Dirichlet distribution ...
- 276