Seminar 4. Probabilistic Topic Model

Topic modeling
Models of a collection of composites
Composites are documents
Parts are words (or phrases, n-grams)
Two outputs:
chance of selecting a particular part when sampling a particular topic
chance of selecting a particular topic when sampling a particular document or composite

Assumptions
semantic information can be derived from a word-document co-occurrence matrix;
topic is a probability distribution over words
to make a new document, one chooses a distribution over topics
for each word in that document, one chooses a topic at random according to this distribution, and draws a word from that topic.
Resulting document is a mixture of topics

Generative model

Probabilistic model

Dirichlet distribution

Geometric interpretation