|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
Object org.apache.spark.mllib.clustering.LDAModel org.apache.spark.mllib.clustering.DistributedLDAModel
public class DistributedLDAModel
:: Experimental ::
Distributed LDA model.
This model stores the inferred topics, the full training dataset, and the topic distributions.
When computing topics for new documents, it may give more accurate answers
than the LocalLDAModel
.
Method Summary | |
---|---|
scala.Tuple2<int[],double[]>[] |
describeTopics(int maxTermsPerTopic)
Return the topics described by weighted terms. |
double[] |
iterationTimes()
|
JavaPairRDD<Long,Vector> |
javaTopicDistributions()
Java-friendly version of topicDistributions |
int |
k()
Number of topics |
double |
logLikelihood()
Log likelihood of the observed tokens in the training set, given the current parameter estimates: log P(docs | topics, topic distributions for docs, alpha, eta) |
double |
logPrior()
Log probability of the current parameter estimate: log P(topics, topic distributions for docs | alpha, eta) |
LocalLDAModel |
toLocal()
Convert model to a local model. |
RDD<scala.Tuple2<Object,Vector>> |
topicDistributions()
For each document in the training set, return the distribution over topics for that document ("theta_doc"). |
Matrix |
topicsMatrix()
Inferred topics, where each topic is represented by a distribution over terms. |
int |
vocabSize()
Vocabulary size (number of terms or terms in the vocabulary) |
Methods inherited from class org.apache.spark.mllib.clustering.LDAModel |
---|
describeTopics |
Methods inherited from class Object |
---|
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Method Detail |
---|
public int k()
LDAModel
k
in class LDAModel
public int vocabSize()
LDAModel
vocabSize
in class LDAModel
public double[] iterationTimes()
public LocalLDAModel toLocal()
public Matrix topicsMatrix()
WARNING: This matrix is collected from an RDD. Beware memory usage when vocabSize, k are large.
topicsMatrix
in class LDAModel
public scala.Tuple2<int[],double[]>[] describeTopics(int maxTermsPerTopic)
LDAModel
This limits the number of terms per topic. This is approximate; it may not return exactly the top-weighted terms for each topic. To get a more precise set of top terms, increase maxTermsPerTopic.
describeTopics
in class LDAModel
maxTermsPerTopic
- Maximum number of terms to collect for each topic.
public double logLikelihood()
Note:
- This excludes the prior; for that, use logPrior
.
- Even with logPrior
, this is NOT the same as the data log likelihood given the
hyperparameters.
public double logPrior()
public RDD<scala.Tuple2<Object,Vector>> topicDistributions()
public JavaPairRDD<Long,Vector> javaTopicDistributions()
topicDistributions
|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |