public class CountVectorizer extends Estimator<CountVectorizerModel> implements CountVectorizerParams, DefaultParamsWritable
CountVectorizerModel
.Constructor and Description |
---|
CountVectorizer() |
CountVectorizer(String uid) |
Modifier and Type | Method and Description |
---|---|
BooleanParam |
binary()
Binary toggle to control the output vector values.
|
CountVectorizer |
copy(ParamMap extra)
Creates a copy of this instance with the same UID and some extra params.
|
CountVectorizerModel |
fit(Dataset<?> dataset)
Fits a model to the input data.
|
Param<String> |
inputCol()
Param for input column name.
|
static CountVectorizer |
load(String path) |
DoubleParam |
maxDF()
Specifies the maximum number of different documents a term could appear in to be included
in the vocabulary.
|
DoubleParam |
minDF()
Specifies the minimum number of different documents a term must appear in to be included
in the vocabulary.
|
DoubleParam |
minTF()
Filter to ignore rare words in a document.
|
Param<String> |
outputCol()
Param for output column name.
|
static MLReader<T> |
read() |
CountVectorizer |
setBinary(boolean value) |
CountVectorizer |
setInputCol(String value) |
CountVectorizer |
setMaxDF(double value) |
CountVectorizer |
setMinDF(double value) |
CountVectorizer |
setMinTF(double value) |
CountVectorizer |
setOutputCol(String value) |
CountVectorizer |
setVocabSize(int value) |
StructType |
transformSchema(StructType schema)
Check transform validity and derive the output schema from the input schema.
|
String |
uid()
An immutable unique ID for the object and its derivatives.
|
IntParam |
vocabSize()
Max size of the vocabulary.
|
params
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
getBinary, getMaxDF, getMinDF, getMinTF, getVocabSize, validateAndTransformSchema
getInputCol
getOutputCol
clear, copyValues, defaultCopy, defaultParamMap, explainParam, explainParams, extractParamMap, extractParamMap, get, getDefault, getOrDefault, getParam, hasDefault, hasParam, isDefined, isSet, onParamChange, paramMap, params, set, set, set, setDefault, setDefault, shouldOwn
toString
write
save
$init$, initializeForcefully, initializeLogIfNecessary, initializeLogIfNecessary, initializeLogIfNecessary$default$2, initLock, isTraceEnabled, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning, org$apache$spark$internal$Logging$$log__$eq, org$apache$spark$internal$Logging$$log_, uninitialize
public CountVectorizer(String uid)
public CountVectorizer()
public static CountVectorizer load(String path)
public static MLReader<T> read()
public IntParam vocabSize()
CountVectorizerParams
Default: 2^18^
vocabSize
in interface CountVectorizerParams
public DoubleParam minDF()
CountVectorizerParams
Default: 1.0
minDF
in interface CountVectorizerParams
public DoubleParam maxDF()
CountVectorizerParams
Default: (2^63^) - 1
maxDF
in interface CountVectorizerParams
public DoubleParam minTF()
CountVectorizerParams
Note that the parameter is only used in transform of CountVectorizerModel
and does not
affect fitting.
Default: 1.0
minTF
in interface CountVectorizerParams
public BooleanParam binary()
CountVectorizerParams
binary
in interface CountVectorizerParams
public final Param<String> outputCol()
HasOutputCol
outputCol
in interface HasOutputCol
public final Param<String> inputCol()
HasInputCol
inputCol
in interface HasInputCol
public String uid()
Identifiable
uid
in interface Identifiable
public CountVectorizer setInputCol(String value)
public CountVectorizer setOutputCol(String value)
public CountVectorizer setVocabSize(int value)
public CountVectorizer setMinDF(double value)
public CountVectorizer setMaxDF(double value)
public CountVectorizer setMinTF(double value)
public CountVectorizer setBinary(boolean value)
public CountVectorizerModel fit(Dataset<?> dataset)
Estimator
fit
in class Estimator<CountVectorizerModel>
dataset
- (undocumented)public StructType transformSchema(StructType schema)
PipelineStage
We check validity for interactions between parameters during transformSchema
and
raise an exception if any parameter value is invalid. Parameter value checks which
do not depend on other parameters are handled by Param.validate()
.
Typical implementation should first conduct verification on schema change and parameter validity, including complex parameter interaction checks.
transformSchema
in class PipelineStage
schema
- (undocumented)public CountVectorizer copy(ParamMap extra)
Params
defaultCopy()
.copy
in interface Params
copy
in class Estimator<CountVectorizerModel>
extra
- (undocumented)