BaseReadWrite

class pyspark.ml.util.BaseReadWrite[source]

Base class for MLWriter and MLReader. Stores information about the SparkContext and SparkSession.

New in version 2.3.0.

Methods

session(sparkSession)

Sets the Spark Session to use for saving/loading.

Attributes

sc

Returns the underlying SparkContext.

sparkSession

Returns the user-specified Spark Session or the default.

Methods Documentation

session(sparkSession: pyspark.sql.session.SparkSession) → RW[source]

Sets the Spark Session to use for saving/loading.

Attributes Documentation

sc

Returns the underlying SparkContext.

sparkSession

Returns the user-specified Spark Session or the default.