Type Members

1.  class JavaDoubleRDD extends AbstractJavaRDDLike[Double, JavaDoubleRDD]
2.  trait JavaFutureAction[T] extends Future[T]
3.  class JavaHadoopRDD[K, V] extends JavaPairRDD[K, V]

   Annotations

   @DeveloperApi()

4.  class JavaNewHadoopRDD[K, V] extends JavaPairRDD[K, V]

   Annotations

   @DeveloperApi()

5.  class JavaPairRDD[K, V] extends AbstractJavaRDDLike[(K, V), JavaPairRDD[K, V]]
6.  class JavaRDD[T] extends AbstractJavaRDDLike[T, JavaRDD[T]]
7.  trait JavaRDDLike[T, This <: JavaRDDLike[T, This]] extends Serializable

   Defines operations common to several Java RDD implementations.

   Defines operations common to several Java RDD implementations.

   Note

   This trait is not intended to be implemented by user code.

8.  class JavaSparkContext extends Closeable

   A Java-friendly version of org.apache.spark.SparkContext that returns org.apache.spark.api.java.JavaRDDs and works with Java collections instead of Scala ones.

   A Java-friendly version of org.apache.spark.SparkContext that returns org.apache.spark.api.java.JavaRDDs and works with Java collections instead of Scala ones.

   Note

   Only one SparkContext should be active per JVM. You must stop() the active SparkContext before creating a new one.

9.  class JavaSparkStatusTracker extends AnyRef

   Low-level status reporting APIs for monitoring job and stage progress.

   Low-level status reporting APIs for monitoring job and stage progress.

   These APIs intentionally provide very weak consistency semantics; consumers of these APIs should be prepared to handle empty / missing information. For example, a job's stage ids may be known but the status API may not have any information about the details of those stages, so getStageInfo could potentially return null for a valid stage id.

   To limit memory usage, these APIs only provide information on recent jobs / stages. These APIs will provide information for the last spark.ui.retainedStages stages and spark.ui.retainedJobs jobs.

   Note

   This class's constructor should be considered private and may be subject to change.

10.  final class Optional[T] extends Serializable

    Like java.util.Optional in Java 8, scala.Option in Scala, and com.google.common.base.Optional in Google Guava, this class represents a value of a given type that may or may not exist.

    Like java.util.Optional in Java 8, scala.Option in Scala, and com.google.common.base.Optional in Google Guava, this class represents a value of a given type that may or may not exist. It is used in methods that wish to optionally return a value, in preference to returning null.

    In fact, the class here is a reimplementation of the essential API of both java.util.Optional and com.google.common.base.Optional. From java.util.Optional, it implements:

    • #empty()
    • #of(Object)
    • #ofNullable(Object)
    • #get()
    • #orElse(Object)
    • #isPresent()

    From com.google.common.base.Optional it implements:

    • #absent()
    • #of(Object)
    • #fromNullable(Object)
    • #get()
    • #or(Object)
    • #orNull()
    • #isPresent()

    java.util.Optional itself was not used because at the time, the project did not require Java 8. Using com.google.common.base.Optional has in the past caused serious library version conflicts with Guava that can't be resolved by shading. Hence this work-alike clone.

11.  class StorageLevels extends AnyRef

    Expose some commonly useful storage level constants.