Class InputDStream<T>

All Implemented Interfaces:
Serializable, org.apache.spark.internal.Logging, scala.Serializable
Direct Known Subclasses:
ConstantInputDStream, ReceiverInputDStream

public abstract class InputDStream<T> extends DStream<T>
This is the abstract base class for all input streams. This class provides methods start() and stop() which are called by Spark Streaming system to start and stop receiving data, respectively. Input streams that can generate RDDs from new data by running a service/thread only on the driver node (that is, without running a receiver on worker nodes), can be implemented by directly inheriting this InputDStream. For example, FileInputDStream, a subclass of InputDStream, monitors an HDFS directory from the driver for new files and generates RDDs with the new files. For implementing input streams that requires running a receiver on the worker nodes, use ReceiverInputDStream as the parent class.

param: _ssc Streaming context that will execute this input stream

See Also:
  • Constructor Details

    • InputDStream

      public InputDStream(StreamingContext _ssc, scala.reflect.ClassTag<T> evidence$1)
  • Method Details

    • dependencies

      public scala.collection.immutable.List<DStream<?>> dependencies()
      Description copied from class: DStream
      List of parent DStreams on which this DStream depends on
      Specified by:
      dependencies in class DStream<T>
    • id

      public int id()
      This is a unique identifier for the input stream.
    • slideDuration

      public Duration slideDuration()
      Description copied from class: DStream
      Time interval after which the DStream generates an RDD
      Specified by:
      slideDuration in class DStream<T>
    • start

      public abstract void start()
      Method called to start receiving data. Subclasses must implement this method.
    • stop

      public abstract void stop()
      Method called to stop receiving data. Subclasses must implement this method.