|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
Object org.apache.spark.streaming.dstream.DStream<T> org.apache.spark.streaming.dstream.InputDStream<T>
public abstract class InputDStream<T>
This is the abstract base class for all input streams. This class provides methods
start() and stop() which is called by Spark Streaming system to start and stop receiving data.
Input streams that can generate RDDs from new data by running a service/thread only on
the driver node (that is, without running a receiver on worker nodes), can be
implemented by directly inheriting this InputDStream. For example,
FileInputDStream, a subclass of InputDStream, monitors a HDFS directory from the driver for
new files and generates RDDs with the new files. For implementing input streams
that requires running a receiver on the worker nodes, use
ReceiverInputDStream
as the parent class.
param: ssc_ Streaming context that will execute this input stream
Constructor Summary | |
---|---|
InputDStream(StreamingContext ssc_,
scala.reflect.ClassTag<T> evidence$1)
|
Method Summary | |
---|---|
scala.collection.immutable.List<DStream<?>> |
dependencies()
List of parent DStreams on which this DStream depends on |
int |
id()
This is an unique identifier for the input stream. |
Time |
lastValidTime()
|
Duration |
slideDuration()
Time interval after which the DStream generates a RDD |
abstract void |
start()
Method called to start receiving data. |
abstract void |
stop()
Method called to stop receiving data. |
Methods inherited from class org.apache.spark.streaming.dstream.DStream |
---|
cache, checkpoint, checkpointData, checkpointDuration, compute, context, count, countByValue, countByValueAndWindow, countByWindow, creationSite, filter, flatMap, foreach, foreach, foreachRDD, foreachRDD, generatedRDDs, glom, graph, map, mapPartitions, mustCheckpoint, persist, persist, print, print, reduce, reduceByWindow, reduceByWindow, rememberDuration, repartition, saveAsObjectFiles, saveAsTextFiles, slice, slice, ssc, storageLevel, toPairDStreamFunctions, transform, transform, transformWith, transformWith, union, window, window, zeroTime |
Methods inherited from class Object |
---|
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Methods inherited from interface org.apache.spark.Logging |
---|
initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning |
Constructor Detail |
---|
public InputDStream(StreamingContext ssc_, scala.reflect.ClassTag<T> evidence$1)
Method Detail |
---|
public Time lastValidTime()
public int id()
public scala.collection.immutable.List<DStream<?>> dependencies()
DStream
dependencies
in class DStream<T>
public Duration slideDuration()
DStream
slideDuration
in class DStream<T>
public abstract void start()
public abstract void stop()
|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |