org.apache.spark.scheduler

Class StageInfo

Object

org.apache.spark.scheduler.StageInfo

public class StageInfo
extends Object

:: DeveloperApi :: Stores information about a stage to pass from the scheduler to SparkListeners.

Constructor Summary

Constructors

Constructor and Description

StageInfo(int stageId, int attemptId, String name, int numTasks, scala.collection.Seq<RDDInfo> rddInfos, scala.collection.Seq<Object> parentIds, String details, org.apache.spark.executor.TaskMetrics taskMetrics, scala.collection.Seq<scala.collection.Seq<TaskLocation>> taskLocalityPreferences, scala.Option<Object> shuffleDepId, int resourceProfileId, boolean isShufflePushEnabled, int shuffleMergerCount)

Method Summary

Modifier and Type	Method and Description
scala.collection.mutable.HashMap<Object,AccumulableInfo>	accumulables() Terminal values of accumulables updated during this stage, including all the user-defined accumulators.
int	attemptNumber()
scala.Option<Object>	completionTime() Time when the stage completed or when the stage was cancelled.
String	details()
scala.Option<String>	failureReason() If the stage failed, the reason why.
static StageInfo	fromStage(org.apache.spark.scheduler.Stage stage, int attemptId, scala.Option<Object> numTasks, org.apache.spark.executor.TaskMetrics taskMetrics, scala.collection.Seq<scala.collection.Seq<TaskLocation>> taskLocalityPreferences, int resourceProfileId) Construct a StageInfo from a Stage.
String	name()
int	numTasks()
scala.collection.Seq<Object>	parentIds()
scala.collection.Seq<RDDInfo>	rddInfos()
int	resourceProfileId()
void	stageFailed(String reason)
int	stageId()
scala.Option<Object>	submissionTime() When this stage was submitted from the DAGScheduler to a TaskScheduler.
org.apache.spark.executor.TaskMetrics	taskMetrics()

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StageInfo

public StageInfo(int stageId,
                 int attemptId,
                 String name,
                 int numTasks,
                 scala.collection.Seq<RDDInfo> rddInfos,
                 scala.collection.Seq<Object> parentIds,
                 String details,
                 org.apache.spark.executor.TaskMetrics taskMetrics,
                 scala.collection.Seq<scala.collection.Seq<TaskLocation>> taskLocalityPreferences,
                 scala.Option<Object> shuffleDepId,
                 int resourceProfileId,
                 boolean isShufflePushEnabled,
                 int shuffleMergerCount)

Method Detail

fromStage

public static StageInfo fromStage(org.apache.spark.scheduler.Stage stage,
                                  int attemptId,
                                  scala.Option<Object> numTasks,
                                  org.apache.spark.executor.TaskMetrics taskMetrics,
                                  scala.collection.Seq<scala.collection.Seq<TaskLocation>> taskLocalityPreferences,
                                  int resourceProfileId)

Construct a StageInfo from a Stage.

Each Stage is associated with one or many RDDs, with the boundary of a Stage marked by shuffle dependencies. Therefore, all ancestor RDDs related to this Stage's RDD through a sequence of narrow dependencies should also be associated with this Stage.

Parameters:

stage - (undocumented)

attemptId - (undocumented)

numTasks - (undocumented)

taskMetrics - (undocumented)

taskLocalityPreferences - (undocumented)

resourceProfileId - (undocumented)

Returns:

(undocumented)

stageId

public int stageId()

name

public String name()

numTasks

public int numTasks()

rddInfos

public scala.collection.Seq<RDDInfo> rddInfos()

parentIds

public scala.collection.Seq<Object> parentIds()

details

public String details()

taskMetrics

public org.apache.spark.executor.TaskMetrics taskMetrics()

resourceProfileId

public int resourceProfileId()

submissionTime

public scala.Option<Object> submissionTime()

When this stage was submitted from the DAGScheduler to a TaskScheduler.

completionTime

public scala.Option<Object> completionTime()

Time when the stage completed or when the stage was cancelled.

failureReason

public scala.Option<String> failureReason()

If the stage failed, the reason why.

accumulables

public scala.collection.mutable.HashMap<Object,AccumulableInfo> accumulables()

Terminal values of accumulables updated during this stage, including all the user-defined accumulators.

Returns:

(undocumented)

stageFailed

public void stageFailed(String reason)

attemptNumber

public int attemptNumber()