|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
Object org.apache.spark.rdd.RDD<scala.Tuple2<K,C>> org.apache.spark.rdd.ShuffledRDD<K,V,C>
public class ShuffledRDD<K,V,C>
:: DeveloperApi :: The resulting RDD from a shuffle (e.g. repartitioning of data). param: prev the parent RDD. param: part the partitioner used to partition the RDD
Constructor Summary | |
---|---|
ShuffledRDD(RDD<? extends scala.Product2<K,V>> prev,
Partitioner part)
|
Method Summary | |
---|---|
void |
clearDependencies()
Clears the dependencies of this RDD. |
scala.collection.Iterator<scala.Tuple2<K,C>> |
compute(Partition split,
TaskContext context)
:: DeveloperApi :: Implemented by subclasses to compute a given partition. |
scala.collection.Seq<Dependency<?>> |
getDependencies()
Implemented by subclasses to return how this RDD depends on parent RDDs. |
Partition[] |
getPartitions()
Implemented by subclasses to return the set of partitions in this RDD. |
scala.Some<Partitioner> |
partitioner()
Optionally overridden by subclasses to specify how they are partitioned. |
Object |
prev()
|
ShuffledRDD<K,V,C> |
setAggregator(Aggregator<K,V,C> aggregator)
Set aggregator for RDD's shuffle. |
ShuffledRDD<K,V,C> |
setKeyOrdering(scala.math.Ordering<K> keyOrdering)
Set key ordering for RDD's shuffle. |
ShuffledRDD<K,V,C> |
setMapSideCombine(boolean mapSideCombine)
Set mapSideCombine flag for RDD's shuffle. |
ShuffledRDD<K,V,C> |
setSerializer(Serializer serializer)
Set a serializer for this RDD's shuffle, or null to use the default (spark.serializer) |
Methods inherited from class Object |
---|
equals, getClass, hashCode, notify, notifyAll, wait, wait, wait |
Methods inherited from interface org.apache.spark.Logging |
---|
initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning |
Constructor Detail |
---|
public ShuffledRDD(RDD<? extends scala.Product2<K,V>> prev, Partitioner part)
Method Detail |
---|
public Object prev()
public ShuffledRDD<K,V,C> setSerializer(Serializer serializer)
public ShuffledRDD<K,V,C> setKeyOrdering(scala.math.Ordering<K> keyOrdering)
public ShuffledRDD<K,V,C> setAggregator(Aggregator<K,V,C> aggregator)
public ShuffledRDD<K,V,C> setMapSideCombine(boolean mapSideCombine)
public scala.collection.Seq<Dependency<?>> getDependencies()
RDD
public scala.Some<Partitioner> partitioner()
RDD
partitioner
in class RDD<scala.Tuple2<K,C>>
public Partition[] getPartitions()
RDD
public scala.collection.Iterator<scala.Tuple2<K,C>> compute(Partition split, TaskContext context)
RDD
compute
in class RDD<scala.Tuple2<K,C>>
split
- (undocumented)context
- (undocumented)
public void clearDependencies()
RDD
UnionRDD
for an example.
|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |