org.apache.spark.shuffle.api.metadata

Class MapOutputCommitMessage

Object

org.apache.spark.shuffle.api.metadata.MapOutputCommitMessage

@Private
public final class MapOutputCommitMessage
extends Object

:: Private :: Represents the result of writing map outputs for a shuffle map task.

Partition lengths represents the length of each block written in the map task. This can be used for downstream readers to allocate resources, such as in-memory buffers.

Map output writers can choose to attach arbitrary metadata tags to register with a shuffle output tracker (a module that is currently yet to be built in a future iteration of the shuffle storage APIs).

Method Summary

Modifier and Type	Method and Description
java.util.Optional<MapOutputMetadata>	getMapOutputMetadata()
long[]	getPartitionLengths()
static MapOutputCommitMessage	of(long[] partitionLengths)
static MapOutputCommitMessage	of(long[] partitionLengths, MapOutputMetadata mapOutputMetadata)

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

of

public static MapOutputCommitMessage of(long[] partitionLengths)

of

public static MapOutputCommitMessage of(long[] partitionLengths,
                                        MapOutputMetadata mapOutputMetadata)

getPartitionLengths

public long[] getPartitionLengths()

getMapOutputMetadata

public java.util.Optional<MapOutputMetadata> getMapOutputMetadata()