Package org.apache.spark.storage

Class RandomBlockReplicationPolicy

Object
org.apache.spark.storage.RandomBlockReplicationPolicy
All Implemented Interfaces:
org.apache.spark.internal.Logging, BlockReplicationPolicy
public class RandomBlockReplicationPolicy extends Object implements BlockReplicationPolicy, org.apache.spark.internal.Logging

  • Nested Class Summary

    Nested classes/interfaces inherited from interface org.apache.spark.internal.Logging

    org.apache.spark.internal.Logging.SparkShellLoggingFilter

  • Constructor Summary

    Constructors
    Constructor
    Description
    RandomBlockReplicationPolicy()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    scala.collection.immutable.List<BlockManagerId>
    prioritize(BlockManagerId blockManagerId, scala.collection.Seq<BlockManagerId> peers, scala.collection.mutable.HashSet<BlockManagerId> peersReplicatedTo, BlockId blockId, int numReplicas)
    Method to prioritize a bunch of candidate peers of a block.

    Methods inherited from class java.lang.Object

    equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface org.apache.spark.internal.Logging

    initializeForcefully, initializeLogIfNecessary, initializeLogIfNecessary, initializeLogIfNecessary$default$2, isTraceEnabled, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning, org$apache$spark$internal$Logging$$log_, org$apache$spark$internal$Logging$$log__$eq

  • Constructor Details

    • RandomBlockReplicationPolicy

      public RandomBlockReplicationPolicy()

  • Method Details

    • prioritize

      public scala.collection.immutable.List<BlockManagerId> prioritize(BlockManagerId blockManagerId, scala.collection.Seq<BlockManagerId> peers, scala.collection.mutable.HashSet<BlockManagerId> peersReplicatedTo, BlockId blockId, int numReplicas)
      Method to prioritize a bunch of candidate peers of a block. This is a basic implementation, that just makes sure we put blocks on different hosts, if possible

      Specified by:
      prioritize in interface BlockReplicationPolicy
      Parameters:
      blockManagerId - Id of the current BlockManager for self identification
      peers - A list of peers of a BlockManager
      peersReplicatedTo - Set of peers already replicated to
      blockId - BlockId of the block being replicated. This can be used as a source of randomness if needed.
      numReplicas - Number of peers we need to replicate to
      Returns:
      A prioritized list of peers. Lower the index of a peer, higher its priority