Package org.apache.spark.mllib.rdd

Class RDDFunctions<T>

Object
org.apache.spark.mllib.rdd.RDDFunctions<T>
All Implemented Interfaces:
Serializable, scala.Serializable
public class RDDFunctions<T> extends Object implements scala.Serializable
Machine learning specific RDD functions.
See Also:

  • Constructor Summary

    Constructors
    Constructor
    Description
    RDDFunctions(RDD<T> self, scala.reflect.ClassTag<T> evidence$1)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    static <T> RDDFunctions<T>
    fromRDD(RDD<T> rdd, scala.reflect.ClassTag<T> evidence$2)
    Implicit conversion from an RDD to RDDFunctions.
    RDD<Object>
    sliding(int windowSize)
    sliding(Int, Int)* with step = 1.
    RDD<Object>
    sliding(int windowSize, int step)
    Returns an RDD from grouping items of its parent RDD in fixed size blocks by passing a sliding window over them.

    Methods inherited from class java.lang.Object

    equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • RDDFunctions

      public RDDFunctions(RDD<T> self, scala.reflect.ClassTag<T> evidence$1)

  • Method Details

    • fromRDD

      public static <T> RDDFunctions<T> fromRDD(RDD<T> rdd, scala.reflect.ClassTag<T> evidence$2)
      Implicit conversion from an RDD to RDDFunctions.

    • sliding

      public RDD<Object> sliding(int windowSize, int step)
      Returns an RDD from grouping items of its parent RDD in fixed size blocks by passing a sliding window over them. The ordering is first based on the partition index and then the ordering of items within each partition. This is similar to sliding in Scala collections, except that it becomes an empty RDD if the window size is greater than the total number of items. It needs to trigger a Spark job if the parent RDD has more than one partitions and the window size is greater than 1.
      Parameters:
      windowSize - (undocumented)
      step - (undocumented)
      Returns:
      (undocumented)

    • sliding

      public RDD<Object> sliding(int windowSize)
      sliding(Int, Int)* with step = 1.
      Parameters:
      windowSize - (undocumented)
      Returns:
      (undocumented)