Package org.apache.spark.paths

Class SparkPath

Object
org.apache.spark.paths.SparkPath
All Implemented Interfaces:
Serializable, scala.Equals, scala.Product, scala.Serializable
public class SparkPath extends Object implements scala.Product, scala.Serializable
A canonical representation of a file path. This class is intended to provide type-safety to the way that Spark handles Paths. Paths can be represented as Strings in multiple ways, which are not always compatible. Spark regularly uses two ways: 1. hadoop Path.toString and java URI.toString.
See Also:

  • Constructor Details

    • SparkPath

      public SparkPath()

  • Method Details

    • fromPathString

      public static SparkPath fromPathString(String str)
      Creates a SparkPath from a hadoop Path string. Please be very sure that the provided string is encoded (or not encoded) in the right way.

      Please see the hadoop Path documentation here: https://hadoop.apache.org/docs/stable/api/org/apache/hadoop/fs/Path.html#Path-java.lang.String-

      Parameters:
      str - (undocumented)
      Returns:
      (undocumented)

    • fromPath

      public static SparkPath fromPath(org.apache.hadoop.fs.Path path)

    • fromFileStatus

      public static SparkPath fromFileStatus(org.apache.hadoop.fs.FileStatus fs)

    • fromUrlString

      public static SparkPath fromUrlString(String str)
      Creates a SparkPath from a url-encoded string. Note: It is the responsibility of the caller to ensure that str is a valid url-encoded string.
      Parameters:
      str - (undocumented)
      Returns:
      (undocumented)

    • fromUri

      public static SparkPath fromUri(URI uri)

    • urlEncoded

      public String urlEncoded()

    • toUri

      public URI toUri()

    • toPath

      public org.apache.hadoop.fs.Path toPath()

    • toString

      public String toString()
      Overrides:
      toString in class Object