Package org.apache.spark.sql.types
Class UserDefinedType<UserType>
Object
org.apache.spark.sql.types.DataType
org.apache.spark.sql.types.UserDefinedType<UserType>
- All Implemented Interfaces:
Serializable
,scala.Serializable
- Direct Known Subclasses:
VectorUDT
The data type for User Defined Types (UDTs).
This interface allows a user to make their own classes more interoperable with SparkSQL;
e.g., by creating a UserDefinedType
for a class X, it becomes possible to create
a DataFrame
which has class X in the schema.
For SparkSQL to recognize UDTs, the UDT must be annotated with
SQLUserDefinedType
.
The conversion via serialize
occurs when instantiating a DataFrame
from another RDD.
The conversion via deserialize
occurs when reading from a DataFrame
.
- See Also:
-
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionString representation for the type saved in external catalogs.int
The default size of a value of this data type, used internally for size estimation.abstract UserType
deserialize
(Object datum) Convert a SQL datum to the user typeboolean
int
hashCode()
pyUDT()
Paired Python UDT class, if exists.abstract Object
Convert the user type to a SQL datumSerialized Python UDT class, if exists.sql()
abstract DataType
sqlType()
Underlying storage type for this UDTClass object for the UserTypeMethods inherited from class org.apache.spark.sql.types.DataType
equalsIgnoreCaseAndNullability, equalsIgnoreNullability, equalsStructurally, equalsStructurallyByName, fromDDL, fromJson, json, parseTypeWithFallback, prettyJson, simpleString, typeName
-
Constructor Details
-
UserDefinedType
public UserDefinedType()
-
-
Method Details
-
sqlType
Underlying storage type for this UDT -
pyUDT
Paired Python UDT class, if exists. -
serializedPyClass
Serialized Python UDT class, if exists. -
serialize
Convert the user type to a SQL datum- Parameters:
obj
- (undocumented)- Returns:
- (undocumented)
-
deserialize
Convert a SQL datum to the user type -
userClass
Class object for the UserType- Returns:
- (undocumented)
-
defaultSize
public int defaultSize()Description copied from class:DataType
The default size of a value of this data type, used internally for size estimation.- Specified by:
defaultSize
in classDataType
- Returns:
- (undocumented)
-
sql
-
hashCode
public int hashCode() -
equals
-
catalogString
Description copied from class:DataType
String representation for the type saved in external catalogs.- Overrides:
catalogString
in classDataType
-