|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
Object org.apache.spark.sql.types.DataType org.apache.spark.sql.types.UserDefinedType<UserType>
public abstract class UserDefinedType<UserType>
::DeveloperApi:: The data type for User Defined Types (UDTs).
This interface allows a user to make their own classes more interoperable with SparkSQL;
e.g., by creating a UserDefinedType
for a class X, it becomes possible to create
a DataFrame
which has class X in the schema.
For SparkSQL to recognize UDTs, the UDT must be annotated with
SQLUserDefinedType
.
The conversion via serialize
occurs when instantiating a DataFrame
from another RDD.
The conversion via deserialize
occurs when reading from a DataFrame
.
Constructor Summary | |
---|---|
UserDefinedType()
|
Method Summary | |
---|---|
int |
defaultSize()
The default size of a value of the UserDefinedType is 4096 bytes. |
abstract UserType |
deserialize(Object datum)
Convert a SQL datum to the user type |
String |
pyUDT()
Paired Python UDT class, if exists. |
abstract Object |
serialize(Object obj)
Convert the user type to a SQL datum |
abstract DataType |
sqlType()
Underlying storage type for this UDT |
abstract Class<UserType> |
userClass()
Class object for the UserType |
Methods inherited from class org.apache.spark.sql.types.DataType |
---|
fromCaseClassString, fromJson, json, prettyJson, simpleString, typeName |
Methods inherited from class Object |
---|
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Constructor Detail |
---|
public UserDefinedType()
Method Detail |
---|
public abstract DataType sqlType()
public String pyUDT()
public abstract Object serialize(Object obj)
TODO: Can we make this take obj: UserType? The issue is in CatalystTypeConverters.convertToCatalyst, where we need to convert Any to UserType.
obj
- (undocumented)
public abstract UserType deserialize(Object datum)
public abstract Class<UserType> userClass()
public int defaultSize()
defaultSize
in class DataType
|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |