Package org.apache.spark.sql.avro

Class functions

Object

org.apache.spark.sql.avro.functions

public class functions
extends Object

Constructor Summary

Constructors

Constructor	Description
functions()

Method Summary

Modifier and Type	Method	Description
static Column	from_avro(Column data, String jsonFormatSchema)	Converts a binary column of avro format into its corresponding catalyst value.
static Column	from_avro(Column data, String jsonFormatSchema, Map<String,String> options)	Converts a binary column of Avro format into its corresponding catalyst value.
static Column	to_avro(Column data)	Converts a column into binary of avro format.
static Column	to_avro(Column data, String jsonFormatSchema)	Converts a column into binary of avro format.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

functions

public functions()

Method Details

from_avro

public static Column from_avro(Column data,
 String jsonFormatSchema)

Converts a binary column of avro format into its corresponding catalyst value. The specified schema must match the read data, otherwise the behavior is undefined: it may fail or return arbitrary result.

Parameters:

data - the binary column.

jsonFormatSchema - the avro schema in JSON string format.

Returns:

(undocumented)

Since:

3.0.0

from_avro

public static Column from_avro(Column data,
 String jsonFormatSchema,
 Map<String,String> options)

Converts a binary column of Avro format into its corresponding catalyst value. The specified schema must match actual schema of the read data, otherwise the behavior is undefined: it may fail or return arbitrary result. To deserialize the data with a compatible and evolved schema, the expected Avro schema can be set via the option avroSchema.

Parameters:

data - the binary column.

jsonFormatSchema - the avro schema in JSON string format.

options - options to control how the Avro record is parsed.

Returns:

(undocumented)

Since:

3.0.0

to_avro

public static Column to_avro(Column data)

Converts a column into binary of avro format.

Parameters:

data - the data column.

Returns:

(undocumented)

Since:

3.0.0

to_avro

public static Column to_avro(Column data,
 String jsonFormatSchema)

Converts a column into binary of avro format.

Parameters:

data - the data column.

jsonFormatSchema - user-specified output avro schema in JSON string format.

Returns:

(undocumented)

Since:

3.0.0