public class StructType extends DataType implements scala.collection.Seq<StructField>, scala.Product, scala.Serializable
StructType object can be constructed by
StructType(fields: Seq[StructField])
StructType object, one or multiple StructFields can be extracted by names.
If multiple StructFields are extracted, a StructType object will be returned.
If a provided name does not have a matching field, it will be ignored. For the case
of extracting a single StructField, a null will be returned.
Scala Example:
import org.apache.spark.sql._
import org.apache.spark.sql.types._
val struct =
StructType(
StructField("a", IntegerType, true) ::
StructField("b", LongType, false) ::
StructField("c", BooleanType, false) :: Nil)
// Extract a single StructField.
val singleField = struct("b")
// singleField: StructField = StructField(b,LongType,false)
// If this struct does not have a field called "d", it throws an exception.
struct("d")
// java.lang.IllegalArgumentException: d does not exist.
// ...
// Extract multiple StructFields. Field names are provided in a set.
// A StructType object will be returned.
val twoFields = struct(Set("b", "c"))
// twoFields: StructType =
// StructType(StructField(b,LongType,false), StructField(c,BooleanType,false))
// Any names without matching fields will throw an exception.
// For the case shown below, an exception is thrown due to "d".
struct(Set("b", "c", "d"))
// java.lang.IllegalArgumentException: d does not exist.
// ...
A Row object is used as a value of the StructType.
Scala Example:
import org.apache.spark.sql._
import org.apache.spark.sql.types._
val innerStruct =
StructType(
StructField("f1", IntegerType, true) ::
StructField("f2", LongType, false) ::
StructField("f3", BooleanType, false) :: Nil)
val struct = StructType(
StructField("a", innerStruct, true) :: Nil)
// Create a Row with the schema defined by struct
val row = Row(Row(1, 2, true))
scala.collection.TraversableLike.WithFilterscala.collection.TraversableOnce.BufferedCanBuildFrom<A,CC extends scala.collection.TraversableOnce<Object>>, scala.collection.TraversableOnce.FlattenOps<A>, scala.collection.TraversableOnce.ForceImplicitAmbiguity, scala.collection.TraversableOnce.MonadOps<A>, scala.collection.TraversableOnce.OnceCanBuildFrom<A>| Constructor and Description |
|---|
StructType()
No-arg constructor for kryo.
|
StructType(StructField[] fields) |
| Modifier and Type | Method and Description |
|---|---|
StructType |
add(String name,
DataType dataType)
Creates a new
StructType by adding a new nullable field with no metadata. |
StructType |
add(String name,
DataType dataType,
boolean nullable)
Creates a new
StructType by adding a new field with no metadata. |
StructType |
add(String name,
DataType dataType,
boolean nullable,
Metadata metadata)
Creates a new
StructType by adding a new field and specifying metadata. |
StructType |
add(String name,
DataType dataType,
boolean nullable,
String comment)
Creates a new
StructType by adding a new field and specifying metadata. |
StructType |
add(String name,
String dataType)
Creates a new
StructType by adding a new nullable field with no metadata where the
dataType is specified as a String. |
StructType |
add(String name,
String dataType,
boolean nullable)
Creates a new
StructType by adding a new field with no metadata where the
dataType is specified as a String. |
StructType |
add(String name,
String dataType,
boolean nullable,
Metadata metadata)
Creates a new
StructType by adding a new field and specifying metadata where the
dataType is specified as a String. |
StructType |
add(String name,
String dataType,
boolean nullable,
String comment)
Creates a new
StructType by adding a new field and specifying metadata where the
dataType is specified as a String. |
StructType |
add(StructField field)
Creates a new
StructType by adding a new field. |
StructField |
apply(int fieldIndex) |
StructType |
apply(scala.collection.immutable.Set<String> names)
Returns a
StructType containing StructFields of the given names, preserving the
original order of fields. |
StructField |
apply(String name)
Extracts the
StructField with the given name. |
String |
catalogString()
String representation for the type saved in external catalogs.
|
int |
defaultSize()
The default size of a value of the StructType is the total default sizes of all field types.
|
boolean |
equals(Object that) |
int |
fieldIndex(String name)
Returns the index of a given field.
|
String[] |
fieldNames()
Returns all field names in an array.
|
StructField[] |
fields() |
static scala.Option<StructType> |
findMissingFields(StructType source,
StructType target,
scala.Function2<String,String,Object> resolver)
Returns a
StructType that contains missing fields recursively from source to target. |
static StructType |
fromDDL(String ddl)
Creates StructType for a given DDL-formatted string, which is a comma separated list of field
definitions, e.g., a INT, b STRING.
|
int |
hashCode() |
scala.collection.Iterator<StructField> |
iterator() |
int |
length() |
String[] |
names()
Returns all field names in an array.
|
void |
printTreeString() |
String |
simpleString()
Readable string representation for the type.
|
String |
sql() |
String |
toDDL()
Returns a string containing a schema in DDL format.
|
String |
toString() |
String |
treeString() |
String |
treeString(int maxDepth) |
equalsIgnoreCaseAndNullability, equalsIgnoreNullability, equalsStructurally, equalsStructurallyByName, fromJson, json, parseTypeWithFallback, prettyJson, typeName$init$, canBuildFrom, companion, concat, empty, fill, fill, fill, fill, fill, iterate, range, range, ReusableCBF, seq, tabulate, tabulate, tabulate, tabulate, tabulate, unapplySeq$init$, andThen, applyOrElse, cond, condOpt, isDefinedAt, lift, orElse, runWith$init$, apply, apply$mcDD$sp, apply$mcDF$sp, apply$mcDI$sp, apply$mcDJ$sp, apply$mcFD$sp, apply$mcFF$sp, apply$mcFI$sp, apply$mcFJ$sp, apply$mcID$sp, apply$mcIF$sp, apply$mcII$sp, apply$mcIJ$sp, apply$mcJD$sp, apply$mcJF$sp, apply$mcJI$sp, apply$mcJJ$sp, apply$mcVD$sp, apply$mcVF$sp, apply$mcVI$sp, apply$mcVJ$sp, apply$mcZD$sp, apply$mcZF$sp, apply$mcZI$sp, apply$mcZJ$sp, compose$init$, apply, concat, fill, fill, fill, fill, fill, iterate, range, range, tabulate, tabulate, tabulate, tabulate, tabulate$init$, flatten, foreach, genericBuilder, head, isEmpty, transpose, unzip, unzip3$colon$plus, $init$, $plus$colon, combinations, contains, containsSlice, corresponds, diff, distinct, endsWith, indexOfSlice, indexOfSlice, indexWhere, indices, intersect, isEmpty, lastIndexOfSlice, lastIndexOfSlice, lastIndexWhere, lengthCompare, padTo, parCombiner, patch, permutations, reverse, reverseIterator, reverseMap, segmentLength, size, sortBy, sorted, sortWith, startsWith, thisCollection, toCollection, toSeq, union, updated, view, view$init$, canEqual, copyToArray, drop, dropRight, exists, find, foldRight, forall, foreach, grouped, head, reduceRight, sameElements, slice, sliding, sliding, take, takeRight, takeWhile, toIterable, toIterator, toStream, zip, zipAll, zipWithIndex$init$, $plus$plus, $plus$plus$colon, $plus$plus$colon, collect, dropWhile, filter, filterImpl, filterNot, flatMap, groupBy, hasDefiniteSize, headOption, init, inits, isTraversableAgain, last, lastOption, map, newBuilder, partition, repr, scan, scanLeft, scanRight, sliceWithKnownBound, sliceWithKnownDelta, span, splitAt, stringPrefix, tail, tails, to, toTraversable, withFilter$colon$bslash, $div$colon, $init$, addString, addString, addString, aggregate, alternateImplicit, collectFirst, copyToArray, copyToArray, copyToBuffer, count, flattenTraversableOnce, fold, foldLeft, max, maxBy, min, minBy, mkString, mkString, mkString, MonadOps, nonEmpty, OnceCanBuildFrom, product, reduce, reduceLeft, reduceLeftOption, reduceOption, reduceRightOption, reversed, sum, toArray, toBuffer, toIndexedSeq, toList, toMap, toSet, toVectorpublic StructType(StructField[] fields)
public StructType()
public static StructType fromDDL(String ddl)
ddl - (undocumented)public static scala.Option<StructType> findMissingFields(StructType source, StructType target, scala.Function2<String,String,Object> resolver)
StructType that contains missing fields recursively from source to target.
Note that this doesn't support looking into array type and map type recursively.source - (undocumented)target - (undocumented)resolver - (undocumented)public StructField[] fields()
public String[] fieldNames()
public String[] names()
fieldNames.
public boolean equals(Object that)
equals in interface scala.collection.GenSeqLike<StructField,scala.collection.Seq<StructField>>equals in interface scala.Equalsequals in class Objectpublic String toString()
toString in interface scala.collection.SeqLike<StructField,scala.collection.Seq<StructField>>toString in interface scala.collection.TraversableLike<StructField,scala.collection.Seq<StructField>>toString in interface scala.Function1<Object,StructField>toString in class Objectpublic int hashCode()
hashCode in interface scala.collection.GenSeqLike<StructField,scala.collection.Seq<StructField>>hashCode in class Objectpublic StructType add(StructField field)
StructType by adding a new field.
val struct = (new StructType)
.add(StructField("a", IntegerType, true))
.add(StructField("b", LongType, false))
.add(StructField("c", StringType, true))
field - (undocumented)public StructType add(String name, DataType dataType)
StructType by adding a new nullable field with no metadata.
val struct = (new StructType) .add("a", IntegerType) .add("b", LongType) .add("c", StringType)
name - (undocumented)dataType - (undocumented)public StructType add(String name, DataType dataType, boolean nullable)
StructType by adding a new field with no metadata.
val struct = (new StructType) .add("a", IntegerType, true) .add("b", LongType, false) .add("c", StringType, true)
name - (undocumented)dataType - (undocumented)nullable - (undocumented)public StructType add(String name, DataType dataType, boolean nullable, Metadata metadata)
StructType by adding a new field and specifying metadata.
val struct = (new StructType)
.add("a", IntegerType, true, Metadata.empty)
.add("b", LongType, false, Metadata.empty)
.add("c", StringType, true, Metadata.empty)
name - (undocumented)dataType - (undocumented)nullable - (undocumented)metadata - (undocumented)public StructType add(String name, DataType dataType, boolean nullable, String comment)
StructType by adding a new field and specifying metadata.
val struct = (new StructType)
.add("a", IntegerType, true, "comment1")
.add("b", LongType, false, "comment2")
.add("c", StringType, true, "comment3")
name - (undocumented)dataType - (undocumented)nullable - (undocumented)comment - (undocumented)public StructType add(String name, String dataType)
StructType by adding a new nullable field with no metadata where the
dataType is specified as a String.
val struct = (new StructType)
.add("a", "int")
.add("b", "long")
.add("c", "string")
name - (undocumented)dataType - (undocumented)public StructType add(String name, String dataType, boolean nullable)
StructType by adding a new field with no metadata where the
dataType is specified as a String.
val struct = (new StructType)
.add("a", "int", true)
.add("b", "long", false)
.add("c", "string", true)
name - (undocumented)dataType - (undocumented)nullable - (undocumented)public StructType add(String name, String dataType, boolean nullable, Metadata metadata)
StructType by adding a new field and specifying metadata where the
dataType is specified as a String.
val struct = (new StructType)
.add("a", "int", true, Metadata.empty)
.add("b", "long", false, Metadata.empty)
.add("c", "string", true, Metadata.empty)
name - (undocumented)dataType - (undocumented)nullable - (undocumented)metadata - (undocumented)public StructType add(String name, String dataType, boolean nullable, String comment)
StructType by adding a new field and specifying metadata where the
dataType is specified as a String.
val struct = (new StructType)
.add("a", "int", true, "comment1")
.add("b", "long", false, "comment2")
.add("c", "string", true, "comment3")
name - (undocumented)dataType - (undocumented)nullable - (undocumented)comment - (undocumented)public StructField apply(String name)
StructField with the given name.
name - (undocumented)IllegalArgumentException - if a field with the given name does not existpublic StructType apply(scala.collection.immutable.Set<String> names)
StructType containing StructFields of the given names, preserving the
original order of fields.
names - (undocumented)IllegalArgumentException - if at least one given field name does not existpublic int fieldIndex(String name)
name - (undocumented)IllegalArgumentException - if a field with the given name does not existpublic String treeString()
public String treeString(int maxDepth)
public void printTreeString()
public StructField apply(int fieldIndex)
apply in interface scala.collection.GenSeqLike<StructField,scala.collection.Seq<StructField>>apply in interface scala.collection.SeqLike<StructField,scala.collection.Seq<StructField>>public int length()
length in interface scala.collection.GenSeqLike<StructField,scala.collection.Seq<StructField>>length in interface scala.collection.SeqLike<StructField,scala.collection.Seq<StructField>>public scala.collection.Iterator<StructField> iterator()
iterator in interface scala.collection.GenIterableLike<StructField,scala.collection.Seq<StructField>>iterator in interface scala.collection.IterableLike<StructField,scala.collection.Seq<StructField>>public int defaultSize()
defaultSize in class DataTypepublic String simpleString()
DataTypesimpleString in class DataTypepublic String catalogString()
DataTypecatalogString in class DataTypepublic String toDDL()
StructType(Seq(StructField("eventId", IntegerType), StructField("s", StringType)))
will be converted to eventId INT, s STRING.
The returned DDL schema can be used in a table creation.