pyspark.sql.functions.to_timestamp¶

pyspark.sql.functions.to_timestamp(col: ColumnOrName, format: Optional[str] = None) → pyspark.sql.column.Column[source]¶

Converts a Column into pyspark.sql.types.TimestampType using the optionally specified format. Specify formats according to datetime pattern. By default, it follows casting rules to pyspark.sql.types.TimestampType if the format is omitted. Equivalent to col.cast("timestamp").

New in version 2.2.0.

Changed in version 3.4.0: Supports Spark Connect.

Parameters

colColumn or str: column values to convert.
format: str, optional: format to use to convert timestamp values.

Returns

Column: timestamp value as pyspark.sql.types.TimestampType type.

Examples

>>> df = spark.createDataFrame([('1997-02-28 10:30:00',)], ['t'])
>>> df.select(to_timestamp(df.t).alias('dt')).collect()
[Row(dt=datetime.datetime(1997, 2, 28, 10, 30))]

>>> df = spark.createDataFrame([('1997-02-28 10:30:00',)], ['t'])
>>> df.select(to_timestamp(df.t, 'yyyy-MM-dd HH:mm:ss').alias('dt')).collect()
[Row(dt=datetime.datetime(1997, 2, 28, 10, 30))]

pyspark.sql.functions.unix_timestamp pyspark.sql.functions.to_date