write.stream {SparkR}R Documentation

Write the streaming SparkDataFrame to a data source.

Description

The data source is specified by the source and a set of options (...). If source is not specified, the default data source configured by spark.sql.sources.default will be used.

Usage

write.stream(df, source = NULL, outputMode = NULL, ...)

## S4 method for signature 'SparkDataFrame'
write.stream(df, source = NULL,
  outputMode = NULL, ...)

Arguments

df

a streaming SparkDataFrame.

source

a name for external data source.

outputMode

one of 'append', 'complete', 'update'.

...

additional argument(s) passed to the method.

Details

Additionally, outputMode specifies how data of a streaming SparkDataFrame is written to a output data source. There are three modes:

Note

write.stream since 2.2.0

experimental

See Also

read.stream

Other SparkDataFrame functions: SparkDataFrame-class, agg, arrange, as.data.frame, attach,SparkDataFrame-method, cache, checkpoint, coalesce, collect, colnames, coltypes, createOrReplaceTempView, crossJoin, dapplyCollect, dapply, describe, dim, distinct, dropDuplicates, dropna, drop, dtypes, except, explain, filter, first, gapplyCollect, gapply, getNumPartitions, group_by, head, hint, histogram, insertInto, intersect, isLocal, isStreaming, join, limit, merge, mutate, ncol, nrow, persist, printSchema, randomSplit, rbind, registerTempTable, rename, repartition, sample, saveAsTable, schema, selectExpr, select, showDF, show, storageLevel, str, subset, take, toJSON, union, unpersist, withColumn, with, write.df, write.jdbc, write.json, write.orc, write.parquet, write.text

Examples

## Not run: 
##D sparkR.session()
##D df <- read.stream("socket", host = "localhost", port = 9999)
##D isStreaming(df)
##D wordCounts <- count(group_by(df, "value"))
##D 
##D # console
##D q <- write.stream(wordCounts, "console", outputMode = "complete")
##D # text stream
##D q <- write.stream(df, "text", path = "/home/user/out", checkpointLocation = "/home/user/cp")
##D # memory stream
##D q <- write.stream(wordCounts, "memory", queryName = "outs", outputMode = "complete")
##D head(sql("SELECT * from outs"))
##D queryName(q)
##D 
##D stopQuery(q)
## End(Not run)

[Package SparkR version 2.2.3 Index]