package columnar
Type Members
- trait CachedBatch extends AnyRef
Basic interface that all cached batches of data must support.
Basic interface that all cached batches of data must support. This is primarily to allow for metrics to be handled outside of the encoding and decoding steps in a standard way.
- Annotations
- @DeveloperApi() @Since("3.1.0")
- trait CachedBatchSerializer extends Serializable
Provides APIs that handle transformations of SQL data associated with the cache/persist APIs.
Provides APIs that handle transformations of SQL data associated with the cache/persist APIs.
- Annotations
- @DeveloperApi() @Since("3.1.0")
- trait SimpleMetricsCachedBatch extends CachedBatch
A CachedBatch that stores some simple metrics that can be used for filtering of batches with the SimpleMetricsCachedBatchSerializer.
A CachedBatch that stores some simple metrics that can be used for filtering of batches with the SimpleMetricsCachedBatchSerializer. The metrics are returned by the stats value. For each column in the batch 5 columns of metadata are needed in the row.
- Annotations
- @DeveloperApi() @Since("3.1.0")
- abstract class SimpleMetricsCachedBatchSerializer extends CachedBatchSerializer with Logging
Provides basic filtering for CachedBatchSerializer implementations.
Provides basic filtering for CachedBatchSerializer implementations. The requirement to extend this is that all of the batches produced by your serializer are instances of SimpleMetricsCachedBatch. This does not calculate the metrics needed to be stored in the batches. That is up to each implementation. The metrics required are really just min and max values and those are optional especially for complex types. Because those metrics are simple and it is likely that compression will also be done on the data we thought it best to let each implementation decide on the most efficient way to calculate the metrics, possibly combining them with compression passes that might also be done across the data.
- Annotations
- @DeveloperApi() @Since("3.1.0")