public class ExecutorResourceRequests
extends Object
implements scala.Serializable
Constructor and Description |
---|
ExecutorResourceRequests() |
Modifier and Type | Method and Description |
---|---|
ExecutorResourceRequests |
cores(int amount)
Specify number of cores per Executor.
|
ExecutorResourceRequests |
memory(String amount)
Specify heap memory.
|
ExecutorResourceRequests |
memoryOverhead(String amount)
Specify overhead memory.
|
ExecutorResourceRequests |
offHeapMemory(String amount)
Specify off heap memory.
|
ExecutorResourceRequests |
pysparkMemory(String amount)
Specify pyspark memory.
|
scala.collection.immutable.Map<String,ExecutorResourceRequest> |
requests()
Returns all the resource requests for the executor.
|
java.util.Map<String,ExecutorResourceRequest> |
requestsJMap()
(Java-specific) Returns all the resource requests for the executor.
|
ExecutorResourceRequests |
resource(String resourceName,
long amount,
String discoveryScript,
String vendor)
Amount of a particular custom resource(GPU, FPGA, etc) to use.
|
String |
toString() |
public ExecutorResourceRequests cores(int amount)
ExecutorResourceRequest
for "cores" resource.
amount
- Number of cores to allocate per Executor.public ExecutorResourceRequests memory(String amount)
ExecutorResourceRequest
for "memory" resource.
amount
- Amount of memory. In the same format as JVM memory strings (e.g. 512m, 2g).
Default unit is MiB if not specified.public ExecutorResourceRequests memoryOverhead(String amount)
ExecutorResourceRequest
for "memoryOverhead" resource.
amount
- Amount of memory. In the same format as JVM memory strings (e.g. 512m, 2g).
Default unit is MiB if not specified.public ExecutorResourceRequests offHeapMemory(String amount)
ExecutorResourceRequest
for "offHeap" resource.
amount
- Amount of memory. In the same format as JVM memory strings (e.g. 512m, 2g).
Default unit is MiB if not specified.public ExecutorResourceRequests pysparkMemory(String amount)
ExecutorResourceRequest
for "pyspark.memory" resource.
amount
- Amount of memory. In the same format as JVM memory strings (e.g. 512m, 2g).
Default unit is MiB if not specified.public scala.collection.immutable.Map<String,ExecutorResourceRequest> requests()
public java.util.Map<String,ExecutorResourceRequest> requestsJMap()
public ExecutorResourceRequests resource(String resourceName, long amount, String discoveryScript, String vendor)
ExecutorResourceRequest
for custom resources.
resourceName
- Name of the resource.amount
- amount of that resource per executor to use.discoveryScript
- Optional script used to discover the resources. This is required on
some cluster managers that don't tell Spark the addresses of
the resources allocated. The script runs on Executors startup to
of the resources available.vendor
- Optional vendor, required for some cluster managerspublic String toString()
toString
in class Object