Called once in the driver to bootstrap this module that is specific to this application.
This method is called before submitting executor requests to the cluster manager.
This method should prepare the module with its shuffle components i.e. registering against
an external file servers or shuffle services, or creating tables in a shuffle
storage data database.
additional SparkConf settings necessary for initializing the executor components.
This would include configurations that cannot be statically set on the application, like
the host:port of external services for shuffle storage.
Called once at the end of the Spark application to clean up any existing shuffle state.
default void registerShuffle(int shuffleId)
Called once per shuffle id when the shuffle id is first generated for a shuffle stage.
shuffleId - The unique identifier for the shuffle stage.