Class SynapseSparkJobDefinitionActivity
java.lang.Object
com.azure.analytics.synapse.artifacts.models.Activity
com.azure.analytics.synapse.artifacts.models.ExecutionActivity
com.azure.analytics.synapse.artifacts.models.SynapseSparkJobDefinitionActivity
Execute spark job activity.
-
Constructor Summary
ConstructorDescriptionCreates an instance of SynapseSparkJobDefinitionActivity class. -
Method Summary
Modifier and TypeMethodDescriptionGet the arguments property: User specified arguments to SynapseSparkJobDefinitionActivity.Get the className property: The fully-qualified identifier or the main class that is in the main definition file, which will override the 'className' of the spark job definition you provide.getConf()
Get the conf property: Spark configuration properties, which will override the 'conf' of the spark job definition you provide.Get the configurationType property: The type of the spark config.Get the driverSize property: Number of core and memory to be used for driver allocated in the specified Spark pool for the job, which will be used for overriding 'driverCores' and 'driverMemory' of the spark job definition you provide.Get the executorSize property: Number of core and memory to be used for executors allocated in the specified Spark pool for the job, which will be used for overriding 'executorCores' and 'executorMemory' of the spark job definition you provide.getFile()
Get the file property: The main file used for the job, which will override the 'file' of the spark job definition you provide.getFiles()
Get the files property: (Deprecated.Get the filesV2 property: Additional files used for reference in the main definition file, which will override the 'jars' and 'files' of the spark job definition you provide.Get the numExecutors property: Number of executors to launch for this job, which will override the 'numExecutors' of the spark job definition you provide.Get the pythonCodeReference property: Additional python code files used for reference in the main definition file, which will override the 'pyFiles' of the spark job definition you provide.Get the scanFolder property: Scanning subfolders from the root folder of the main definition file, these files will be added as reference files.Get the sparkConfig property: Spark configuration property.Get the sparkJob property: Synapse spark job reference.Get the targetBigDataPool property: The name of the big data pool which will be used to execute the spark batch job, which will override the 'targetBigDataPool' of the spark job definition you provide.Get the targetSparkConfiguration property: The spark configuration of the spark job.setArguments
(List<Object> arguments) Set the arguments property: User specified arguments to SynapseSparkJobDefinitionActivity.setClassName
(Object className) Set the className property: The fully-qualified identifier or the main class that is in the main definition file, which will override the 'className' of the spark job definition you provide.Set the conf property: Spark configuration properties, which will override the 'conf' of the spark job definition you provide.setConfigurationType
(ConfigurationType configurationType) Set the configurationType property: The type of the spark config.setDependsOn
(List<ActivityDependency> dependsOn) Set the dependsOn property: Activity depends on condition.setDescription
(String description) Set the description property: Activity description.setDriverSize
(Object driverSize) Set the driverSize property: Number of core and memory to be used for driver allocated in the specified Spark pool for the job, which will be used for overriding 'driverCores' and 'driverMemory' of the spark job definition you provide.setExecutorSize
(Object executorSize) Set the executorSize property: Number of core and memory to be used for executors allocated in the specified Spark pool for the job, which will be used for overriding 'executorCores' and 'executorMemory' of the spark job definition you provide.Set the file property: The main file used for the job, which will override the 'file' of the spark job definition you provide.Set the files property: (Deprecated.setFilesV2
(List<Object> filesV2) Set the filesV2 property: Additional files used for reference in the main definition file, which will override the 'jars' and 'files' of the spark job definition you provide.setLinkedServiceName
(LinkedServiceReference linkedServiceName) Set the linkedServiceName property: Linked service reference.Set the name property: Activity name.setNumExecutors
(Object numExecutors) Set the numExecutors property: Number of executors to launch for this job, which will override the 'numExecutors' of the spark job definition you provide.setPolicy
(ActivityPolicy policy) Set the policy property: Activity policy.setPythonCodeReference
(List<Object> pythonCodeReference) Set the pythonCodeReference property: Additional python code files used for reference in the main definition file, which will override the 'pyFiles' of the spark job definition you provide.setScanFolder
(Object scanFolder) Set the scanFolder property: Scanning subfolders from the root folder of the main definition file, these files will be added as reference files.setSparkConfig
(Map<String, Object> sparkConfig) Set the sparkConfig property: Spark configuration property.setSparkJob
(SynapseSparkJobReference sparkJob) Set the sparkJob property: Synapse spark job reference.setTargetBigDataPool
(BigDataPoolParametrizationReference targetBigDataPool) Set the targetBigDataPool property: The name of the big data pool which will be used to execute the spark batch job, which will override the 'targetBigDataPool' of the spark job definition you provide.setTargetSparkConfiguration
(SparkConfigurationParametrizationReference targetSparkConfiguration) Set the targetSparkConfiguration property: The spark configuration of the spark job.setUserProperties
(List<UserProperty> userProperties) Set the userProperties property: Activity user properties.Methods inherited from class com.azure.analytics.synapse.artifacts.models.ExecutionActivity
getLinkedServiceName, getPolicy
Methods inherited from class com.azure.analytics.synapse.artifacts.models.Activity
getAdditionalProperties, getDependsOn, getDescription, getName, getUserProperties, setAdditionalProperties
-
Constructor Details
-
SynapseSparkJobDefinitionActivity
public SynapseSparkJobDefinitionActivity()Creates an instance of SynapseSparkJobDefinitionActivity class.
-
-
Method Details
-
getSparkJob
Get the sparkJob property: Synapse spark job reference.- Returns:
- the sparkJob value.
-
setSparkJob
Set the sparkJob property: Synapse spark job reference.- Parameters:
sparkJob
- the sparkJob value to set.- Returns:
- the SynapseSparkJobDefinitionActivity object itself.
-
getArguments
Get the arguments property: User specified arguments to SynapseSparkJobDefinitionActivity.- Returns:
- the arguments value.
-
setArguments
Set the arguments property: User specified arguments to SynapseSparkJobDefinitionActivity.- Parameters:
arguments
- the arguments value to set.- Returns:
- the SynapseSparkJobDefinitionActivity object itself.
-
getFile
Get the file property: The main file used for the job, which will override the 'file' of the spark job definition you provide. Type: string (or Expression with resultType string).- Returns:
- the file value.
-
setFile
Set the file property: The main file used for the job, which will override the 'file' of the spark job definition you provide. Type: string (or Expression with resultType string).- Parameters:
file
- the file value to set.- Returns:
- the SynapseSparkJobDefinitionActivity object itself.
-
getScanFolder
Get the scanFolder property: Scanning subfolders from the root folder of the main definition file, these files will be added as reference files. The folders named 'jars', 'pyFiles', 'files' or 'archives' will be scanned, and the folders name are case sensitive. Type: boolean (or Expression with resultType boolean).- Returns:
- the scanFolder value.
-
setScanFolder
Set the scanFolder property: Scanning subfolders from the root folder of the main definition file, these files will be added as reference files. The folders named 'jars', 'pyFiles', 'files' or 'archives' will be scanned, and the folders name are case sensitive. Type: boolean (or Expression with resultType boolean).- Parameters:
scanFolder
- the scanFolder value to set.- Returns:
- the SynapseSparkJobDefinitionActivity object itself.
-
getClassName
Get the className property: The fully-qualified identifier or the main class that is in the main definition file, which will override the 'className' of the spark job definition you provide. Type: string (or Expression with resultType string).- Returns:
- the className value.
-
setClassName
Set the className property: The fully-qualified identifier or the main class that is in the main definition file, which will override the 'className' of the spark job definition you provide. Type: string (or Expression with resultType string).- Parameters:
className
- the className value to set.- Returns:
- the SynapseSparkJobDefinitionActivity object itself.
-
getFiles
Get the files property: (Deprecated. Please use pythonCodeReference and filesV2) Additional files used for reference in the main definition file, which will override the 'files' of the spark job definition you provide.- Returns:
- the files value.
-
setFiles
Set the files property: (Deprecated. Please use pythonCodeReference and filesV2) Additional files used for reference in the main definition file, which will override the 'files' of the spark job definition you provide.- Parameters:
files
- the files value to set.- Returns:
- the SynapseSparkJobDefinitionActivity object itself.
-
getPythonCodeReference
Get the pythonCodeReference property: Additional python code files used for reference in the main definition file, which will override the 'pyFiles' of the spark job definition you provide.- Returns:
- the pythonCodeReference value.
-
setPythonCodeReference
Set the pythonCodeReference property: Additional python code files used for reference in the main definition file, which will override the 'pyFiles' of the spark job definition you provide.- Parameters:
pythonCodeReference
- the pythonCodeReference value to set.- Returns:
- the SynapseSparkJobDefinitionActivity object itself.
-
getFilesV2
Get the filesV2 property: Additional files used for reference in the main definition file, which will override the 'jars' and 'files' of the spark job definition you provide.- Returns:
- the filesV2 value.
-
setFilesV2
Set the filesV2 property: Additional files used for reference in the main definition file, which will override the 'jars' and 'files' of the spark job definition you provide.- Parameters:
filesV2
- the filesV2 value to set.- Returns:
- the SynapseSparkJobDefinitionActivity object itself.
-
getTargetBigDataPool
Get the targetBigDataPool property: The name of the big data pool which will be used to execute the spark batch job, which will override the 'targetBigDataPool' of the spark job definition you provide.- Returns:
- the targetBigDataPool value.
-
setTargetBigDataPool
public SynapseSparkJobDefinitionActivity setTargetBigDataPool(BigDataPoolParametrizationReference targetBigDataPool) Set the targetBigDataPool property: The name of the big data pool which will be used to execute the spark batch job, which will override the 'targetBigDataPool' of the spark job definition you provide.- Parameters:
targetBigDataPool
- the targetBigDataPool value to set.- Returns:
- the SynapseSparkJobDefinitionActivity object itself.
-
getExecutorSize
Get the executorSize property: Number of core and memory to be used for executors allocated in the specified Spark pool for the job, which will be used for overriding 'executorCores' and 'executorMemory' of the spark job definition you provide. Type: string (or Expression with resultType string).- Returns:
- the executorSize value.
-
setExecutorSize
Set the executorSize property: Number of core and memory to be used for executors allocated in the specified Spark pool for the job, which will be used for overriding 'executorCores' and 'executorMemory' of the spark job definition you provide. Type: string (or Expression with resultType string).- Parameters:
executorSize
- the executorSize value to set.- Returns:
- the SynapseSparkJobDefinitionActivity object itself.
-
getConf
Get the conf property: Spark configuration properties, which will override the 'conf' of the spark job definition you provide.- Returns:
- the conf value.
-
setConf
Set the conf property: Spark configuration properties, which will override the 'conf' of the spark job definition you provide.- Parameters:
conf
- the conf value to set.- Returns:
- the SynapseSparkJobDefinitionActivity object itself.
-
getDriverSize
Get the driverSize property: Number of core and memory to be used for driver allocated in the specified Spark pool for the job, which will be used for overriding 'driverCores' and 'driverMemory' of the spark job definition you provide. Type: string (or Expression with resultType string).- Returns:
- the driverSize value.
-
setDriverSize
Set the driverSize property: Number of core and memory to be used for driver allocated in the specified Spark pool for the job, which will be used for overriding 'driverCores' and 'driverMemory' of the spark job definition you provide. Type: string (or Expression with resultType string).- Parameters:
driverSize
- the driverSize value to set.- Returns:
- the SynapseSparkJobDefinitionActivity object itself.
-
getNumExecutors
Get the numExecutors property: Number of executors to launch for this job, which will override the 'numExecutors' of the spark job definition you provide. Type: integer (or Expression with resultType integer).- Returns:
- the numExecutors value.
-
setNumExecutors
Set the numExecutors property: Number of executors to launch for this job, which will override the 'numExecutors' of the spark job definition you provide. Type: integer (or Expression with resultType integer).- Parameters:
numExecutors
- the numExecutors value to set.- Returns:
- the SynapseSparkJobDefinitionActivity object itself.
-
getConfigurationType
Get the configurationType property: The type of the spark config.- Returns:
- the configurationType value.
-
setConfigurationType
Set the configurationType property: The type of the spark config.- Parameters:
configurationType
- the configurationType value to set.- Returns:
- the SynapseSparkJobDefinitionActivity object itself.
-
getTargetSparkConfiguration
Get the targetSparkConfiguration property: The spark configuration of the spark job.- Returns:
- the targetSparkConfiguration value.
-
setTargetSparkConfiguration
public SynapseSparkJobDefinitionActivity setTargetSparkConfiguration(SparkConfigurationParametrizationReference targetSparkConfiguration) Set the targetSparkConfiguration property: The spark configuration of the spark job.- Parameters:
targetSparkConfiguration
- the targetSparkConfiguration value to set.- Returns:
- the SynapseSparkJobDefinitionActivity object itself.
-
getSparkConfig
Get the sparkConfig property: Spark configuration property.- Returns:
- the sparkConfig value.
-
setSparkConfig
Set the sparkConfig property: Spark configuration property.- Parameters:
sparkConfig
- the sparkConfig value to set.- Returns:
- the SynapseSparkJobDefinitionActivity object itself.
-
setLinkedServiceName
public SynapseSparkJobDefinitionActivity setLinkedServiceName(LinkedServiceReference linkedServiceName) Set the linkedServiceName property: Linked service reference.- Overrides:
setLinkedServiceName
in classExecutionActivity
- Parameters:
linkedServiceName
- the linkedServiceName value to set.- Returns:
- the ExecutionActivity object itself.
-
setPolicy
Set the policy property: Activity policy.- Overrides:
setPolicy
in classExecutionActivity
- Parameters:
policy
- the policy value to set.- Returns:
- the ExecutionActivity object itself.
-
setName
Set the name property: Activity name.- Overrides:
setName
in classExecutionActivity
- Parameters:
name
- the name value to set.- Returns:
- the Activity object itself.
-
setDescription
Set the description property: Activity description.- Overrides:
setDescription
in classExecutionActivity
- Parameters:
description
- the description value to set.- Returns:
- the Activity object itself.
-
setDependsOn
Set the dependsOn property: Activity depends on condition.- Overrides:
setDependsOn
in classExecutionActivity
- Parameters:
dependsOn
- the dependsOn value to set.- Returns:
- the Activity object itself.
-
setUserProperties
Set the userProperties property: Activity user properties.- Overrides:
setUserProperties
in classExecutionActivity
- Parameters:
userProperties
- the userProperties value to set.- Returns:
- the Activity object itself.
-