Package com.azure.resourcemanager.datafactory.fluent.models

Class SynapseSparkJobActivityTypeProperties

java.lang.Object
com.azure.resourcemanager.datafactory.fluent.models.SynapseSparkJobActivityTypeProperties
public final class SynapseSparkJobActivityTypeProperties extends Object
Execute spark job activity properties.

  • Constructor Details

    • SynapseSparkJobActivityTypeProperties

      public SynapseSparkJobActivityTypeProperties()
      Creates an instance of SynapseSparkJobActivityTypeProperties class.

  • Method Details

    • sparkJob

      public SynapseSparkJobReference sparkJob()
      Get the sparkJob property: Synapse spark job reference.
      Returns:
      the sparkJob value.

    • withSparkJob

      public SynapseSparkJobActivityTypeProperties withSparkJob(SynapseSparkJobReference sparkJob)
      Set the sparkJob property: Synapse spark job reference.
      Parameters:
      sparkJob - the sparkJob value to set.
      Returns:
      the SynapseSparkJobActivityTypeProperties object itself.

    • arguments

      public List<Object> arguments()
      Get the arguments property: User specified arguments to SynapseSparkJobDefinitionActivity.
      Returns:
      the arguments value.

    • withArguments

      public SynapseSparkJobActivityTypeProperties withArguments(List<Object> arguments)
      Set the arguments property: User specified arguments to SynapseSparkJobDefinitionActivity.
      Parameters:
      arguments - the arguments value to set.
      Returns:
      the SynapseSparkJobActivityTypeProperties object itself.

    • file

      public Object file()
      Get the file property: The main file used for the job, which will override the 'file' of the spark job definition you provide. Type: string (or Expression with resultType string).
      Returns:
      the file value.

    • withFile

      public SynapseSparkJobActivityTypeProperties withFile(Object file)
      Set the file property: The main file used for the job, which will override the 'file' of the spark job definition you provide. Type: string (or Expression with resultType string).
      Parameters:
      file - the file value to set.
      Returns:
      the SynapseSparkJobActivityTypeProperties object itself.

    • scanFolder

      public Object scanFolder()
      Get the scanFolder property: Scanning subfolders from the root folder of the main definition file, these files will be added as reference files. The folders named 'jars', 'pyFiles', 'files' or 'archives' will be scanned, and the folders name are case sensitive. Type: boolean (or Expression with resultType boolean).
      Returns:
      the scanFolder value.

    • withScanFolder

      public SynapseSparkJobActivityTypeProperties withScanFolder(Object scanFolder)
      Set the scanFolder property: Scanning subfolders from the root folder of the main definition file, these files will be added as reference files. The folders named 'jars', 'pyFiles', 'files' or 'archives' will be scanned, and the folders name are case sensitive. Type: boolean (or Expression with resultType boolean).
      Parameters:
      scanFolder - the scanFolder value to set.
      Returns:
      the SynapseSparkJobActivityTypeProperties object itself.

    • className

      public Object className()
      Get the className property: The fully-qualified identifier or the main class that is in the main definition file, which will override the 'className' of the spark job definition you provide. Type: string (or Expression with resultType string).
      Returns:
      the className value.

    • withClassName

      public SynapseSparkJobActivityTypeProperties withClassName(Object className)
      Set the className property: The fully-qualified identifier or the main class that is in the main definition file, which will override the 'className' of the spark job definition you provide. Type: string (or Expression with resultType string).
      Parameters:
      className - the className value to set.
      Returns:
      the SynapseSparkJobActivityTypeProperties object itself.

    • files

      public List<Object> files()
      Get the files property: (Deprecated. Please use pythonCodeReference and filesV2) Additional files used for reference in the main definition file, which will override the 'files' of the spark job definition you provide.
      Returns:
      the files value.

    • withFiles

      public SynapseSparkJobActivityTypeProperties withFiles(List<Object> files)
      Set the files property: (Deprecated. Please use pythonCodeReference and filesV2) Additional files used for reference in the main definition file, which will override the 'files' of the spark job definition you provide.
      Parameters:
      files - the files value to set.
      Returns:
      the SynapseSparkJobActivityTypeProperties object itself.

    • pythonCodeReference

      public List<Object> pythonCodeReference()
      Get the pythonCodeReference property: Additional python code files used for reference in the main definition file, which will override the 'pyFiles' of the spark job definition you provide.
      Returns:
      the pythonCodeReference value.

    • withPythonCodeReference

      public SynapseSparkJobActivityTypeProperties withPythonCodeReference(List<Object> pythonCodeReference)
      Set the pythonCodeReference property: Additional python code files used for reference in the main definition file, which will override the 'pyFiles' of the spark job definition you provide.
      Parameters:
      pythonCodeReference - the pythonCodeReference value to set.
      Returns:
      the SynapseSparkJobActivityTypeProperties object itself.

    • filesV2

      public List<Object> filesV2()
      Get the filesV2 property: Additional files used for reference in the main definition file, which will override the 'jars' and 'files' of the spark job definition you provide.
      Returns:
      the filesV2 value.

    • withFilesV2

      public SynapseSparkJobActivityTypeProperties withFilesV2(List<Object> filesV2)
      Set the filesV2 property: Additional files used for reference in the main definition file, which will override the 'jars' and 'files' of the spark job definition you provide.
      Parameters:
      filesV2 - the filesV2 value to set.
      Returns:
      the SynapseSparkJobActivityTypeProperties object itself.

    • targetBigDataPool

      public BigDataPoolParametrizationReference targetBigDataPool()
      Get the targetBigDataPool property: The name of the big data pool which will be used to execute the spark batch job, which will override the 'targetBigDataPool' of the spark job definition you provide.
      Returns:
      the targetBigDataPool value.

    • withTargetBigDataPool

      public SynapseSparkJobActivityTypeProperties withTargetBigDataPool(BigDataPoolParametrizationReference targetBigDataPool)
      Set the targetBigDataPool property: The name of the big data pool which will be used to execute the spark batch job, which will override the 'targetBigDataPool' of the spark job definition you provide.
      Parameters:
      targetBigDataPool - the targetBigDataPool value to set.
      Returns:
      the SynapseSparkJobActivityTypeProperties object itself.

    • executorSize

      public Object executorSize()
      Get the executorSize property: Number of core and memory to be used for executors allocated in the specified Spark pool for the job, which will be used for overriding 'executorCores' and 'executorMemory' of the spark job definition you provide. Type: string (or Expression with resultType string).
      Returns:
      the executorSize value.

    • withExecutorSize

      public SynapseSparkJobActivityTypeProperties withExecutorSize(Object executorSize)
      Set the executorSize property: Number of core and memory to be used for executors allocated in the specified Spark pool for the job, which will be used for overriding 'executorCores' and 'executorMemory' of the spark job definition you provide. Type: string (or Expression with resultType string).
      Parameters:
      executorSize - the executorSize value to set.
      Returns:
      the SynapseSparkJobActivityTypeProperties object itself.

    • conf

      public Object conf()
      Get the conf property: Spark configuration properties, which will override the 'conf' of the spark job definition you provide.
      Returns:
      the conf value.

    • withConf

      public SynapseSparkJobActivityTypeProperties withConf(Object conf)
      Set the conf property: Spark configuration properties, which will override the 'conf' of the spark job definition you provide.
      Parameters:
      conf - the conf value to set.
      Returns:
      the SynapseSparkJobActivityTypeProperties object itself.

    • driverSize

      public Object driverSize()
      Get the driverSize property: Number of core and memory to be used for driver allocated in the specified Spark pool for the job, which will be used for overriding 'driverCores' and 'driverMemory' of the spark job definition you provide. Type: string (or Expression with resultType string).
      Returns:
      the driverSize value.

    • withDriverSize

      public SynapseSparkJobActivityTypeProperties withDriverSize(Object driverSize)
      Set the driverSize property: Number of core and memory to be used for driver allocated in the specified Spark pool for the job, which will be used for overriding 'driverCores' and 'driverMemory' of the spark job definition you provide. Type: string (or Expression with resultType string).
      Parameters:
      driverSize - the driverSize value to set.
      Returns:
      the SynapseSparkJobActivityTypeProperties object itself.

    • numExecutors

      public Object numExecutors()
      Get the numExecutors property: Number of executors to launch for this job, which will override the 'numExecutors' of the spark job definition you provide. Type: integer (or Expression with resultType integer).
      Returns:
      the numExecutors value.

    • withNumExecutors

      public SynapseSparkJobActivityTypeProperties withNumExecutors(Object numExecutors)
      Set the numExecutors property: Number of executors to launch for this job, which will override the 'numExecutors' of the spark job definition you provide. Type: integer (or Expression with resultType integer).
      Parameters:
      numExecutors - the numExecutors value to set.
      Returns:
      the SynapseSparkJobActivityTypeProperties object itself.

    • configurationType

      public ConfigurationType configurationType()
      Get the configurationType property: The type of the spark config.
      Returns:
      the configurationType value.

    • withConfigurationType

      public SynapseSparkJobActivityTypeProperties withConfigurationType(ConfigurationType configurationType)
      Set the configurationType property: The type of the spark config.
      Parameters:
      configurationType - the configurationType value to set.
      Returns:
      the SynapseSparkJobActivityTypeProperties object itself.

    • targetSparkConfiguration

      public SparkConfigurationParametrizationReference targetSparkConfiguration()
      Get the targetSparkConfiguration property: The spark configuration of the spark job.
      Returns:
      the targetSparkConfiguration value.

    • withTargetSparkConfiguration

      public SynapseSparkJobActivityTypeProperties withTargetSparkConfiguration(SparkConfigurationParametrizationReference targetSparkConfiguration)
      Set the targetSparkConfiguration property: The spark configuration of the spark job.
      Parameters:
      targetSparkConfiguration - the targetSparkConfiguration value to set.
      Returns:
      the SynapseSparkJobActivityTypeProperties object itself.

    • sparkConfig

      public Map<String,Object> sparkConfig()
      Get the sparkConfig property: Spark configuration property.
      Returns:
      the sparkConfig value.

    • withSparkConfig

      public SynapseSparkJobActivityTypeProperties withSparkConfig(Map<String,Object> sparkConfig)
      Set the sparkConfig property: Spark configuration property.
      Parameters:
      sparkConfig - the sparkConfig value to set.
      Returns:
      the SynapseSparkJobActivityTypeProperties object itself.

    • validate

      public void validate()
      Validates the instance.
      Throws:
      IllegalArgumentException - thrown if the instance is not valid.