azure.mgmt.datalake.analytics.job.models module

class azure.mgmt.datalake.analytics.job.models.JobInnerError(**kwargs)[source]

Bases: msrest.serialization.Model

The Data Lake Analytics job error details.

Variables are only populated by the server, and will be ignored when sending a request.

Variables

  • error_id (str) – The specific identifier for the type of error encountered in the job.

  • severity (str or SeverityTypes) – The severity level of the failure. Possible values include: ‘Warning’, ‘Error’, ‘Info’, ‘SevereWarning’, ‘Deprecated’, ‘UserWarning’

  • source (str) – The ultimate source of the failure (usually either SYSTEM or USER).

  • message (str) – The user friendly error message for the failure.

  • description (str) – The error message description.

  • details (str) – The details of the error message.

  • diagnostic_code (int) – The diagnostic error code.

  • component (str) – The component that failed.

  • resolution (str) – The recommended resolution for the failure, if any.

  • help_link (str) – The link to MSDN or Azure help for this type of error, if any.

  • internal_diagnostics (str) – The internal diagnostic stack trace if the user requesting the job error details has sufficient permissions it will be retrieved, otherwise it will be empty.

  • inner_error (JobInnerError) – The inner error of this specific job error message, if any.

class azure.mgmt.datalake.analytics.job.models.JobErrorDetails(**kwargs)[source]

Bases: msrest.serialization.Model

The Data Lake Analytics job error details.

Variables are only populated by the server, and will be ignored when sending a request.

Variables

  • error_id (str) – The specific identifier for the type of error encountered in the job.

  • severity (str or SeverityTypes) – The severity level of the failure. Possible values include: ‘Warning’, ‘Error’, ‘Info’, ‘SevereWarning’, ‘Deprecated’, ‘UserWarning’

  • source (str) – The ultimate source of the failure (usually either SYSTEM or USER).

  • message (str) – The user friendly error message for the failure.

  • description (str) – The error message description.

  • details (str) – The details of the error message.

  • line_number (int) – The specific line number in the job where the error occured.

  • start_offset (int) – The start offset in the job where the error was found

  • end_offset (int) – The end offset in the job where the error was found.

  • resolution (str) – The recommended resolution for the failure, if any.

  • file_path (str) – The path to any supplemental error files, if any.

  • help_link (str) – The link to MSDN or Azure help for this type of error, if any.

  • internal_diagnostics (str) – The internal diagnostic stack trace if the user requesting the job error details has sufficient permissions it will be retrieved, otherwise it will be empty.

  • inner_error (JobInnerError) – The inner error of this specific job error message, if any.

class azure.mgmt.datalake.analytics.job.models.JobStateAuditRecord(**kwargs)[source]

Bases: msrest.serialization.Model

The Data Lake Analytics job state audit records for tracking the lifecycle of a job.

Variables are only populated by the server, and will be ignored when sending a request.

Variables

  • new_state (str) – The new state the job is in.

  • time_stamp (datetime) – The time stamp that the state change took place.

  • requested_by_user (str) – The user who requests the change.

  • details (str) – The details of the audit log.

class azure.mgmt.datalake.analytics.job.models.JobProperties(*, script: str, runtime_version: str = None, **kwargs)[source]

Bases: msrest.serialization.Model

The common Data Lake Analytics job properties.

You probably want to use the sub-classes and not this class directly. Known sub-classes are: USqlJobProperties, HiveJobProperties, ScopeJobProperties

All required parameters must be populated in order to send to Azure.

Parameters

  • runtime_version (str) – The runtime version of the Data Lake Analytics engine to use for the specific type of job being run.

  • script (str) – Required. The script to run. Please note that the maximum script size is 3 MB.

  • type (str) – Required. Constant filled by server.

class azure.mgmt.datalake.analytics.job.models.JobInformation(*, name: str, type, properties, degree_of_parallelism: int = 1, priority: int = None, log_file_patterns=None, related=None, tags=None, **kwargs)[source]

Bases: azure.mgmt.datalake.analytics.job.models.job_information_basic_py3.JobInformationBasic

The extended Data Lake Analytics job information properties returned when retrieving a specific job.

Variables are only populated by the server, and will be ignored when sending a request.

All required parameters must be populated in order to send to Azure.

Variables

  • job_id (str) – The job’s unique identifier (a GUID).

  • submitter (str) – The user or account that submitted the job.

  • submit_time (datetime) – The time the job was submitted to the service.

  • start_time (datetime) – The start time of the job.

  • end_time (datetime) – The completion time of the job.

  • state (str or JobState) – The job state. When the job is in the Ended state, refer to Result and ErrorMessage for details. Possible values include: ‘Accepted’, ‘Compiling’, ‘Ended’, ‘New’, ‘Queued’, ‘Running’, ‘Scheduling’, ‘Starting’, ‘Paused’, ‘WaitingForCapacity’

  • result (str or JobResult) – The result of job execution or the current result of the running job. Possible values include: ‘None’, ‘Succeeded’, ‘Cancelled’, ‘Failed’

  • log_folder (str) – The log folder path to use in the following format: adl://<accountName>.azuredatalakestore.net/system/jobservice/jobs/Usql/2016/03/13/17/18/5fe51957-93bc-4de0-8ddc-c5a4753b068b/logs/.

  • error_message (list[JobErrorDetails]) – The error message details for the job, if the job failed.

  • state_audit_records (list[JobStateAuditRecord]) – The job state audit records, indicating when various operations have been performed on this job.

Parameters

  • name (str) – Required. The friendly name of the job.

  • type (str or JobType) – Required. The job type of the current job (Hive, USql, or Scope (for internal use only)). Possible values include: ‘USql’, ‘Hive’, ‘Scope’

  • degree_of_parallelism (int) – The degree of parallelism used for this job. This must be greater than 0, if set to less than 0 it will default to 1. Default value: 1 .

  • priority (int) – The priority value for the current job. Lower numbers have a higher priority. By default, a job has a priority of 1000. This must be greater than 0.

  • log_file_patterns (list[str]) – The list of log file name patterns to find in the logFolder. ‘*’ is the only matching character allowed. Example format: jobExecution*.log or mylog.txt

  • related (JobRelationshipProperties) – The recurring job relationship information properties.

  • tags (dict[str, str]) – The key-value pairs used to add additional metadata to the job information. (Only for use internally with Scope job type.)

  • properties (JobProperties) – Required. The job specific properties.

class azure.mgmt.datalake.analytics.job.models.JobRelationshipProperties(*, recurrence_id: str, pipeline_id: str = None, pipeline_name: str = None, pipeline_uri: str = None, run_id: str = None, recurrence_name: str = None, **kwargs)[source]

Bases: msrest.serialization.Model

Job relationship information properties including pipeline information, correlation information, etc.

All required parameters must be populated in order to send to Azure.

Parameters

  • pipeline_id (str) – The job relationship pipeline identifier (a GUID).

  • pipeline_name (str) – The friendly name of the job relationship pipeline, which does not need to be unique.

  • pipeline_uri (str) – The pipeline uri, unique, links to the originating service for this pipeline.

  • run_id (str) – The run identifier (a GUID), unique identifier of the iteration of this pipeline.

  • recurrence_id (str) – Required. The recurrence identifier (a GUID), unique per activity/script, regardless of iterations. This is something to link different occurrences of the same job together.

  • recurrence_name (str) – The recurrence name, user friendly name for the correlation between jobs.

class azure.mgmt.datalake.analytics.job.models.JobInformationBasic(*, name: str, type, degree_of_parallelism: int = 1, priority: int = None, log_file_patterns=None, related=None, tags=None, **kwargs)[source]

Bases: msrest.serialization.Model

The common Data Lake Analytics job information properties.

Variables are only populated by the server, and will be ignored when sending a request.

All required parameters must be populated in order to send to Azure.

Variables

  • job_id (str) – The job’s unique identifier (a GUID).

  • submitter (str) – The user or account that submitted the job.

  • submit_time (datetime) – The time the job was submitted to the service.

  • start_time (datetime) – The start time of the job.

  • end_time (datetime) – The completion time of the job.

  • state (str or JobState) – The job state. When the job is in the Ended state, refer to Result and ErrorMessage for details. Possible values include: ‘Accepted’, ‘Compiling’, ‘Ended’, ‘New’, ‘Queued’, ‘Running’, ‘Scheduling’, ‘Starting’, ‘Paused’, ‘WaitingForCapacity’

  • result (str or JobResult) – The result of job execution or the current result of the running job. Possible values include: ‘None’, ‘Succeeded’, ‘Cancelled’, ‘Failed’

  • log_folder (str) – The log folder path to use in the following format: adl://<accountName>.azuredatalakestore.net/system/jobservice/jobs/Usql/2016/03/13/17/18/5fe51957-93bc-4de0-8ddc-c5a4753b068b/logs/.

Parameters

  • name (str) – Required. The friendly name of the job.

  • type (str or JobType) – Required. The job type of the current job (Hive, USql, or Scope (for internal use only)). Possible values include: ‘USql’, ‘Hive’, ‘Scope’

  • degree_of_parallelism (int) – The degree of parallelism used for this job. This must be greater than 0, if set to less than 0 it will default to 1. Default value: 1 .

  • priority (int) – The priority value for the current job. Lower numbers have a higher priority. By default, a job has a priority of 1000. This must be greater than 0.

  • log_file_patterns (list[str]) – The list of log file name patterns to find in the logFolder. ‘*’ is the only matching character allowed. Example format: jobExecution*.log or mylog.txt

  • related (JobRelationshipProperties) – The recurring job relationship information properties.

  • tags (dict[str, str]) – The key-value pairs used to add additional metadata to the job information. (Only for use internally with Scope job type.)

class azure.mgmt.datalake.analytics.job.models.JobResource(*, name: str = None, resource_path: str = None, type=None, **kwargs)[source]

Bases: msrest.serialization.Model

The Data Lake Analytics job resources.

Parameters

  • name (str) – The name of the resource.

  • resource_path (str) – The path to the resource.

  • type (str or JobResourceType) – The job resource type. Possible values include: ‘VertexResource’, ‘JobManagerResource’, ‘StatisticsResource’, ‘VertexResourceInUserFolder’, ‘JobManagerResourceInUserFolder’, ‘StatisticsResourceInUserFolder’

class azure.mgmt.datalake.analytics.job.models.JobStatisticsVertex(**kwargs)[source]

Bases: msrest.serialization.Model

The detailed information for a vertex.

Variables are only populated by the server, and will be ignored when sending a request.

Variables

  • name (str) – The name of the vertex.

  • vertex_id (str) – The id of the vertex.

  • execution_time (timedelta) – The amount of execution time of the vertex.

  • data_read (long) – The amount of data read of the vertex, in bytes.

  • peak_mem_usage (long) – The amount of peak memory usage of the vertex, in bytes.

class azure.mgmt.datalake.analytics.job.models.ResourceUsageStatistics(**kwargs)[source]

Bases: msrest.serialization.Model

The statistics information for resource usage.

Variables are only populated by the server, and will be ignored when sending a request.

Variables

  • average (float) – The average value.

  • minimum (long) – The minimum value.

  • maximum (long) – The maximum value.

class azure.mgmt.datalake.analytics.job.models.JobStatisticsVertexStage(*, max_data_read_vertex=None, max_execution_time_vertex=None, max_peak_mem_usage_vertex=None, allocated_container_cpu_core_count=None, allocated_container_mem_size=None, used_vertex_cpu_core_count=None, used_vertex_peak_mem_size=None, **kwargs)[source]

Bases: msrest.serialization.Model

The Data Lake Analytics job statistics vertex stage information.

Variables are only populated by the server, and will be ignored when sending a request.

Variables

  • data_read (long) – The amount of data read, in bytes.

  • data_read_cross_pod (long) – The amount of data read across multiple pods, in bytes.

  • data_read_intra_pod (long) – The amount of data read in one pod, in bytes.

  • data_to_read (long) – The amount of data remaining to be read, in bytes.

  • data_written (long) – The amount of data written, in bytes.

  • duplicate_discard_count (int) – The number of duplicates that were discarded.

  • failed_count (int) – The number of failures that occured in this stage.

  • max_vertex_data_read (long) – The maximum amount of data read in a single vertex, in bytes.

  • min_vertex_data_read (long) – The minimum amount of data read in a single vertex, in bytes.

  • read_failure_count (int) – The number of read failures in this stage.

  • revocation_count (int) – The number of vertices that were revoked during this stage.

  • running_count (int) – The number of currently running vertices in this stage.

  • scheduled_count (int) – The number of currently scheduled vertices in this stage.

  • stage_name (str) – The name of this stage in job execution.

  • succeeded_count (int) – The number of vertices that succeeded in this stage.

  • temp_data_written (long) – The amount of temporary data written, in bytes.

  • total_count (int) – The total vertex count for this stage.

  • total_failed_time (timedelta) – The amount of time that failed vertices took up in this stage.

  • total_progress (int) – The current progress of this stage, as a percentage.

  • total_succeeded_time (timedelta) – The amount of time all successful vertices took in this stage.

  • total_peak_mem_usage (long) – The sum of the peak memory usage of all the vertices in the stage, in bytes.

  • total_execution_time (timedelta) – The sum of the total execution time of all the vertices in the stage.

  • estimated_vertex_cpu_core_count (int) – The estimated vertex CPU core count.

  • estimated_vertex_peak_cpu_core_count (int) – The estimated vertex peak CPU core count.

  • estimated_vertex_mem_size (long) – The estimated vertex memory size, in bytes.

Parameters

  • max_data_read_vertex (JobStatisticsVertex) – the vertex with the maximum amount of data read.

  • max_execution_time_vertex (JobStatisticsVertex) – the vertex with the maximum execution time.

  • max_peak_mem_usage_vertex (JobStatisticsVertex) – the vertex with the maximum peak memory usage.

  • allocated_container_cpu_core_count (ResourceUsageStatistics) – The statistics information for the allocated container CPU core count.

  • allocated_container_mem_size (ResourceUsageStatistics) – The statistics information for the allocated container memory size.

  • used_vertex_cpu_core_count (ResourceUsageStatistics) – The statistics information for the used vertex CPU core count.

  • used_vertex_peak_mem_size (ResourceUsageStatistics) – The statistics information for the used vertex peak memory size.

class azure.mgmt.datalake.analytics.job.models.JobStatistics(**kwargs)[source]

Bases: msrest.serialization.Model

The Data Lake Analytics job execution statistics.

Variables are only populated by the server, and will be ignored when sending a request.

Variables

  • last_update_time_utc (datetime) – The last update time for the statistics.

  • finalizing_time_utc (datetime) – The job finalizing start time.

  • stages (list[JobStatisticsVertexStage]) – The list of stages for the job.

class azure.mgmt.datalake.analytics.job.models.JobDataPath(**kwargs)[source]

Bases: msrest.serialization.Model

A Data Lake Analytics job data path item.

Variables are only populated by the server, and will be ignored when sending a request.

Variables

  • job_id (str) – The ID of the job this data is for.

  • command (str) – The command that this job data relates to.

  • paths (list[str]) – The list of paths to all of the job data.

class azure.mgmt.datalake.analytics.job.models.Diagnostics(**kwargs)[source]

Bases: msrest.serialization.Model

Error diagnostic information for failed jobs.

Variables are only populated by the server, and will be ignored when sending a request.

Variables

  • message (str) – The error message.

  • severity (str or SeverityTypes) – The severity of the error. Possible values include: ‘Warning’, ‘Error’, ‘Info’, ‘SevereWarning’, ‘Deprecated’, ‘UserWarning’

  • line_number (int) – The line number the error occured on.

  • column_number (int) – The column where the error occured.

  • start (int) – The starting index of the error.

  • end (int) – The ending index of the error.

class azure.mgmt.datalake.analytics.job.models.USqlJobProperties(*, script: str, runtime_version: str = None, statistics=None, debug_data=None, **kwargs)[source]

Bases: azure.mgmt.datalake.analytics.job.models.job_properties_py3.JobProperties

U-SQL job properties used when retrieving U-SQL jobs.

Variables are only populated by the server, and will be ignored when sending a request.

All required parameters must be populated in order to send to Azure.

Parameters

  • runtime_version (str) – The runtime version of the Data Lake Analytics engine to use for the specific type of job being run.

  • script (str) – Required. The script to run. Please note that the maximum script size is 3 MB.

  • type (str) – Required. Constant filled by server.

  • statistics (JobStatistics) – The job specific statistics.

  • debug_data (JobDataPath) – The job specific debug data locations.

Variables

  • resources (list[JobResource]) – The list of resources that are required by the job.

  • diagnostics (list[Diagnostics]) – The diagnostics for the job.

  • algebra_file_path (str) – The algebra file path after the job has completed.

  • total_compilation_time (timedelta) – The total time this job spent compiling. This value should not be set by the user and will be ignored if it is.

  • total_queued_time (timedelta) – The total time this job spent queued. This value should not be set by the user and will be ignored if it is.

  • total_running_time (timedelta) – The total time this job spent executing. This value should not be set by the user and will be ignored if it is.

  • total_paused_time (timedelta) – The total time this job spent paused. This value should not be set by the user and will be ignored if it is.

  • root_process_node_id (str) – The ID used to identify the job manager coordinating job execution. This value should not be set by the user and will be ignored if it is.

  • yarn_application_id (str) – The ID used to identify the yarn application executing the job. This value should not be set by the user and will be ignored if it is.

  • yarn_application_time_stamp (long) – The timestamp (in ticks) for the yarn application executing the job. This value should not be set by the user and will be ignored if it is.

  • compile_mode (str or CompileMode) – The specific compilation mode for the job used during execution. If this is not specified during submission, the server will determine the optimal compilation mode. Possible values include: ‘Semantic’, ‘Full’, ‘SingleBox’

class azure.mgmt.datalake.analytics.job.models.HiveJobProperties(*, script: str, runtime_version: str = None, **kwargs)[source]

Bases: azure.mgmt.datalake.analytics.job.models.job_properties_py3.JobProperties

Hive job properties used when retrieving Hive jobs.

Variables are only populated by the server, and will be ignored when sending a request.

All required parameters must be populated in order to send to Azure.

Parameters

  • runtime_version (str) – The runtime version of the Data Lake Analytics engine to use for the specific type of job being run.

  • script (str) – Required. The script to run. Please note that the maximum script size is 3 MB.

  • type (str) – Required. Constant filled by server.

Variables

  • logs_location (str) – The Hive logs location.

  • output_location (str) – The location of Hive job output files (both execution output and results).

  • statement_count (int) – The number of statements that will be run based on the script.

  • executed_statement_count (int) – The number of statements that have been run based on the script.

class azure.mgmt.datalake.analytics.job.models.ScopeJobResource(*, name: str = None, path: str = None, **kwargs)[source]

Bases: msrest.serialization.Model

The Scope job resources. (Only for use internally with Scope job type.).

Parameters

  • name (str) – The name of the resource.

  • path (str) – The path to the resource.

class azure.mgmt.datalake.analytics.job.models.ScopeJobProperties(*, script: str, runtime_version: str = None, notifier: str = None, **kwargs)[source]

Bases: azure.mgmt.datalake.analytics.job.models.job_properties_py3.JobProperties

Scope job properties used when submitting and retrieving Scope jobs. (Only for use internally with Scope job type.).

Variables are only populated by the server, and will be ignored when sending a request.

All required parameters must be populated in order to send to Azure.

Parameters

  • runtime_version (str) – The runtime version of the Data Lake Analytics engine to use for the specific type of job being run.

  • script (str) – Required. The script to run. Please note that the maximum script size is 3 MB.

  • type (str) – Required. Constant filled by server.

  • notifier (str) – The list of email addresses, separated by semi-colons, to notify when the job reaches a terminal state.

Variables

  • resources (list[ScopeJobResource]) – The list of resources that are required by the job.

  • user_algebra_path (str) – The algebra file path after the job has completed.

  • total_compilation_time (timedelta) – The total time this job spent compiling. This value should not be set by the user and will be ignored if it is.

  • total_queued_time (timedelta) – The total time this job spent queued. This value should not be set by the user and will be ignored if it is.

  • total_running_time (timedelta) – The total time this job spent executing. This value should not be set by the user and will be ignored if it is.

  • total_paused_time (timedelta) – The total time this job spent paused. This value should not be set by the user and will be ignored if it is.

  • root_process_node_id (str) – The ID used to identify the job manager coordinating job execution. This value should not be set by the user and will be ignored if it is.

  • yarn_application_id (str) – The ID used to identify the yarn application executing the job. This value should not be set by the user and will be ignored if it is.

class azure.mgmt.datalake.analytics.job.models.JobPipelineRunInformation(**kwargs)[source]

Bases: msrest.serialization.Model

Run info for a specific job pipeline.

Variables are only populated by the server, and will be ignored when sending a request.

Variables

  • run_id (str) – The run identifier of an instance of pipeline executions (a GUID).

  • last_submit_time (datetime) – The time this instance was last submitted.

class azure.mgmt.datalake.analytics.job.models.JobPipelineInformation(**kwargs)[source]

Bases: msrest.serialization.Model

Job Pipeline Information, showing the relationship of jobs and recurrences of those jobs in a pipeline.

Variables are only populated by the server, and will be ignored when sending a request.

Variables

  • pipeline_id (str) – The job relationship pipeline identifier (a GUID).

  • pipeline_name (str) – The friendly name of the job relationship pipeline, which does not need to be unique.

  • pipeline_uri (str) – The pipeline uri, unique, links to the originating service for this pipeline.

  • num_jobs_failed (int) – The number of jobs in this pipeline that have failed.

  • num_jobs_canceled (int) – The number of jobs in this pipeline that have been canceled.

  • num_jobs_succeeded (int) – The number of jobs in this pipeline that have succeeded.

  • au_hours_failed (float) – The number of job execution hours that resulted in failed jobs.

  • au_hours_canceled (float) – The number of job execution hours that resulted in canceled jobs.

  • au_hours_succeeded (float) – The number of job execution hours that resulted in successful jobs.

  • last_submit_time (datetime) – The last time a job in this pipeline was submitted.

  • runs (list[JobPipelineRunInformation]) – The list of recurrence identifiers representing each run of this pipeline.

  • recurrences (list[str]) – The list of recurrence identifiers representing each run of this pipeline.

class azure.mgmt.datalake.analytics.job.models.JobRecurrenceInformation(**kwargs)[source]

Bases: msrest.serialization.Model

Recurrence job information for a specific recurrence.

Variables are only populated by the server, and will be ignored when sending a request.

Variables

  • recurrence_id (str) – The recurrence identifier (a GUID), unique per activity/script, regardless of iterations. This is something to link different occurrences of the same job together.

  • recurrence_name (str) – The recurrence name, user friendly name for the correlation between jobs.

  • num_jobs_failed (int) – The number of jobs in this recurrence that have failed.

  • num_jobs_canceled (int) – The number of jobs in this recurrence that have been canceled.

  • num_jobs_succeeded (int) – The number of jobs in this recurrence that have succeeded.

  • au_hours_failed (float) – The number of job execution hours that resulted in failed jobs.

  • au_hours_canceled (float) – The number of job execution hours that resulted in canceled jobs.

  • au_hours_succeeded (float) – The number of job execution hours that resulted in successful jobs.

  • last_submit_time (datetime) – The last time a job in this recurrence was submitted.

class azure.mgmt.datalake.analytics.job.models.CreateJobProperties(*, script: str, runtime_version: str = None, **kwargs)[source]

Bases: msrest.serialization.Model

The common Data Lake Analytics job properties for job submission.

You probably want to use the sub-classes and not this class directly. Known sub-classes are: CreateUSqlJobProperties, CreateScopeJobProperties

All required parameters must be populated in order to send to Azure.

Parameters

  • runtime_version (str) – The runtime version of the Data Lake Analytics engine to use for the specific type of job being run.

  • script (str) – Required. The script to run. Please note that the maximum script size is 3 MB.

  • type (str) – Required. Constant filled by server.

class azure.mgmt.datalake.analytics.job.models.BaseJobParameters(*, type, properties, **kwargs)[source]

Bases: msrest.serialization.Model

Data Lake Analytics Job Parameters base class for build and submit.

All required parameters must be populated in order to send to Azure.

Parameters

  • type (str or JobType) – Required. The job type of the current job (Hive, USql, or Scope (for internal use only)). Possible values include: ‘USql’, ‘Hive’, ‘Scope’

  • properties (CreateJobProperties) – Required. The job specific properties.

class azure.mgmt.datalake.analytics.job.models.CreateJobParameters(*, type, properties, name: str, degree_of_parallelism: int = 1, priority: int = None, log_file_patterns=None, related=None, **kwargs)[source]

Bases: azure.mgmt.datalake.analytics.job.models.base_job_parameters_py3.BaseJobParameters

The parameters used to submit a new Data Lake Analytics job.

All required parameters must be populated in order to send to Azure.

Parameters

  • type (str or JobType) – Required. The job type of the current job (Hive, USql, or Scope (for internal use only)). Possible values include: ‘USql’, ‘Hive’, ‘Scope’

  • properties (CreateJobProperties) – Required. The job specific properties.

  • name (str) – Required. The friendly name of the job to submit.

  • degree_of_parallelism (int) – The degree of parallelism to use for this job. This must be greater than 0, if set to less than 0 it will default to 1. Default value: 1 .

  • priority (int) – The priority value to use for the current job. Lower numbers have a higher priority. By default, a job has a priority of 1000. This must be greater than 0.

  • log_file_patterns (list[str]) – The list of log file name patterns to find in the logFolder. ‘*’ is the only matching character allowed. Example format: jobExecution*.log or mylog.txt

  • related (JobRelationshipProperties) – The recurring job relationship information properties.

class azure.mgmt.datalake.analytics.job.models.CreateScopeJobParameters(*, type, properties, name: str, degree_of_parallelism: int = 1, priority: int = None, log_file_patterns=None, related=None, tags=None, **kwargs)[source]

Bases: azure.mgmt.datalake.analytics.job.models.create_job_parameters_py3.CreateJobParameters

The parameters used to submit a new Data Lake Analytics Scope job. (Only for use internally with Scope job type.).

All required parameters must be populated in order to send to Azure.

Parameters

  • type (str or JobType) – Required. The job type of the current job (Hive, USql, or Scope (for internal use only)). Possible values include: ‘USql’, ‘Hive’, ‘Scope’

  • properties (CreateJobProperties) – Required. The job specific properties.

  • name (str) – Required. The friendly name of the job to submit.

  • degree_of_parallelism (int) – The degree of parallelism to use for this job. This must be greater than 0, if set to less than 0 it will default to 1. Default value: 1 .

  • priority (int) – The priority value to use for the current job. Lower numbers have a higher priority. By default, a job has a priority of 1000. This must be greater than 0.

  • log_file_patterns (list[str]) – The list of log file name patterns to find in the logFolder. ‘*’ is the only matching character allowed. Example format: jobExecution*.log or mylog.txt

  • related (JobRelationshipProperties) – The recurring job relationship information properties.

  • tags (dict[str, str]) – The key-value pairs used to add additional metadata to the job information.

class azure.mgmt.datalake.analytics.job.models.CreateUSqlJobProperties(*, script: str, runtime_version: str = None, compile_mode=None, **kwargs)[source]

Bases: azure.mgmt.datalake.analytics.job.models.create_job_properties_py3.CreateJobProperties

U-SQL job properties used when submitting U-SQL jobs.

All required parameters must be populated in order to send to Azure.

Parameters

  • runtime_version (str) – The runtime version of the Data Lake Analytics engine to use for the specific type of job being run.

  • script (str) – Required. The script to run. Please note that the maximum script size is 3 MB.

  • type (str) – Required. Constant filled by server.

  • compile_mode (str or CompileMode) – The specific compilation mode for the job used during execution. If this is not specified during submission, the server will determine the optimal compilation mode. Possible values include: ‘Semantic’, ‘Full’, ‘SingleBox’

class azure.mgmt.datalake.analytics.job.models.CreateScopeJobProperties(*, script: str, runtime_version: str = None, resources=None, notifier: str = None, **kwargs)[source]

Bases: azure.mgmt.datalake.analytics.job.models.create_job_properties_py3.CreateJobProperties

Scope job properties used when submitting Scope jobs. (Only for use internally with Scope job type.).

All required parameters must be populated in order to send to Azure.

Parameters

  • runtime_version (str) – The runtime version of the Data Lake Analytics engine to use for the specific type of job being run.

  • script (str) – Required. The script to run. Please note that the maximum script size is 3 MB.

  • type (str) – Required. Constant filled by server.

  • resources (list[ScopeJobResource]) – The list of resources that are required by the job.

  • notifier (str) – The list of email addresses, separated by semi-colons, to notify when the job reaches a terminal state.

class azure.mgmt.datalake.analytics.job.models.BuildJobParameters(*, type, properties, name: str = None, **kwargs)[source]

Bases: azure.mgmt.datalake.analytics.job.models.base_job_parameters_py3.BaseJobParameters

The parameters used to build a new Data Lake Analytics job.

All required parameters must be populated in order to send to Azure.

Parameters

  • type (str or JobType) – Required. The job type of the current job (Hive, USql, or Scope (for internal use only)). Possible values include: ‘USql’, ‘Hive’, ‘Scope’

  • properties (CreateJobProperties) – Required. The job specific properties.

  • name (str) – The friendly name of the job to build.

class azure.mgmt.datalake.analytics.job.models.UpdateJobParameters(*, degree_of_parallelism: int = None, priority: int = None, tags=None, **kwargs)[source]

Bases: msrest.serialization.Model

The parameters that can be used to update existing Data Lake Analytics job information properties. (Only for use internally with Scope job type.).

Parameters

  • degree_of_parallelism (int) – The degree of parallelism used for this job. This must be greater than 0, if set to less than 0 it will default to 1.

  • priority (int) – The priority value for the current job. Lower numbers have a higher priority. By default, a job has a priority of 1000. This must be greater than 0.

  • tags (dict[str, str]) – The key-value pairs used to add additional metadata to the job information.

class azure.mgmt.datalake.analytics.job.models.JobInformationBasicPaged(*args, **kwargs)[source]

Bases: msrest.paging.Paged

A paging container for iterating over a list of JobInformationBasic object

Bring async to Paging.

“async_command” is mandatory keyword argument for this mixin to work.

class azure.mgmt.datalake.analytics.job.models.JobPipelineInformationPaged(*args, **kwargs)[source]

Bases: msrest.paging.Paged

A paging container for iterating over a list of JobPipelineInformation object

Bring async to Paging.

“async_command” is mandatory keyword argument for this mixin to work.

class azure.mgmt.datalake.analytics.job.models.JobRecurrenceInformationPaged(*args, **kwargs)[source]

Bases: msrest.paging.Paged

A paging container for iterating over a list of JobRecurrenceInformation object

Bring async to Paging.

“async_command” is mandatory keyword argument for this mixin to work.

class azure.mgmt.datalake.analytics.job.models.SeverityTypes[source]

Bases: str, enum.Enum

An enumeration.

deprecated = 'Deprecated'
error = 'Error'
info = 'Info'
severe_warning = 'SevereWarning'
user_warning = 'UserWarning'
warning = 'Warning'
class azure.mgmt.datalake.analytics.job.models.JobType[source]

Bases: str, enum.Enum

An enumeration.

hive = 'Hive'
scope = 'Scope'
usql = 'USql'
class azure.mgmt.datalake.analytics.job.models.JobState[source]

Bases: str, enum.Enum

An enumeration.

accepted = 'Accepted'
compiling = 'Compiling'
ended = 'Ended'
new = 'New'
paused = 'Paused'
queued = 'Queued'
running = 'Running'
scheduling = 'Scheduling'
starting = 'Starting'
waiting_for_capacity = 'WaitingForCapacity'
class azure.mgmt.datalake.analytics.job.models.JobResult[source]

Bases: str, enum.Enum

An enumeration.

cancelled = 'Cancelled'
failed = 'Failed'
none = 'None'
succeeded = 'Succeeded'
class azure.mgmt.datalake.analytics.job.models.JobResourceType[source]

Bases: str, enum.Enum

An enumeration.

job_manager_resource = 'JobManagerResource'
job_manager_resource_in_user_folder = 'JobManagerResourceInUserFolder'
statistics_resource = 'StatisticsResource'
statistics_resource_in_user_folder = 'StatisticsResourceInUserFolder'
vertex_resource = 'VertexResource'
vertex_resource_in_user_folder = 'VertexResourceInUserFolder'
class azure.mgmt.datalake.analytics.job.models.CompileMode[source]

Bases: str, enum.Enum

An enumeration.

full = 'Full'
semantic = 'Semantic'
single_box = 'SingleBox'