azure.storage.filedatalake package¶
-
class
azure.storage.filedatalake.
AccessControlChangeCounters
(directories_successful, files_successful, failure_count)[source]¶ AccessControlChangeCounters contains counts of operations that change Access Control Lists recursively.
- Variables
-
get
(key, default=None)¶
-
has_key
(k)¶
-
items
()¶
-
keys
()¶
-
update
(*args, **kwargs)¶
-
values
()¶
-
class
azure.storage.filedatalake.
AccessControlChangeFailure
(name, is_directory, error_message)[source]¶ Represents an entry that failed to update Access Control List.
- Variables
-
get
(key, default=None)¶
-
has_key
(k)¶
-
items
()¶
-
keys
()¶
-
update
(*args, **kwargs)¶
-
values
()¶
-
class
azure.storage.filedatalake.
AccessControlChangeResult
(counters, continuation)[source]¶ AccessControlChangeResult contains result of operations that change Access Control Lists recursively.
- Variables
counters (AccessControlChangeCounters) – Contains counts of paths changed from start of the operation.
continuation (str) – Optional continuation token. Value is present when operation is split into multiple batches and can be used to resume progress.
-
get
(key, default=None)¶
-
has_key
(k)¶
-
items
()¶
-
keys
()¶
-
update
(*args, **kwargs)¶
-
values
()¶
-
class
azure.storage.filedatalake.
AccessControlChanges
(batch_counters, aggregate_counters, batch_failures, continuation)[source]¶ AccessControlChanges contains batch and cumulative counts of operations that change Access Control Lists recursively. Additionally it exposes path entries that failed to update while these operations progress.
- Variables
batch_counters (AccessControlChangeCounters) – Contains counts of paths changed within single batch.
aggregate_counters (AccessControlChangeCounters) – Contains counts of paths changed from start of the operation.
batch_failures (list(AccessControlChangeFailure)) – List of path entries that failed to update Access Control List within single batch.
continuation (str) – An opaque continuation token that may be used to resume the operations in case of failures.
-
get
(key, default=None)¶
-
has_key
(k)¶
-
items
()¶
-
keys
()¶
-
update
(*args, **kwargs)¶
-
values
()¶
-
class
azure.storage.filedatalake.
AccessPolicy
(permission=None, expiry=None, **kwargs)[source]¶ Access Policy class used by the set and get access policy methods in each service.
A stored access policy can specify the start time, expiry time, and permissions for the Shared Access Signatures with which it’s associated. Depending on how you want to control access to your resource, you can specify all of these parameters within the stored access policy, and omit them from the URL for the Shared Access Signature. Doing so permits you to modify the associated signature’s behavior at any time, as well as to revoke it. Or you can specify one or more of the access policy parameters within the stored access policy, and the others on the URL. Finally, you can specify all of the parameters on the URL. In this case, you can use the stored access policy to revoke the signature, but not to modify its behavior.
Together the Shared Access Signature and the stored access policy must include all fields required to authenticate the signature. If any required fields are missing, the request will fail. Likewise, if a field is specified both in the Shared Access Signature URL and in the stored access policy, the request will fail with status code 400 (Bad Request).
- Parameters
permission (str or FileSystemSasPermissions) – The permissions associated with the shared access signature. The user is restricted to operations allowed by the permissions. Required unless an id is given referencing a stored access policy which contains this field. This field must be omitted if it has been specified in an associated stored access policy.
expiry (datetime or str) – The time at which the shared access signature becomes invalid. Required unless an id is given referencing a stored access policy which contains this field. This field must be omitted if it has been specified in an associated stored access policy. Azure will always convert values to UTC. If a date is passed in without timezone info, it is assumed to be UTC.
- Keyword Arguments
start (str) – The time at which the shared access signature becomes valid. If omitted, start time for this call is assumed to be the time when the storage service receives the request. Azure will always convert values to UTC. If a date is passed in without timezone info, it is assumed to be UTC.
start – the date-time the policy is active.
expiry (str) – the date-time the policy expires.
permission (str) – the permissions for the acl policy.
-
as_dict
(keep_readonly=True, key_transformer=<function attribute_transformer>, **kwargs)¶ Return a dict that can be JSONify using json.dump.
Advanced usage might optionally use a callback as parameter:
Key is the attribute name used in Python. Attr_desc is a dict of metadata. Currently contains ‘type’ with the msrest type and ‘key’ with the RestAPI encoded key. Value is the current value in this object.
The string returned will be used to serialize the key. If the return type is a list, this is considered hierarchical result dict.
See the three examples in this file:
attribute_transformer
full_restapi_key_transformer
last_restapi_key_transformer
If you want XML serialization, you can pass the kwargs is_xml=True.
- Parameters
key_transformer (function) – A key transformer function.
- Returns
A dict JSON compatible object
- Return type
-
classmethod
deserialize
(data, content_type=None)¶ Parse a str using the RestAPI syntax and return a model.
-
classmethod
enable_additional_properties_sending
()¶
-
classmethod
from_dict
(data, key_extractors=None, content_type=None)¶ Parse a dict using given key extractor return a model.
By default consider key extractors (rest_key_case_insensitive_extractor, attribute_key_case_insensitive_extractor and last_rest_key_case_insensitive_extractor)
-
classmethod
is_xml_model
()¶
-
serialize
(keep_readonly=False, **kwargs)¶ Return the JSON that would be sent to azure from this model.
This is an alias to as_dict(full_restapi_key_transformer, keep_readonly=False).
If you want XML serialization, you can pass the kwargs is_xml=True.
-
class
azure.storage.filedatalake.
AccountSasPermissions
(read=False, write=False, delete=False, list=False, create=False)[source]¶ -
classmethod
from_string
(permission)[source]¶ Create AccountSasPermissions from a string.
To specify read, write, delete, etc. permissions you need only to include the first letter of the word in the string. E.g. for read and write permissions you would provide a string “rw”.
- Parameters
permission (str) – Specify permissions in the string with the first letter of the word.
- Returns
An AccountSasPermissions object
- Return type
AccountSasPermissions
-
classmethod
-
class
azure.storage.filedatalake.
AnalyticsLogging
(**kwargs)[source]¶ Azure Analytics Logging settings.
- Keyword Arguments
version (str) – The version of Storage Analytics to configure. The default value is 1.0.
delete (bool) – Indicates whether all delete requests should be logged. The default value is False.
read (bool) – Indicates whether all read requests should be logged. The default value is False.
write (bool) – Indicates whether all write requests should be logged. The default value is False.
retention_policy (RetentionPolicy) – Determines how long the associated data should persist. If not specified the retention policy will be disabled by default.
version – The version of Storage Analytics to configure. Required.
delete – Indicates whether all delete requests should be logged. Required.
read – Indicates whether all read requests should be logged. Required.
write – Indicates whether all write requests should be logged. Required.
retention_policy – the retention policy which determines how long the associated data should persist. Required.
-
as_dict
(keep_readonly=True, key_transformer=<function attribute_transformer>, **kwargs)¶ Return a dict that can be JSONify using json.dump.
Advanced usage might optionally use a callback as parameter:
Key is the attribute name used in Python. Attr_desc is a dict of metadata. Currently contains ‘type’ with the msrest type and ‘key’ with the RestAPI encoded key. Value is the current value in this object.
The string returned will be used to serialize the key. If the return type is a list, this is considered hierarchical result dict.
See the three examples in this file:
attribute_transformer
full_restapi_key_transformer
last_restapi_key_transformer
If you want XML serialization, you can pass the kwargs is_xml=True.
- Parameters
key_transformer (function) – A key transformer function.
- Returns
A dict JSON compatible object
- Return type
-
classmethod
deserialize
(data, content_type=None)¶ Parse a str using the RestAPI syntax and return a model.
-
classmethod
enable_additional_properties_sending
()¶
-
classmethod
from_dict
(data, key_extractors=None, content_type=None)¶ Parse a dict using given key extractor return a model.
By default consider key extractors (rest_key_case_insensitive_extractor, attribute_key_case_insensitive_extractor and last_rest_key_case_insensitive_extractor)
-
classmethod
is_xml_model
()¶
-
serialize
(keep_readonly=False, **kwargs)¶ Return the JSON that would be sent to azure from this model.
This is an alias to as_dict(full_restapi_key_transformer, keep_readonly=False).
If you want XML serialization, you can pass the kwargs is_xml=True.
-
class
azure.storage.filedatalake.
ArrowDialect
(type, **kwargs)[source]¶ field of an arrow schema.
All required parameters must be populated in order to send to Azure.
- Parameters
type (str) – Required.
- Keyword Arguments
-
as_dict
(keep_readonly=True, key_transformer=<function attribute_transformer>, **kwargs)¶ Return a dict that can be JSONify using json.dump.
Advanced usage might optionally use a callback as parameter:
Key is the attribute name used in Python. Attr_desc is a dict of metadata. Currently contains ‘type’ with the msrest type and ‘key’ with the RestAPI encoded key. Value is the current value in this object.
The string returned will be used to serialize the key. If the return type is a list, this is considered hierarchical result dict.
See the three examples in this file:
attribute_transformer
full_restapi_key_transformer
last_restapi_key_transformer
If you want XML serialization, you can pass the kwargs is_xml=True.
- Parameters
key_transformer (function) – A key transformer function.
- Returns
A dict JSON compatible object
- Return type
-
classmethod
deserialize
(data, content_type=None)¶ Parse a str using the RestAPI syntax and return a model.
-
classmethod
enable_additional_properties_sending
()¶
-
classmethod
from_dict
(data, key_extractors=None, content_type=None)¶ Parse a dict using given key extractor return a model.
By default consider key extractors (rest_key_case_insensitive_extractor, attribute_key_case_insensitive_extractor and last_rest_key_case_insensitive_extractor)
-
classmethod
is_xml_model
()¶
-
serialize
(keep_readonly=False, **kwargs)¶ Return the JSON that would be sent to azure from this model.
This is an alias to as_dict(full_restapi_key_transformer, keep_readonly=False).
If you want XML serialization, you can pass the kwargs is_xml=True.
-
class
azure.storage.filedatalake.
ArrowType
(value)[source]¶ An enumeration.
-
BOOL
= 'bool'¶
-
DECIMAL
= 'decimal'¶
-
DOUBLE
= 'double'¶
-
INT64
= 'int64'¶
-
STRING
= 'string'¶
-
TIMESTAMP_MS
= 'timestamp[ms]'¶
-
-
class
azure.storage.filedatalake.
ContentSettings
(**kwargs)[source]¶ The content settings of a file or directory.
- Variables
content_type (str) – The content type specified for the file or directory. If no content type was specified, the default content type is application/octet-stream.
content_encoding (str) – If the content_encoding has previously been set for the file, that value is stored.
content_language (str) – If the content_language has previously been set for the file, that value is stored.
content_disposition (str) – content_disposition conveys additional information about how to process the response payload, and also can be used to attach additional metadata. If content_disposition has previously been set for the file, that value is stored.
cache_control (str) – If the cache_control has previously been set for the file, that value is stored.
content_md5 (bytearray) – If the content_md5 has been set for the file, this response header is stored so that the client can check for message content integrity.
- Keyword Arguments
content_type (str) – The content type specified for the file or directory. If no content type was specified, the default content type is application/octet-stream.
content_encoding (str) – If the content_encoding has previously been set for the file, that value is stored.
content_language (str) – If the content_language has previously been set for the file, that value is stored.
content_disposition (str) – content_disposition conveys additional information about how to process the response payload, and also can be used to attach additional metadata. If content_disposition has previously been set for the file, that value is stored.
cache_control (str) – If the cache_control has previously been set for the file, that value is stored.
content_md5 (bytearray) – If the content_md5 has been set for the file, this response header is stored so that the client can check for message content integrity.
-
get
(key, default=None)¶
-
has_key
(k)¶
-
items
()¶
-
keys
()¶
-
update
(*args, **kwargs)¶
-
values
()¶
-
class
azure.storage.filedatalake.
CorsRule
(allowed_origins, allowed_methods, **kwargs)[source]¶ CORS is an HTTP feature that enables a web application running under one domain to access resources in another domain. Web browsers implement a security restriction known as same-origin policy that prevents a web page from calling APIs in a different domain; CORS provides a secure way to allow one domain (the origin domain) to call APIs in another domain.
- Parameters
allowed_origins (list(str)) – A list of origin domains that will be allowed via CORS, or “*” to allow all domains. The list of must contain at least one entry. Limited to 64 origin domains. Each allowed origin can have up to 256 characters.
allowed_methods (list(str)) – A list of HTTP methods that are allowed to be executed by the origin. The list of must contain at least one entry. For Azure Storage, permitted methods are DELETE, GET, HEAD, MERGE, POST, OPTIONS or PUT.
- Keyword Arguments
allowed_headers (str) – Defaults to an empty list. A list of headers allowed to be part of the cross-origin request. Limited to 64 defined headers and 2 prefixed headers. Each header can be up to 256 characters.
exposed_headers (str) – Defaults to an empty list. A list of response headers to expose to CORS clients. Limited to 64 defined headers and two prefixed headers. Each header can be up to 256 characters.
max_age_in_seconds (int) – The number of seconds that the client/browser should cache a preflight response.
allowed_origins (str) – The origin domains that are permitted to make a request against the storage service via CORS. The origin domain is the domain from which the request originates. Note that the origin must be an exact case-sensitive match with the origin that the user age sends to the service. You can also use the wildcard character ‘*’ to allow all origin domains to make requests via CORS. Required.
allowed_methods (str) – The methods (HTTP request verbs) that the origin domain may use for a CORS request. (comma separated). Required.
allowed_headers – the request headers that the origin domain may specify on the CORS request. Required.
exposed_headers – The response headers that may be sent in the response to the CORS request and exposed by the browser to the request issuer. Required.
max_age_in_seconds – The maximum amount time that a browser should cache the preflight OPTIONS request. Required.
-
as_dict
(keep_readonly=True, key_transformer=<function attribute_transformer>, **kwargs)¶ Return a dict that can be JSONify using json.dump.
Advanced usage might optionally use a callback as parameter:
Key is the attribute name used in Python. Attr_desc is a dict of metadata. Currently contains ‘type’ with the msrest type and ‘key’ with the RestAPI encoded key. Value is the current value in this object.
The string returned will be used to serialize the key. If the return type is a list, this is considered hierarchical result dict.
See the three examples in this file:
attribute_transformer
full_restapi_key_transformer
last_restapi_key_transformer
If you want XML serialization, you can pass the kwargs is_xml=True.
- Parameters
key_transformer (function) – A key transformer function.
- Returns
A dict JSON compatible object
- Return type
-
classmethod
deserialize
(data, content_type=None)¶ Parse a str using the RestAPI syntax and return a model.
-
classmethod
enable_additional_properties_sending
()¶
-
classmethod
from_dict
(data, key_extractors=None, content_type=None)¶ Parse a dict using given key extractor return a model.
By default consider key extractors (rest_key_case_insensitive_extractor, attribute_key_case_insensitive_extractor and last_rest_key_case_insensitive_extractor)
-
classmethod
is_xml_model
()¶
-
serialize
(keep_readonly=False, **kwargs)¶ Return the JSON that would be sent to azure from this model.
This is an alias to as_dict(full_restapi_key_transformer, keep_readonly=False).
If you want XML serialization, you can pass the kwargs is_xml=True.
-
class
azure.storage.filedatalake.
CustomerProvidedEncryptionKey
(key_value, key_hash)[source]¶ All data in Azure Storage is encrypted at-rest using an account-level encryption key. In versions 2021-06-08 and newer, you can manage the key used to encrypt file contents and application metadata per-file by providing an AES-256 encryption key in requests to the storage service.
When you use a customer-provided key, Azure Storage does not manage or persist your key. When writing data to a file, the provided key is used to encrypt your data before writing it to disk. A SHA-256 hash of the encryption key is written alongside the file contents, and is used to verify that all subsequent operations against the file use the same encryption key. This hash cannot be used to retrieve the encryption key or decrypt the contents of the file. When reading a file, the provided key is used to decrypt your data after reading it from disk. In both cases, the provided encryption key is securely discarded as soon as the encryption or decryption process completes.
-
class
azure.storage.filedatalake.
DataLakeDirectoryClient
(account_url: str, file_system_name: str, directory_name: str, credential: Optional[Union[str, Dict[str, str], AzureNamedKeyCredential, AzureSasCredential, "TokenCredential"]] # pylint: disable=line-too-long = None, **kwargs: Any)[source]¶ A client to interact with the DataLake directory, even if the directory may not yet exist.
For operations relating to a specific subdirectory or file under the directory, a directory client or file client can be retrieved using the
get_sub_directory_client()
orget_file_client()
functions.- Variables
url (str) – The full endpoint URL to the file system, including SAS token if used.
primary_endpoint (str) – The full primary endpoint URL.
primary_hostname (str) – The hostname of the primary endpoint.
- Parameters
account_url (str) – The URI to the storage account.
file_system_name (str) – The file system for the directory or files.
directory_name (str) – The whole path of the directory. eg. {directory under file system}/{directory to interact with}
credential – The credentials with which to authenticate. This is optional if the account URL already has a SAS token. The value can be a SAS token string, an instance of a AzureSasCredential or AzureNamedKeyCredential from azure.core.credentials, an account shared access key, or an instance of a TokenCredentials class from azure.identity. If the resource URI already contains a SAS token, this will be ignored in favor of an explicit credential - except in the case of AzureSasCredential, where the conflicting SAS tokens will raise a ValueError. If using an instance of AzureNamedKeyCredential, “name” should be the storage account name, and “key” should be the storage account key.
- Keyword Arguments
api_version (str) – The Storage API version to use for requests. Default value is the most recent service version that is compatible with the current SDK. Setting to an older version may result in reduced feature compatibility.
Example:
from azure.storage.filedatalake import DataLakeDirectoryClient DataLakeDirectoryClient.from_connection_string(connection_string, "myfilesystem", "mydirectory")
-
acquire_lease
(lease_duration: Optional[int] = - 1, lease_id: Optional[str] = None, **kwargs) → azure.storage.filedatalake._data_lake_lease.DataLakeLeaseClient¶ Requests a new lease. If the file or directory does not have an active lease, the DataLake service creates a lease on the file/directory and returns a new lease ID.
- Parameters
lease_duration (int) – Specifies the duration of the lease, in seconds, or negative one (-1) for a lease that never expires. A non-infinite lease can be between 15 and 60 seconds. A lease duration cannot be changed using renew or change. Default is -1 (infinite lease).
lease_id (str) – Proposed lease ID, in a GUID string format. The DataLake service returns 400 (Invalid request) if the proposed lease ID is not in the correct format.
- Keyword Arguments
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
A DataLakeLeaseClient object, that can be run in a context manager.
- Return type
-
close
() → None¶ This method is to close the sockets opened by the client. It need not be used when using with a context manager.
-
create_directory
(metadata: Optional[Dict[str, str]] = None, **kwargs) → Dict[str, Union[str, datetime]][source]¶ Create a new directory.
- Parameters
metadata (dict(str, str)) – Name-value pairs associated with the file as metadata.
- Keyword Arguments
content_settings (ContentSettings) – ContentSettings object used to set path properties.
lease (DataLakeLeaseClient or str) – Required if the file has an active lease. Value can be a DataLakeLeaseClient object or the lease ID as a string.
umask (str) – Optional and only valid if Hierarchical Namespace is enabled for the account. When creating a file or directory and the parent folder does not have a default ACL, the umask restricts the permissions of the file or directory to be created. The resulting permission is given by p & ^u, where p is the permission and u is the umask. For example, if p is 0777 and u is 0057, then the resulting permission is 0720. The default permission is 0777 for a directory and 0666 for a file. The default umask is 0027. The umask must be specified in 4-digit octal notation (e.g. 0766).
owner (str) – The owner of the file or directory.
group (str) – The owning group of the file or directory.
acl (str) – Sets POSIX access control rights on files and directories. The value is a comma-separated list of access control entries. Each access control entry (ACE) consists of a scope, a type, a user or group identifier, and permissions in the format “[scope:][type]:[id]:[permissions]”.
lease_id (str) – Proposed lease ID, in a GUID string format. The DataLake service returns 400 (Invalid request) if the proposed lease ID is not in the correct format.
lease_duration (int) – Specifies the duration of the lease, in seconds, or negative one (-1) for a lease that never expires. A non-infinite lease can be between 15 and 60 seconds. A lease duration cannot be changed using renew or change.
permissions (str) – Optional and only valid if Hierarchical Namespace is enabled for the account. Sets POSIX access permissions for the file owner, the file owning group, and others. Each class may be granted read, write, or execute permission. The sticky bit is also supported. Both symbolic (rwxrw-rw-) and 4-digit octal notation (e.g. 0766) are supported.
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
cpk (CustomerProvidedEncryptionKey) – Encrypts the data on the service-side with the given key. Use of customer-provided keys must be done over HTTPS.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
A dictionary of response headers.
- Return type
Example:
directory_client.create_directory()
-
create_file
(file: Union[azure.storage.filedatalake._models.FileProperties, str], **kwargs) → azure.storage.filedatalake._data_lake_file_client.DataLakeFileClient[source]¶ Create a new file and return the file client to be interacted with.
- Parameters
file (str or FileProperties) – The file with which to interact. This can either be the name of the file, or an instance of FileProperties.
- Keyword Arguments
content_settings (ContentSettings) – ContentSettings object used to set path properties.
metadata – Name-value pairs associated with the file as metadata.
lease (DataLakeLeaseClient or str) – Required if the file has an active lease. Value can be a DataLakeLeaseClient object or the lease ID as a string.
umask (str) – Optional and only valid if Hierarchical Namespace is enabled for the account. When creating a file or directory and the parent folder does not have a default ACL, the umask restricts the permissions of the file or directory to be created. The resulting permission is given by p & ^u, where p is the permission and u is the umask. For example, if p is 0777 and u is 0057, then the resulting permission is 0720. The default permission is 0777 for a directory and 0666 for a file. The default umask is 0027. The umask must be specified in 4-digit octal notation (e.g. 0766).
owner (str) – The owner of the file or directory.
group (str) – The owning group of the file or directory.
acl (str) – Sets POSIX access control rights on files and directories. The value is a comma-separated list of access control entries. Each access control entry (ACE) consists of a scope, a type, a user or group identifier, and permissions in the format “[scope:][type]:[id]:[permissions]”.
lease_id (str) – Proposed lease ID, in a GUID string format. The DataLake service returns 400 (Invalid request) if the proposed lease ID is not in the correct format.
lease_duration (int) – Specifies the duration of the lease, in seconds, or negative one (-1) for a lease that never expires. A non-infinite lease can be between 15 and 60 seconds. A lease duration cannot be changed using renew or change.
expires_on (datetime or int) – The time to set the file to expiry. If the type of expires_on is an int, expiration time will be set as the number of milliseconds elapsed from creation time. If the type of expires_on is datetime, expiration time will be set absolute to the time provided. If no time zone info is provided, this will be interpreted as UTC.
permissions (str) – Optional and only valid if Hierarchical Namespace is enabled for the account. Sets POSIX access permissions for the file owner, the file owning group, and others. Each class may be granted read, write, or execute permission. The sticky bit is also supported. Both symbolic (rwxrw-rw-) and 4-digit octal notation (e.g. 0766) are supported.
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
cpk (CustomerProvidedEncryptionKey) – Encrypts the data on the service-side with the given key. Use of customer-provided keys must be done over HTTPS.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
DataLakeFileClient
-
create_sub_directory
(sub_directory: Union[azure.storage.filedatalake._models.DirectoryProperties, str], metadata: Optional[Dict[str, str]] = None, **kwargs) → azure.storage.filedatalake._data_lake_directory_client.DataLakeDirectoryClient[source]¶ Create a subdirectory and return the subdirectory client to be interacted with.
- Parameters
sub_directory (str or DirectoryProperties) – The directory with which to interact. This can either be the name of the directory, or an instance of DirectoryProperties.
metadata (dict(str, str)) – Name-value pairs associated with the file as metadata.
- Keyword Arguments
content_settings (ContentSettings) – ContentSettings object used to set path properties.
lease (DataLakeLeaseClient or str) – Required if the file has an active lease. Value can be a DataLakeLeaseClient object or the lease ID as a string.
umask (str) – Optional and only valid if Hierarchical Namespace is enabled for the account. When creating a file or directory and the parent folder does not have a default ACL, the umask restricts the permissions of the file or directory to be created. The resulting permission is given by p & ^u, where p is the permission and u is the umask. For example, if p is 0777 and u is 0057, then the resulting permission is 0720. The default permission is 0777 for a directory and 0666 for a file. The default umask is 0027. The umask must be specified in 4-digit octal notation (e.g. 0766).
owner (str) – The owner of the file or directory.
group (str) – The owning group of the file or directory.
acl (str) – Sets POSIX access control rights on files and directories. The value is a comma-separated list of access control entries. Each access control entry (ACE) consists of a scope, a type, a user or group identifier, and permissions in the format “[scope:][type]:[id]:[permissions]”.
lease_id (str) – Proposed lease ID, in a GUID string format. The DataLake service returns 400 (Invalid request) if the proposed lease ID is not in the correct format.
lease_duration (int) – Specifies the duration of the lease, in seconds, or negative one (-1) for a lease that never expires. A non-infinite lease can be between 15 and 60 seconds. A lease duration cannot be changed using renew or change.
permissions (str) – Optional and only valid if Hierarchical Namespace is enabled for the account. Sets POSIX access permissions for the file owner, the file owning group, and others. Each class may be granted read, write, or execute permission. The sticky bit is also supported. Both symbolic (rwxrw-rw-) and 4-digit octal notation (e.g. 0766) are supported.
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
cpk (CustomerProvidedEncryptionKey) – Encrypts the data on the service-side with the given key. Use of customer-provided keys must be done over HTTPS.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
DataLakeDirectoryClient for the subdirectory.
-
delete_directory
(**kwargs) → None[source]¶ Marks the specified directory for deletion.
- Keyword Arguments
lease (DataLakeLeaseClient or str) – Required if the file has an active lease. Value can be a LeaseClient object or the lease ID as a string.
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
None
Example:
new_directory.delete_directory()
-
delete_sub_directory
(sub_directory: Union[azure.storage.filedatalake._models.DirectoryProperties, str], **kwargs) → azure.storage.filedatalake._data_lake_directory_client.DataLakeDirectoryClient[source]¶ Marks the specified subdirectory for deletion.
- Parameters
sub_directory (str or DirectoryProperties) – The directory with which to interact. This can either be the name of the directory, or an instance of DirectoryProperties.
- Keyword Arguments
lease (DataLakeLeaseClient or str) – Required if the file has an active lease. Value can be a LeaseClient object or the lease ID as a string.
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
DataLakeDirectoryClient for the subdirectory
-
exists
(**kwargs: Any) → bool[source]¶ Returns True if a directory exists and returns False otherwise.
- Keyword Arguments
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
boolean
-
classmethod
from_connection_string
(conn_str: str, file_system_name: str, directory_name: str, credential: Optional[Union[str, Dict[str, str], AzureNamedKeyCredential, AzureSasCredential, "TokenCredential"]] # pylint: disable=line-too-long = None, **kwargs: Any) → ClassType[source]¶ Create DataLakeDirectoryClient from a Connection String.
- Parameters
conn_str (str) – A connection string to an Azure Storage account.
file_system_name (str) – The name of file system to interact with.
directory_name (str) – The name of directory to interact with. The directory is under file system.
credential – The credentials with which to authenticate. This is optional if the account URL already has a SAS token, or the connection string already has shared access key values. The value can be a SAS token string, an instance of a AzureSasCredential or AzureNamedKeyCredential from azure.core.credentials, an account shared access key, or an instance of a TokenCredentials class from azure.identity. Credentials provided here will take precedence over those in the connection string. If using an instance of AzureNamedKeyCredential, “name” should be the storage account name, and “key” should be the storage account key.
- Returns
a DataLakeDirectoryClient
- Return type
-
get_access_control
(upn: Optional[bool] = None, **kwargs) → Dict[str, Any]¶ - Parameters
upn (bool) – Optional. Valid only when Hierarchical Namespace is enabled for the account. If “true”, the user identity values returned in the x-ms-owner, x-ms-group, and x-ms-acl response headers will be transformed from Azure Active Directory Object IDs to User Principal Names. If “false”, the values will be returned as Azure Active Directory Object IDs. The default value is false. Note that group and application Object IDs are not translated because they do not have unique friendly names.
- Keyword Arguments
lease (DataLakeLeaseClient or str) – Required if the file/directory has an active lease. Value can be a LeaseClient object or the lease ID as a string.
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
timeout (int) – The timeout parameter is expressed in seconds.
- Keyword
response dict.
-
get_directory_properties
(**kwargs: Any) → azure.storage.filedatalake._models.DirectoryProperties[source]¶ Returns all user-defined metadata, standard HTTP properties, and system properties for the directory. It does not return the content of the directory.
- Keyword Arguments
lease (DataLakeLeaseClient or str) – Required if the directory or file has an active lease. Value can be a DataLakeLeaseClient object or the lease ID as a string.
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
cpk (CustomerProvidedEncryptionKey) – Decrypts the data on the service-side with the given key. Use of customer-provided keys must be done over HTTPS. Required if the directory was created with a customer-provided key.
timeout (int) – The timeout parameter is expressed in seconds.
- Return type
Example:
props = new_directory.get_directory_properties()
-
get_file_client
(file: Union[azure.storage.filedatalake._models.FileProperties, str]) → azure.storage.filedatalake._data_lake_file_client.DataLakeFileClient[source]¶ Get a client to interact with the specified file.
The file need not already exist.
- Parameters
file (str or FileProperties) – The file with which to interact. This can either be the name of the file, or an instance of FileProperties. eg. directory/subdirectory/file
- Returns
A DataLakeFileClient.
- Return type
-
get_sub_directory_client
(sub_directory: Union[azure.storage.filedatalake._models.DirectoryProperties, str]) → azure.storage.filedatalake._data_lake_directory_client.DataLakeDirectoryClient[source]¶ Get a client to interact with the specified subdirectory of the current directory.
The sub subdirectory need not already exist.
- Parameters
sub_directory (str or DirectoryProperties) – The directory with which to interact. This can either be the name of the directory, or an instance of DirectoryProperties.
- Returns
A DataLakeDirectoryClient.
- Return type
-
remove_access_control_recursive
(acl: str, **kwargs: Any) → azure.storage.filedatalake._models.AccessControlChangeResult¶ Removes the Access Control on a path and sub-paths.
- Parameters
acl (str) – Removes POSIX access control rights on files and directories. The value is a comma-separated list of access control entries. Each access control entry (ACE) consists of a scope, a type, and a user or group identifier in the format “[scope:][type]:[id]”.
- Keyword Arguments
progress_hook (func(AccessControlChanges)) – Callback where the caller can track progress of the operation as well as collect paths that failed to change Access Control.
continuation_token (str) – Optional continuation token that can be used to resume previously stopped operation.
batch_size (int) – Optional. If data set size exceeds batch size then operation will be split into multiple requests so that progress can be tracked. Batch size should be between 1 and 2000. The default when unspecified is 2000.
max_batches (int) – Optional. Defines maximum number of batches that single change Access Control operation can execute. If maximum is reached before all sub-paths are processed then, continuation token can be used to resume operation. Empty value indicates that maximum number of batches in unbound and operation continues till end.
continue_on_failure (bool) – If set to False, the operation will terminate quickly on encountering user errors (4XX). If True, the operation will ignore user errors and proceed with the operation on other sub-entities of the directory. Continuation token will only be returned when continue_on_failure is True in case of user errors. If not set the default value is False for this.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
A summary of the recursive operations, including the count of successes and failures, as well as a continuation token in case the operation was terminated prematurely.
- Return type
- Raises
AzureError – User can restart the operation using continuation_token field of AzureError if the token is available.
-
rename_directory
(new_name: str, **kwargs: Any) → azure.storage.filedatalake._data_lake_directory_client.DataLakeDirectoryClient[source]¶ Rename the source directory.
- Parameters
new_name (str) – the new directory name the user want to rename to. The value must have the following format: “{filesystem}/{directory}/{subdirectory}”.
- Keyword Arguments
source_lease (DataLakeLeaseClient or str) – A lease ID for the source path. If specified, the source path must have an active lease and the leaase ID must match.
lease (DataLakeLeaseClient or str) – Required if the file/directory has an active lease. Value can be a LeaseClient object or the lease ID as a string.
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
source_if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
source_if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
source_etag (str) – The source ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
source_match_condition (MatchConditions) – The source match condition to use upon the etag.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
DataLakeDirectoryClient
Example:
new_dir_name = "testdir2" print("Renaming the directory named '{}' to '{}'.".format(dir_name, new_dir_name)) new_directory = directory_client\ .rename_directory(new_name=directory_client.file_system_name + '/' + new_dir_name)
-
set_access_control
(owner: Optional[str] = None, group: Optional[str] = None, permissions: Optional[str] = None, acl: Optional[str] = None, **kwargs) → Dict[str, Union[str, datetime.datetime]]¶ Set the owner, group, permissions, or access control list for a path.
- Parameters
owner (str) – Optional. The owner of the file or directory.
group (str) – Optional. The owning group of the file or directory.
permissions (str) – Optional and only valid if Hierarchical Namespace is enabled for the account. Sets POSIX access permissions for the file owner, the file owning group, and others. Each class may be granted read, write, or execute permission. The sticky bit is also supported. Both symbolic (rwxrw-rw-) and 4-digit octal notation (e.g. 0766) are supported. permissions and acl are mutually exclusive.
acl (str) – Sets POSIX access control rights on files and directories. The value is a comma-separated list of access control entries. Each access control entry (ACE) consists of a scope, a type, a user or group identifier, and permissions in the format “[scope:][type]:[id]:[permissions]”. permissions and acl are mutually exclusive.
- Keyword Arguments
lease (DataLakeLeaseClient or str) – Required if the file/directory has an active lease. Value can be a LeaseClient object or the lease ID as a string.
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
timeout (int) – The timeout parameter is expressed in seconds.
- Keyword
response dict (Etag and last modified).
-
set_access_control_recursive
(acl: str, **kwargs: Any) → azure.storage.filedatalake._models.AccessControlChangeResult¶ Sets the Access Control on a path and sub-paths.
- Parameters
acl (str) – Sets POSIX access control rights on files and directories. The value is a comma-separated list of access control entries. Each access control entry (ACE) consists of a scope, a type, a user or group identifier, and permissions in the format “[scope:][type]:[id]:[permissions]”.
- Keyword Arguments
progress_hook (func(AccessControlChanges)) – Callback where the caller can track progress of the operation as well as collect paths that failed to change Access Control.
continuation_token (str) – Optional continuation token that can be used to resume previously stopped operation.
batch_size (int) – Optional. If data set size exceeds batch size then operation will be split into multiple requests so that progress can be tracked. Batch size should be between 1 and 2000. The default when unspecified is 2000.
max_batches (int) – Optional. Defines maximum number of batches that single change Access Control operation can execute. If maximum is reached before all sub-paths are processed, then continuation token can be used to resume operation. Empty value indicates that maximum number of batches in unbound and operation continues till end.
continue_on_failure (bool) – If set to False, the operation will terminate quickly on encountering user errors (4XX). If True, the operation will ignore user errors and proceed with the operation on other sub-entities of the directory. Continuation token will only be returned when continue_on_failure is True in case of user errors. If not set the default value is False for this.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
A summary of the recursive operations, including the count of successes and failures, as well as a continuation token in case the operation was terminated prematurely.
- Return type
- Raises
AzureError – User can restart the operation using continuation_token field of AzureError if the token is available.
-
set_http_headers
(content_settings: Optional[ContentSettings] = None, **kwargs) → Dict[str, Any]¶ Sets system properties on the file or directory.
If one property is set for the content_settings, all properties will be overriden.
- Parameters
content_settings (ContentSettings) – ContentSettings object used to set file/directory properties.
- Keyword Arguments
lease (DataLakeLeaseClient or str) – If specified, set_file_system_metadata only succeeds if the file system’s lease is active and matches this ID.
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
file/directory-updated property dict (Etag and last modified)
- Return type
Dict[str, Any]
-
set_metadata
(metadata: Dict[str, str], **kwargs) → Dict[str, Union[str, datetime.datetime]]¶ Sets one or more user-defined name-value pairs for the specified file system. Each call to this operation replaces all existing metadata attached to the file system. To remove all metadata from the file system, call this operation with no metadata dict.
- Parameters
metadata (Dict[str, str]) – A dict containing name-value pairs to associate with the file system as metadata. Example: {‘category’:’test’}
- Keyword Arguments
lease (DataLakeLeaseClient or str) – If specified, set_file_system_metadata only succeeds if the file system’s lease is active and matches this ID.
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
cpk (CustomerProvidedEncryptionKey) – Encrypts the data on the service-side with the given key. Use of customer-provided keys must be done over HTTPS.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
file system-updated property dict (Etag and last modified).
-
update_access_control_recursive
(acl: str, **kwargs: Any) → azure.storage.filedatalake._models.AccessControlChangeResult¶ Modifies the Access Control on a path and sub-paths.
- Parameters
acl (str) – Modifies POSIX access control rights on files and directories. The value is a comma-separated list of access control entries. Each access control entry (ACE) consists of a scope, a type, a user or group identifier, and permissions in the format “[scope:][type]:[id]:[permissions]”.
- Keyword Arguments
progress_hook (func(AccessControlChanges)) – Callback where the caller can track progress of the operation as well as collect paths that failed to change Access Control.
continuation_token (str) – Optional continuation token that can be used to resume previously stopped operation.
batch_size (int) – Optional. If data set size exceeds batch size then operation will be split into multiple requests so that progress can be tracked. Batch size should be between 1 and 2000. The default when unspecified is 2000.
max_batches (int) – Optional. Defines maximum number of batches that single change Access Control operation can execute. If maximum is reached before all sub-paths are processed, then continuation token can be used to resume operation. Empty value indicates that maximum number of batches in unbound and operation continues till end.
continue_on_failure (bool) – If set to False, the operation will terminate quickly on encountering user errors (4XX). If True, the operation will ignore user errors and proceed with the operation on other sub-entities of the directory. Continuation token will only be returned when continue_on_failure is True in case of user errors. If not set the default value is False for this.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
A summary of the recursive operations, including the count of successes and failures, as well as a continuation token in case the operation was terminated prematurely.
- Return type
- Raises
AzureError – User can restart the operation using continuation_token field of AzureError if the token is available.
-
property
location_mode
¶ The location mode that the client is currently using.
By default this will be “primary”. Options include “primary” and “secondary”.
- Type
-
property
secondary_endpoint
¶ The full secondary endpoint URL if configured.
If not available a ValueError will be raised. To explicitly specify a secondary hostname, use the optional secondary_hostname keyword argument on instantiation.
- Type
- Raises
-
property
secondary_hostname
¶ The hostname of the secondary endpoint.
If not available this will be None. To explicitly specify a secondary hostname, use the optional secondary_hostname keyword argument on instantiation.
-
property
url
¶ The full endpoint URL to this entity, including SAS token if used.
This could be either the primary endpoint, or the secondary endpoint depending on the current
location_mode()
.
-
class
azure.storage.filedatalake.
DataLakeFileClient
(account_url: str, file_system_name: str, file_path: str, credential: Optional[Union[str, Dict[str, str], AzureNamedKeyCredential, AzureSasCredential, "TokenCredential"]] # pylint: disable=line-too-long = None, **kwargs: Any)[source]¶ A client to interact with the DataLake file, even if the file may not yet exist.
- Variables
url (str) – The full endpoint URL to the file system, including SAS token if used.
primary_endpoint (str) – The full primary endpoint URL.
primary_hostname (str) – The hostname of the primary endpoint.
- Parameters
account_url (str) – The URI to the storage account.
file_system_name (str) – The file system for the directory or files.
file_path (str) – The whole file path, so that to interact with a specific file. eg. “{directory}/{subdirectory}/{file}”
credential – The credentials with which to authenticate. This is optional if the account URL already has a SAS token. The value can be a SAS token string, an instance of a AzureSasCredential or AzureNamedKeyCredential from azure.core.credentials, an account shared access key, or an instance of a TokenCredentials class from azure.identity. If the resource URI already contains a SAS token, this will be ignored in favor of an explicit credential - except in the case of AzureSasCredential, where the conflicting SAS tokens will raise a ValueError. If using an instance of AzureNamedKeyCredential, “name” should be the storage account name, and “key” should be the storage account key.
- Keyword Arguments
api_version (str) – The Storage API version to use for requests. Default value is the most recent service version that is compatible with the current SDK. Setting to an older version may result in reduced feature compatibility.
Example:
from azure.storage.filedatalake import DataLakeFileClient DataLakeFileClient.from_connection_string(connection_string, "myfilesystem", "mydirectory", "myfile")
-
acquire_lease
(lease_duration: Optional[int] = - 1, lease_id: Optional[str] = None, **kwargs) → azure.storage.filedatalake._data_lake_lease.DataLakeLeaseClient¶ Requests a new lease. If the file or directory does not have an active lease, the DataLake service creates a lease on the file/directory and returns a new lease ID.
- Parameters
lease_duration (int) – Specifies the duration of the lease, in seconds, or negative one (-1) for a lease that never expires. A non-infinite lease can be between 15 and 60 seconds. A lease duration cannot be changed using renew or change. Default is -1 (infinite lease).
lease_id (str) – Proposed lease ID, in a GUID string format. The DataLake service returns 400 (Invalid request) if the proposed lease ID is not in the correct format.
- Keyword Arguments
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
A DataLakeLeaseClient object, that can be run in a context manager.
- Return type
-
append_data
(data: Union[bytes, str, Iterable[AnyStr], IO[AnyStr]], offset: int, length: Optional[int] = None, **kwargs) → Dict[str, Union[str, datetime, int]][source]¶ Append data to the file.
- Parameters
data – Content to be appended to file
offset – start position of the data to be appended to.
length – Size of the data in bytes.
- Keyword Arguments
flush (bool) – If true, will commit the data after it is appended.
validate_content (bool) – If true, calculates an MD5 hash of the block content. The storage service checks the hash of the content that has arrived with the hash that was sent. This is primarily valuable for detecting bitflips on the wire if using http instead of https as https (the default) will already validate. Note that this MD5 hash is not stored with the file.
lease (DataLakeLeaseClient or str) – Required if the file has an active lease. Value can be a DataLakeLeaseClient object or the lease ID as a string.
cpk (CustomerProvidedEncryptionKey) – Encrypts the data on the service-side with the given key. Use of customer-provided keys must be done over HTTPS.
- Returns
dict of the response header
Example:
file_client.append_data(data=file_content[2048:3072], offset=2048, length=1024)
-
close
() → None¶ This method is to close the sockets opened by the client. It need not be used when using with a context manager.
-
create_file
(content_settings: Optional[ContentSettings] = None, metadata: Optional[Dict[str, str]] = None, **kwargs) → Dict[str, Union[str, datetime]][source]¶ Create a new file.
- Parameters
content_settings (ContentSettings) – ContentSettings object used to set path properties.
metadata (dict(str, str)) – Name-value pairs associated with the file as metadata.
- Keyword Arguments
lease (DataLakeLeaseClient or str) – Required if the file has an active lease. Value can be a DataLakeLeaseClient object or the lease ID as a string.
umask (str) – Optional and only valid if Hierarchical Namespace is enabled for the account. When creating a file or directory and the parent folder does not have a default ACL, the umask restricts the permissions of the file or directory to be created. The resulting permission is given by p & ^u, where p is the permission and u is the umask. For example, if p is 0777 and u is 0057, then the resulting permission is 0720. The default permission is 0777 for a directory and 0666 for a file. The default umask is 0027. The umask must be specified in 4-digit octal notation (e.g. 0766).
owner (str) – The owner of the file or directory.
group (str) – The owning group of the file or directory.
acl (str) – Sets POSIX access control rights on files and directories. The value is a comma-separated list of access control entries. Each access control entry (ACE) consists of a scope, a type, a user or group identifier, and permissions in the format “[scope:][type]:[id]:[permissions]”.
lease_id (str) – Proposed lease ID, in a GUID string format. The DataLake service returns 400 (Invalid request) if the proposed lease ID is not in the correct format.
lease_duration (int) – Specifies the duration of the lease, in seconds, or negative one (-1) for a lease that never expires. A non-infinite lease can be between 15 and 60 seconds. A lease duration cannot be changed using renew or change.
expires_on (datetime or int) – The time to set the file to expiry. If the type of expires_on is an int, expiration time will be set as the number of milliseconds elapsed from creation time. If the type of expires_on is datetime, expiration time will be set absolute to the time provided. If no time zone info is provided, this will be interpreted as UTC.
permissions (str) – Optional and only valid if Hierarchical Namespace is enabled for the account. Sets POSIX access permissions for the file owner, the file owning group, and others. Each class may be granted read, write, or execute permission. The sticky bit is also supported. Both symbolic (rwxrw-rw-) and 4-digit octal notation (e.g. 0766) are supported.
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
cpk (CustomerProvidedEncryptionKey) – Encrypts the data on the service-side with the given key. Use of customer-provided keys must be done over HTTPS.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
response dict (Etag and last modified).
Example:
file_client = filesystem_client.get_file_client(file_name) file_client.create_file()
-
delete_file
(**kwargs) → None[source]¶ Marks the specified file for deletion.
- Keyword Arguments
lease (DataLakeLeaseClient or str) – Required if the file has an active lease. Value can be a LeaseClient object or the lease ID as a string.
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
None
Example:
new_client.delete_file()
-
download_file
(offset: Optional[int] = None, length: Optional[int] = None, **kwargs: Any) → azure.storage.filedatalake._download.StorageStreamDownloader[source]¶ Downloads a file to the StorageStreamDownloader. The readall() method must be used to read all the content, or readinto() must be used to download the file into a stream. Using chunks() returns an iterator which allows the user to iterate over the content in chunks.
- Parameters
- Keyword Arguments
lease (DataLakeLeaseClient or str) – If specified, download only succeeds if the file’s lease is active and matches this ID. Required if the file has an active lease.
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
cpk (CustomerProvidedEncryptionKey) – Decrypts the data on the service-side with the given key. Use of customer-provided keys must be done over HTTPS. Required if the file was created with a Customer-Provided Key.
max_concurrency (int) – The number of parallel connections with which to download.
timeout (int) – The timeout parameter is expressed in seconds. This method may make multiple calls to the Azure service and the timeout will apply to each call individually.
- Returns
A streaming object (StorageStreamDownloader)
- Return type
Example:
download = file_client.download_file() downloaded_bytes = download.readall()
-
exists
(**kwargs: Any) → bool[source]¶ Returns True if a file exists and returns False otherwise.
- Keyword Arguments
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
boolean
-
flush_data
(offset: int, retain_uncommitted_data: Optional[bool] = False, **kwargs) → Dict[str, Union[str, datetime]][source]¶ Commit the previous appended data.
- Parameters
offset – offset is equal to the length of the file after commit the previous appended data.
retain_uncommitted_data (bool) – Valid only for flush operations. If “true”, uncommitted data is retained after the flush operation completes; otherwise, the uncommitted data is deleted after the flush operation. The default is false. Data at offsets less than the specified position are written to the file when flush succeeds, but this optional parameter allows data after the flush position to be retained for a future flush operation.
- Keyword Arguments
content_settings (ContentSettings) – ContentSettings object used to set path properties.
close (bool) – Azure Storage Events allow applications to receive notifications when files change. When Azure Storage Events are enabled, a file changed event is raised. This event has a property indicating whether this is the final change to distinguish the difference between an intermediate flush to a file stream and the final close of a file stream. The close query parameter is valid only when the action is “flush” and change notifications are enabled. If the value of close is “true” and the flush operation completes successfully, the service raises a file change notification with a property indicating that this is the final update (the file stream has been closed). If “false” a change notification is raised indicating the file has changed. The default is false. This query parameter is set to true by the Hadoop ABFS driver to indicate that the file stream has been closed.”
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
cpk (CustomerProvidedEncryptionKey) – Encrypts the data on the service-side with the given key. Use of customer-provided keys must be done over HTTPS.
- Returns
response header in dict
Example:
with open(SOURCE_FILE, "rb") as data: file_client = file_system_client.get_file_client("myfile") file_client.create_file() file_client.append_data(data, 0) file_client.flush_data(data.tell())
-
classmethod
from_connection_string
(conn_str: str, file_system_name: str, file_path: str, credential: Optional[Union[str, Dict[str, str], AzureNamedKeyCredential, AzureSasCredential, "TokenCredential"]] # pylint: disable=line-too-long = None, **kwargs: Any) → ClassType[source]¶ Create DataLakeFileClient from a Connection String.
- Parameters
conn_str (str) – A connection string to an Azure Storage account.
file_system_name (str) – The name of file system to interact with.
directory_name (str) – The name of directory to interact with. The directory is under file system.
file_name (str) – The name of file to interact with. The file is under directory.
credential – The credentials with which to authenticate. This is optional if the account URL already has a SAS token, or the connection string already has shared access key values. The value can be a SAS token string, an instance of a AzureSasCredential or AzureNamedKeyCredential from azure.core.credentials, an account shared access key, or an instance of a TokenCredentials class from azure.identity. Credentials provided here will take precedence over those in the connection string. If using an instance of AzureNamedKeyCredential, “name” should be the storage account name, and “key” should be the storage account key.
:return a DataLakeFileClient :rtype ~azure.storage.filedatalake.DataLakeFileClient
-
get_access_control
(upn: Optional[bool] = None, **kwargs) → Dict[str, Any]¶ - Parameters
upn (bool) – Optional. Valid only when Hierarchical Namespace is enabled for the account. If “true”, the user identity values returned in the x-ms-owner, x-ms-group, and x-ms-acl response headers will be transformed from Azure Active Directory Object IDs to User Principal Names. If “false”, the values will be returned as Azure Active Directory Object IDs. The default value is false. Note that group and application Object IDs are not translated because they do not have unique friendly names.
- Keyword Arguments
lease (DataLakeLeaseClient or str) – Required if the file/directory has an active lease. Value can be a LeaseClient object or the lease ID as a string.
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
timeout (int) – The timeout parameter is expressed in seconds.
- Keyword
response dict.
-
get_file_properties
(**kwargs: Any) → azure.storage.filedatalake._models.FileProperties[source]¶ Returns all user-defined metadata, standard HTTP properties, and system properties for the file. It does not return the content of the file.
- Keyword Arguments
lease – Required if the directory or file has an active lease. Value can be a DataLakeLeaseClient object or the lease ID as a string.
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
cpk (CustomerProvidedEncryptionKey) – Decrypts the data on the service-side with the given key. Use of customer-provided keys must be done over HTTPS. Required if the file was created with a customer-provided key.
timeout (int) – The timeout parameter is expressed in seconds.
- Return type
Example:
properties = file_client.get_file_properties()
-
query_file
(query_expression: str, **kwargs: Any) → azure.storage.filedatalake._quick_query_helper.DataLakeFileQueryReader[source]¶ Enables users to select/project on datalake file data by providing simple query expressions. This operations returns a DataLakeFileQueryReader, users need to use readall() or readinto() to get query data.
- Parameters
query_expression (str) – Required. a query statement. eg. Select * from DataLakeStorage
- Keyword Arguments
on_error (Callable[DataLakeFileQueryError]) – A function to be called on any processing errors returned by the service.
file_format (DelimitedTextDialect or DelimitedJsonDialect or QuickQueryDialect or str) – Optional. Defines the serialization of the data currently stored in the file. The default is to treat the file data as CSV data formatted in the default dialect. This can be overridden with a custom DelimitedTextDialect, or DelimitedJsonDialect or “ParquetDialect” (passed as a string or enum). These dialects can be passed through their respective classes, the QuickQueryDialect enum or as a string.
output_format (DelimitedTextDialect or DelimitedJsonDialect or list[ArrowDialect] or QuickQueryDialect or str) – Optional. Defines the output serialization for the data stream. By default the data will be returned as it is represented in the file. By providing an output format, the file data will be reformatted according to that profile. This value can be a DelimitedTextDialect or a DelimitedJsonDialect or ArrowDialect. These dialects can be passed through their respective classes, the QuickQueryDialect enum or as a string.
lease (DataLakeLeaseClient or str) – Required if the file has an active lease. Value can be a DataLakeLeaseClient object or the lease ID as a string.
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
cpk (CustomerProvidedEncryptionKey) – Decrypts the data on the service-side with the given key. Use of customer-provided keys must be done over HTTPS. Required if the file was created with a Customer-Provided Key.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
A streaming object (DataLakeFileQueryReader)
- Return type
DataLakeFileQueryReader
Example:
errors = [] def on_error(error): errors.append(error) # upload the csv file file_client = datalake_service_client.get_file_client(filesystem_name, "csvfile") file_client.upload_data(CSV_DATA, overwrite=True) # select the second column of the csv file query_expression = "SELECT _2 from DataLakeStorage" input_format = DelimitedTextDialect(delimiter=',', quotechar='"', lineterminator='\n', escapechar="", has_header=False) output_format = DelimitedJsonDialect(delimiter='\n') reader = file_client.query_file(query_expression, on_error=on_error, file_format=input_format, output_format=output_format) content = reader.readall()
-
remove_access_control_recursive
(acl: str, **kwargs: Any) → azure.storage.filedatalake._models.AccessControlChangeResult¶ Removes the Access Control on a path and sub-paths.
- Parameters
acl (str) – Removes POSIX access control rights on files and directories. The value is a comma-separated list of access control entries. Each access control entry (ACE) consists of a scope, a type, and a user or group identifier in the format “[scope:][type]:[id]”.
- Keyword Arguments
progress_hook (func(AccessControlChanges)) – Callback where the caller can track progress of the operation as well as collect paths that failed to change Access Control.
continuation_token (str) – Optional continuation token that can be used to resume previously stopped operation.
batch_size (int) – Optional. If data set size exceeds batch size then operation will be split into multiple requests so that progress can be tracked. Batch size should be between 1 and 2000. The default when unspecified is 2000.
max_batches (int) – Optional. Defines maximum number of batches that single change Access Control operation can execute. If maximum is reached before all sub-paths are processed then, continuation token can be used to resume operation. Empty value indicates that maximum number of batches in unbound and operation continues till end.
continue_on_failure (bool) – If set to False, the operation will terminate quickly on encountering user errors (4XX). If True, the operation will ignore user errors and proceed with the operation on other sub-entities of the directory. Continuation token will only be returned when continue_on_failure is True in case of user errors. If not set the default value is False for this.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
A summary of the recursive operations, including the count of successes and failures, as well as a continuation token in case the operation was terminated prematurely.
- Return type
- Raises
AzureError – User can restart the operation using continuation_token field of AzureError if the token is available.
-
rename_file
(new_name: str, **kwargs: Any) → azure.storage.filedatalake._data_lake_file_client.DataLakeFileClient[source]¶ Rename the source file.
- Parameters
new_name (str) – the new file name the user want to rename to. The value must have the following format: “{filesystem}/{directory}/{subdirectory}/{file}”.
- Keyword Arguments
content_settings (ContentSettings) – ContentSettings object used to set path properties.
source_lease (DataLakeLeaseClient or str) – A lease ID for the source path. If specified, the source path must have an active lease and the leaase ID must match.
lease – Required if the file/directory has an active lease. Value can be a LeaseClient object or the lease ID as a string.
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
source_if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
source_if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
source_etag (str) – The source ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
source_match_condition (MatchConditions) – The source match condition to use upon the etag.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
the renamed file client
- Return type
Example:
new_client = file_client.rename_file(file_client.file_system_name + '/' + 'newname')
-
set_access_control
(owner: Optional[str] = None, group: Optional[str] = None, permissions: Optional[str] = None, acl: Optional[str] = None, **kwargs) → Dict[str, Union[str, datetime.datetime]]¶ Set the owner, group, permissions, or access control list for a path.
- Parameters
owner (str) – Optional. The owner of the file or directory.
group (str) – Optional. The owning group of the file or directory.
permissions (str) – Optional and only valid if Hierarchical Namespace is enabled for the account. Sets POSIX access permissions for the file owner, the file owning group, and others. Each class may be granted read, write, or execute permission. The sticky bit is also supported. Both symbolic (rwxrw-rw-) and 4-digit octal notation (e.g. 0766) are supported. permissions and acl are mutually exclusive.
acl (str) – Sets POSIX access control rights on files and directories. The value is a comma-separated list of access control entries. Each access control entry (ACE) consists of a scope, a type, a user or group identifier, and permissions in the format “[scope:][type]:[id]:[permissions]”. permissions and acl are mutually exclusive.
- Keyword Arguments
lease (DataLakeLeaseClient or str) – Required if the file/directory has an active lease. Value can be a LeaseClient object or the lease ID as a string.
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
timeout (int) – The timeout parameter is expressed in seconds.
- Keyword
response dict (Etag and last modified).
-
set_access_control_recursive
(acl: str, **kwargs: Any) → azure.storage.filedatalake._models.AccessControlChangeResult¶ Sets the Access Control on a path and sub-paths.
- Parameters
acl (str) – Sets POSIX access control rights on files and directories. The value is a comma-separated list of access control entries. Each access control entry (ACE) consists of a scope, a type, a user or group identifier, and permissions in the format “[scope:][type]:[id]:[permissions]”.
- Keyword Arguments
progress_hook (func(AccessControlChanges)) – Callback where the caller can track progress of the operation as well as collect paths that failed to change Access Control.
continuation_token (str) – Optional continuation token that can be used to resume previously stopped operation.
batch_size (int) – Optional. If data set size exceeds batch size then operation will be split into multiple requests so that progress can be tracked. Batch size should be between 1 and 2000. The default when unspecified is 2000.
max_batches (int) – Optional. Defines maximum number of batches that single change Access Control operation can execute. If maximum is reached before all sub-paths are processed, then continuation token can be used to resume operation. Empty value indicates that maximum number of batches in unbound and operation continues till end.
continue_on_failure (bool) – If set to False, the operation will terminate quickly on encountering user errors (4XX). If True, the operation will ignore user errors and proceed with the operation on other sub-entities of the directory. Continuation token will only be returned when continue_on_failure is True in case of user errors. If not set the default value is False for this.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
A summary of the recursive operations, including the count of successes and failures, as well as a continuation token in case the operation was terminated prematurely.
- Return type
- Raises
AzureError – User can restart the operation using continuation_token field of AzureError if the token is available.
-
set_file_expiry
(expiry_options: str, expires_on: Optional[Union[datetime, int]] = None, **kwargs) → None[source]¶ Sets the time a file will expire and be deleted.
- Parameters
expiry_options (str) – Required. Indicates mode of the expiry time. Possible values include: ‘NeverExpire’, ‘RelativeToCreation’, ‘RelativeToNow’, ‘Absolute’
or int expires_on (datetime) – The time to set the file to expiry. When expiry_options is RelativeTo*, expires_on should be an int in milliseconds. If the type of expires_on is datetime, it should be in UTC time.
- Keyword Arguments
timeout (int) – The timeout parameter is expressed in seconds.
- Return type
-
set_http_headers
(content_settings: Optional[ContentSettings] = None, **kwargs) → Dict[str, Any]¶ Sets system properties on the file or directory.
If one property is set for the content_settings, all properties will be overriden.
- Parameters
content_settings (ContentSettings) – ContentSettings object used to set file/directory properties.
- Keyword Arguments
lease (DataLakeLeaseClient or str) – If specified, set_file_system_metadata only succeeds if the file system’s lease is active and matches this ID.
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
file/directory-updated property dict (Etag and last modified)
- Return type
Dict[str, Any]
-
set_metadata
(metadata: Dict[str, str], **kwargs) → Dict[str, Union[str, datetime.datetime]]¶ Sets one or more user-defined name-value pairs for the specified file system. Each call to this operation replaces all existing metadata attached to the file system. To remove all metadata from the file system, call this operation with no metadata dict.
- Parameters
metadata (Dict[str, str]) – A dict containing name-value pairs to associate with the file system as metadata. Example: {‘category’:’test’}
- Keyword Arguments
lease (DataLakeLeaseClient or str) – If specified, set_file_system_metadata only succeeds if the file system’s lease is active and matches this ID.
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
cpk (CustomerProvidedEncryptionKey) – Encrypts the data on the service-side with the given key. Use of customer-provided keys must be done over HTTPS.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
file system-updated property dict (Etag and last modified).
-
update_access_control_recursive
(acl: str, **kwargs: Any) → azure.storage.filedatalake._models.AccessControlChangeResult¶ Modifies the Access Control on a path and sub-paths.
- Parameters
acl (str) – Modifies POSIX access control rights on files and directories. The value is a comma-separated list of access control entries. Each access control entry (ACE) consists of a scope, a type, a user or group identifier, and permissions in the format “[scope:][type]:[id]:[permissions]”.
- Keyword Arguments
progress_hook (func(AccessControlChanges)) – Callback where the caller can track progress of the operation as well as collect paths that failed to change Access Control.
continuation_token (str) – Optional continuation token that can be used to resume previously stopped operation.
batch_size (int) – Optional. If data set size exceeds batch size then operation will be split into multiple requests so that progress can be tracked. Batch size should be between 1 and 2000. The default when unspecified is 2000.
max_batches (int) – Optional. Defines maximum number of batches that single change Access Control operation can execute. If maximum is reached before all sub-paths are processed, then continuation token can be used to resume operation. Empty value indicates that maximum number of batches in unbound and operation continues till end.
continue_on_failure (bool) – If set to False, the operation will terminate quickly on encountering user errors (4XX). If True, the operation will ignore user errors and proceed with the operation on other sub-entities of the directory. Continuation token will only be returned when continue_on_failure is True in case of user errors. If not set the default value is False for this.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
A summary of the recursive operations, including the count of successes and failures, as well as a continuation token in case the operation was terminated prematurely.
- Return type
- Raises
AzureError – User can restart the operation using continuation_token field of AzureError if the token is available.
-
upload_data
(data: Union[bytes, str, Iterable, IO], length: Optional[int] = None, overwrite: Optional[bool] = False, **kwargs) → Dict[str, Any][source]¶ Upload data to a file.
- Parameters
- Keyword Arguments
content_settings (ContentSettings) – ContentSettings object used to set path properties.
metadata (dict(str, str)) – Name-value pairs associated with the blob as metadata.
or str lease (DataLakeLeaseClient) – Required if the blob has an active lease. Value can be a DataLakeLeaseClient object or the lease ID as a string.
umask (str) – Optional and only valid if Hierarchical Namespace is enabled for the account. When creating a file or directory and the parent folder does not have a default ACL, the umask restricts the permissions of the file or directory to be created. The resulting permission is given by p & ^u, where p is the permission and u is the umask. For example, if p is 0777 and u is 0057, then the resulting permission is 0720. The default permission is 0777 for a directory and 0666 for a file. The default umask is 0027. The umask must be specified in 4-digit octal notation (e.g. 0766).
permissions (str) – Optional and only valid if Hierarchical Namespace is enabled for the account. Sets POSIX access permissions for the file owner, the file owning group, and others. Each class may be granted read, write, or execute permission. The sticky bit is also supported. Both symbolic (rwxrw-rw-) and 4-digit octal notation (e.g. 0766) are supported.
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
validate_content (bool) – If true, calculates an MD5 hash for each chunk of the file. The storage service checks the hash of the content that has arrived with the hash that was sent. This is primarily valuable for detecting bitflips on the wire if using http instead of https, as https (the default), will already validate. Note that this MD5 hash is not stored with the blob. Also note that if enabled, the memory-efficient upload algorithm will not be used because computing the MD5 hash requires buffering entire blocks, and doing so defeats the purpose of the memory-efficient algorithm.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
cpk (CustomerProvidedEncryptionKey) – Encrypts the data on the service-side with the given key. Use of customer-provided keys must be done over HTTPS.
timeout (int) – The timeout parameter is expressed in seconds.
chunk_size (int) – The maximum chunk size for uploading a file in chunks. Defaults to 100*1024*1024, or 100MB.
- Returns
response dict (Etag and last modified).
-
property
location_mode
¶ The location mode that the client is currently using.
By default this will be “primary”. Options include “primary” and “secondary”.
- Type
-
property
secondary_endpoint
¶ The full secondary endpoint URL if configured.
If not available a ValueError will be raised. To explicitly specify a secondary hostname, use the optional secondary_hostname keyword argument on instantiation.
- Type
- Raises
-
property
secondary_hostname
¶ The hostname of the secondary endpoint.
If not available this will be None. To explicitly specify a secondary hostname, use the optional secondary_hostname keyword argument on instantiation.
-
property
url
¶ The full endpoint URL to this entity, including SAS token if used.
This could be either the primary endpoint, or the secondary endpoint depending on the current
location_mode()
.
-
class
azure.storage.filedatalake.
DataLakeFileQueryError
(error=None, is_fatal=False, description=None, position=None)[source]¶ The error happened during quick query operation.
- Variables
error (str) – The name of the error.
is_fatal (bool) – If true, this error prevents further query processing. More result data may be returned, but there is no guarantee that all of the original data will be processed. If false, this error does not prevent further query processing.
description (str) – A description of the error.
position (int) – The blob offset at which the error occurred.
-
class
azure.storage.filedatalake.
DataLakeLeaseClient
(client: Union[FileSystemClient, DataLakeDirectoryClient, DataLakeFileClient], lease_id: Optional[str] = None)[source]¶ Creates a new DataLakeLeaseClient.
This client provides lease operations on a FileSystemClient, DataLakeDirectoryClient or DataLakeFileClient.
- Variables
id (str) – The ID of the lease currently being maintained. This will be None if no lease has yet been acquired.
etag (str) – The ETag of the lease currently being maintained. This will be None if no lease has yet been acquired or modified.
last_modified (datetime) – The last modified timestamp of the lease currently being maintained. This will be None if no lease has yet been acquired or modified.
- Parameters
client (FileSystemClient or DataLakeDirectoryClient or DataLakeFileClient) – The client of the file system, directory, or file to lease.
lease_id (str) – A string representing the lease ID of an existing lease. This value does not need to be specified in order to acquire a new lease, or break one.
-
acquire
(lease_duration: int = - 1, **kwargs: Optional[int]) → None[source]¶ Requests a new lease.
If the file/file system does not have an active lease, the DataLake service creates a lease on the file/file system and returns a new lease ID.
- Parameters
lease_duration (int) – Specifies the duration of the lease, in seconds, or negative one (-1) for a lease that never expires. A non-infinite lease can be between 15 and 60 seconds. A lease duration cannot be changed using renew or change. Default is -1 (infinite lease).
- Keyword Arguments
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
timeout (int) – The timeout parameter is expressed in seconds.
- Return type
-
break_lease
(lease_break_period: Optional[int] = None, **kwargs: Any) → int[source]¶ Break the lease, if the file system or file has an active lease.
Once a lease is broken, it cannot be renewed. Any authorized request can break the lease; the request is not required to specify a matching lease ID. When a lease is broken, the lease break period is allowed to elapse, during which time no lease operation except break and release can be performed on the file system or file. When a lease is successfully broken, the response indicates the interval in seconds until a new lease can be acquired.
- Parameters
lease_break_period (int) – This is the proposed duration of seconds that the lease should continue before it is broken, between 0 and 60 seconds. This break period is only used if it is shorter than the time remaining on the lease. If longer, the time remaining on the lease is used. A new lease will not be available before the break period has expired, but the lease may be held for longer than the break period. If this header does not appear with a break operation, a fixed-duration lease breaks after the remaining lease period elapses, and an infinite lease breaks immediately.
- Keyword Arguments
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
Approximate time remaining in the lease period, in seconds.
- Return type
-
change
(proposed_lease_id: str, **kwargs: Any) → None[source]¶ Change the lease ID of an active lease.
- Parameters
proposed_lease_id (str) – Proposed lease ID, in a GUID string format. The DataLake service returns 400 (Invalid request) if the proposed lease ID is not in the correct format.
- Keyword Arguments
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
None
-
release
(**kwargs: Any) → None[source]¶ Release the lease.
The lease may be released if the client lease id specified matches that associated with the file system or file. Releasing the lease allows another client to immediately acquire the lease for the file system or file as soon as the release is complete.
- Keyword Arguments
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
None
-
renew
(**kwargs: Any) → None[source]¶ Renews the lease.
The lease can be renewed if the lease ID specified in the lease client matches that associated with the file system or file. Note that the lease may be renewed even if it has expired as long as the file system or file has not been leased again since the expiration of that lease. When you renew a lease, the lease duration clock resets.
- Keyword Arguments
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
None
-
class
azure.storage.filedatalake.
DataLakeServiceClient
(account_url: str, credential: Optional[Any] = None, **kwargs: Any)[source]¶ A client to interact with the DataLake Service at the account level.
This client provides operations to retrieve and configure the account properties as well as list, create and delete file systems within the account. For operations relating to a specific file system, directory or file, clients for those entities can also be retrieved using the get_client functions.
- Variables
url (str) – The full endpoint URL to the datalake service endpoint.
primary_endpoint (str) – The full primary endpoint URL.
primary_hostname (str) – The hostname of the primary endpoint.
- Parameters
account_url (str) – The URL to the DataLake storage account. Any other entities included in the URL path (e.g. file system or file) will be discarded. This URL can be optionally authenticated with a SAS token.
credential – The credentials with which to authenticate. This is optional if the account URL already has a SAS token. The value can be a SAS token string, an instance of a AzureSasCredential or AzureNamedKeyCredential from azure.core.credentials, an account shared access key, or an instance of a TokenCredentials class from azure.identity. If the resource URI already contains a SAS token, this will be ignored in favor of an explicit credential - except in the case of AzureSasCredential, where the conflicting SAS tokens will raise a ValueError. If using an instance of AzureNamedKeyCredential, “name” should be the storage account name, and “key” should be the storage account key.
- Keyword Arguments
api_version (str) – The Storage API version to use for requests. Default value is the most recent service version that is compatible with the current SDK. Setting to an older version may result in reduced feature compatibility.
Example:
from azure.storage.filedatalake import DataLakeServiceClient datalake_service_client = DataLakeServiceClient.from_connection_string(self.connection_string)
from azure.identity import ClientSecretCredential token_credential = ClientSecretCredential( self.active_directory_tenant_id, self.active_directory_application_id, self.active_directory_application_secret, ) datalake_service_client = DataLakeServiceClient("https://{}.dfs.core.windows.net".format(self.account_name), credential=token_credential)
-
close
() → None[source]¶ This method is to close the sockets opened by the client. It need not be used when using with a context manager.
-
create_file_system
(file_system: Union[FileSystemProperties, str], metadata: Optional[Dict[str, str]] = None, public_access: Optional[PublicAccess] = None, **kwargs) → FileSystemClient[source]¶ Creates a new file system under the specified account.
If the file system with the same name already exists, a ResourceExistsError will be raised. This method returns a client with which to interact with the newly created file system.
- Parameters
file_system (str) – The name of the file system to create.
metadata (dict(str, str)) – A dict with name-value pairs to associate with the file system as metadata. Example: {‘Category’:’test’}
public_access (PublicAccess) – Possible values include: file system, file.
- Keyword Arguments
encryption_scope_options (dict or EncryptionScopeOptions) –
Specifies the default encryption scope to set on the file system and use for all future writes.
New in version 12.9.0.
timeout (int) – The timeout parameter is expressed in seconds.
- Return type
Example:
datalake_service_client.create_file_system("filesystem")
-
delete_file_system
(file_system: Union[FileSystemProperties, str], **kwargs) → FileSystemClient[source]¶ Marks the specified file system for deletion.
The file system and any files contained within it are later deleted during garbage collection. If the file system is not found, a ResourceNotFoundError will be raised.
- Parameters
file_system (str or FileSystemProperties) – The file system to delete. This can either be the name of the file system, or an instance of FileSystemProperties.
- Keyword Arguments
lease (DataLakeLeaseClient or str) – If specified, delete_file_system only succeeds if the file system’s lease is active and matches this ID. Required if the file system has an active lease.
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
timeout (int) – The timeout parameter is expressed in seconds.
- Return type
Example:
datalake_service_client.delete_file_system("filesystem")
-
classmethod
from_connection_string
(conn_str: str, credential: Optional[Any] = None, **kwargs: Any) → ClassType[source]¶ Create DataLakeServiceClient from a Connection String.
- Parameters
conn_str (str) – A connection string to an Azure Storage account.
credential – The credentials with which to authenticate. This is optional if the account URL already has a SAS token, or the connection string already has shared access key values. The value can be a SAS token string, an instance of a AzureSasCredential from azure.core.credentials, an account shared access key, or an instance of a TokenCredentials class from azure.identity. Credentials provided here will take precedence over those in the connection string.
:return a DataLakeServiceClient :rtype ~azure.storage.filedatalake.DataLakeServiceClient
Example:
from azure.storage.filedatalake import DataLakeServiceClient datalake_service_client = DataLakeServiceClient.from_connection_string(self.connection_string)
-
get_directory_client
(file_system: Union[FileSystemProperties, str], directory: Union[DirectoryProperties, str]) → DataLakeDirectoryClient[source]¶ Get a client to interact with the specified directory.
The directory need not already exist.
- Parameters
file_system (str or FileSystemProperties) – The file system that the directory is in. This can either be the name of the file system, or an instance of FileSystemProperties.
directory (str or DirectoryProperties) – The directory with which to interact. This can either be the name of the directory, or an instance of DirectoryProperties.
- Returns
A DataLakeDirectoryClient.
- Return type
Example:
directory_client = datalake_service_client.get_directory_client(file_system_client.file_system_name, "mydirectory")
-
get_file_client
(file_system: Union[FileSystemProperties, str], file_path: Union[FileProperties, str]) → DataLakeFileClient[source]¶ Get a client to interact with the specified file.
The file need not already exist.
- Parameters
file_system (str or FileSystemProperties) – The file system that the file is in. This can either be the name of the file system, or an instance of FileSystemProperties.
file_path (str or FileProperties) – The file with which to interact. This can either be the full path of the file(from the root directory), or an instance of FileProperties. eg. directory/subdirectory/file
- Returns
A DataLakeFileClient.
- Return type
Example:
file_client = datalake_service_client.get_file_client(file_system_client.file_system_name, "myfile")
-
get_file_system_client
(file_system: Union[FileSystemProperties, str]) → FileSystemClient[source]¶ Get a client to interact with the specified file system.
The file system need not already exist.
- Parameters
file_system (str or FileSystemProperties) – The file system. This can either be the name of the file system, or an instance of FileSystemProperties.
- Returns
A FileSystemClient.
- Return type
Example:
# Instantiate a DataLakeServiceClient using a connection string from azure.storage.filedatalake import DataLakeServiceClient datalake_service_client = DataLakeServiceClient.from_connection_string(self.connection_string) # Instantiate a FileSystemClient file_system_client = datalake_service_client.get_file_system_client("mynewfilesystem")
-
get_service_properties
(**kwargs: Any) → Dict[str, Any][source]¶ Gets the properties of a storage account’s datalake service, including Azure Storage Analytics.
New in version 12.4.0: This operation was introduced in API version ‘2020-06-12’.
-
get_user_delegation_key
(key_start_time: datetime, key_expiry_time: datetime, **kwargs: Any) → UserDelegationKey[source]¶ Obtain a user delegation key for the purpose of signing SAS tokens. A token credential must be present on the service object for this request to succeed.
- Parameters
- Keyword Arguments
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
The user delegation key.
- Return type
Example:
from datetime import datetime, timedelta user_delegation_key = datalake_service_client.get_user_delegation_key(datetime.utcnow(), datetime.utcnow() + timedelta(hours=1))
-
list_file_systems
(name_starts_with: Optional[str] = None, include_metadata: Optional[bool] = None, **kwargs) → ItemPaged[FileSystemProperties][source]¶ Returns a generator to list the file systems under the specified account.
The generator will lazily follow the continuation tokens returned by the service and stop when all file systems have been returned.
- Parameters
- Keyword Arguments
results_per_page (int) – The maximum number of file system names to retrieve per API call. If the request does not specify the server will return up to 5,000 items per page.
timeout (int) – The timeout parameter is expressed in seconds.
include_deleted (bool) – Specifies that deleted file systems to be returned in the response. This is for file system restore enabled account. The default value is False. .. versionadded:: 12.3.0
include_system (bool) – Flag specifying that system filesystems should be included. .. versionadded:: 12.6.0
- Returns
An iterable (auto-paging) of FileSystemProperties.
- Return type
Example:
file_systems = datalake_service_client.list_file_systems() for file_system in file_systems: print(file_system.name)
-
set_service_properties
(**kwargs: Any) → None[source]¶ Sets the properties of a storage account’s Datalake service, including Azure Storage Analytics.
New in version 12.4.0: This operation was introduced in API version ‘2020-06-12’.
If an element (e.g. analytics_logging) is left as None, the existing settings on the service for that functionality are preserved.
- Keyword Arguments
analytics_logging – Groups the Azure Analytics Logging settings.
hour_metrics – The hour metrics settings provide a summary of request statistics grouped by API in hourly aggregates.
minute_metrics – The minute metrics settings provide request statistics for each minute.
cors – You can include up to five CorsRule elements in the list. If an empty list is specified, all CORS rules will be deleted, and CORS will be disabled for the service.
target_version (str) – Indicates the default version to use for requests if an incoming request’s version is not specified.
delete_retention_policy – The delete retention policy specifies whether to retain deleted files/directories. It also specifies the number of days and versions of file/directory to keep.
static_website – Specifies whether the static website feature is enabled, and if yes, indicates the index document and 404 error document to use.
timeout (int) – The timeout parameter is expressed in seconds.
- Return type
-
undelete_file_system
(name: str, deleted_version: str, **kwargs: Any) → azure.storage.filedatalake._file_system_client.FileSystemClient[source]¶ Restores soft-deleted filesystem.
Operation will only be successful if used within the specified number of days set in the delete retention policy.
New in version 12.3.0: This operation was introduced in API version ‘2019-12-12’.
- Parameters
- Keyword Arguments
timeout (int) – The timeout parameter is expressed in seconds.
- Return type
-
property
location_mode
¶ The location mode that the client is currently using.
By default this will be “primary”. Options include “primary” and “secondary”.
- Type
-
property
secondary_endpoint
¶ The full secondary endpoint URL if configured.
If not available a ValueError will be raised. To explicitly specify a secondary hostname, use the optional secondary_hostname keyword argument on instantiation.
- Type
- Raises
-
property
secondary_hostname
¶ The hostname of the secondary endpoint.
If not available this will be None. To explicitly specify a secondary hostname, use the optional secondary_hostname keyword argument on instantiation.
-
property
url
¶ The full endpoint URL to this entity, including SAS token if used.
This could be either the primary endpoint, or the secondary endpoint depending on the current
location_mode()
.
-
class
azure.storage.filedatalake.
DeletedPathProperties
(**kwargs)[source]¶ Properties populated for a deleted path.
- Variables
name (str) – The name of the file in the path.
deleted_time (datetime) – A datetime object representing the time at which the path was deleted.
remaining_retention_days (int) – The number of days that the path will be retained before being permanently deleted by the service.
deletion_id (str) – The id associated with the deleted path.
-
get
(key, default=None)¶
-
has_key
(k)¶
-
items
()¶
-
keys
()¶
-
update
(*args, **kwargs)¶
-
values
()¶
-
class
azure.storage.filedatalake.
DelimitedJsonDialect
(**kwargs)[source]¶ Defines the input or output JSON serialization for a datalake query.
- keyword str delimiter
The line separator character, default value is ‘
‘
-
get
(key, default=None)¶
-
has_key
(k)¶
-
items
()¶
-
keys
()¶
-
update
(*args, **kwargs)¶
-
values
()¶
-
class
azure.storage.filedatalake.
DelimitedTextDialect
(**kwargs)[source]¶ Defines the input or output delimited (CSV) serialization for a datalake query request.
- keyword str delimiter
Column separator, defaults to ‘,’.
- keyword str quotechar
Field quote, defaults to ‘”’.
- keyword str lineterminator
Record separator, defaults to ‘
- ‘.
- keyword str escapechar
Escape char, defaults to empty.
- keyword bool has_header
Whether the blob data includes headers in the first line. The default value is False, meaning that the data will be returned inclusive of the first line. If set to True, the data will be returned exclusive of the first line.
-
get
(key, default=None)¶
-
has_key
(k)¶
-
items
()¶
-
keys
()¶
-
update
(*args, **kwargs)¶
-
values
()¶
-
class
azure.storage.filedatalake.
DirectoryProperties
(**kwargs)[source]¶ - Variables
name (str) – name of the directory
etag (str) – The ETag contains a value that you can use to perform operations conditionally.
deleted (bool) – if the current directory marked as deleted
metadata (dict) – Name-value pairs associated with the directory as metadata.
encryption_scope (str) – A predefined encryption scope used to encrypt the data on the service. An encryption scope can be created using the Management API and referenced here by name. If a default encryption scope has been defined at the file system, this value will override it if the file system level scope is configured to allow overrides. Otherwise an error will be raised.
lease (LeaseProperties) – Stores all the lease information for the directory.
last_modified (datetime) – A datetime object representing the last time the directory was modified.
creation_time (datetime) – Indicates when the directory was created, in UTC.
remaining_retention_days (int) – The number of days that the directory will be retained before being permanently deleted by the service.
content_settings (ContentSettings) –
-
get
(key, default=None)¶
-
has_key
(k)¶
-
items
()¶
-
keys
()¶
-
update
(*args, **kwargs)¶
-
values
()¶
-
class
azure.storage.filedatalake.
DirectorySasPermissions
(read=False, create=False, write=False, delete=False, **kwargs)[source]¶ DirectorySasPermissions class to be used with the
generate_directory_sas()
function.- Parameters
- Keyword Arguments
add (bool) – Append data to a file in the directory.
list (bool) – List any files in the directory. Implies Execute.
move (bool) – Move any file in the directory to a new location. Note the move operation can optionally be restricted to the child file or directory owner or the parent directory owner if the saoid parameter is included in the token and the sticky bit is set on the parent directory.
execute (bool) – Get the status (system defined properties) and ACL of any file in the directory. If the caller is the owner, set access control on any file in the directory.
manage_ownership (bool) – Allows the user to set owner, owning group, or act as the owner when renaming or deleting a file or directory within a folder that has the sticky bit set.
manage_access_control (bool) – Allows the user to set permissions and POSIX ACLs on files and directories.
-
classmethod
from_string
(permission)[source]¶ Create a DirectorySasPermissions from a string.
To specify read, create, write, or delete permissions you need only to include the first letter of the word in the string. E.g. For read and write permissions, you would provide a string “rw”.
- Parameters
permission (str) – The string which dictates the read, add, create, write, or delete permissions.
- Returns
A DirectorySasPermissions object
- Return type
-
class
azure.storage.filedatalake.
EncryptionScopeOptions
(default_encryption_scope, **kwargs)[source]¶ The default encryption scope configuration for a file system.
This scope is used implicitly for all future writes within the file system, but can be overridden per blob operation.
New in version 12.9.0.
- Parameters
default_encryption_scope (str) – Specifies the default encryption scope to set on the file system and use for all future writes.
prevent_encryption_scope_override (bool) – If true, prevents any request from specifying a different encryption scope than the scope set on the file system. Default value is false.
-
class
azure.storage.filedatalake.
ExponentialRetry
(initial_backoff=15, increment_base=3, retry_total=3, retry_to_secondary=False, random_jitter_range=3, **kwargs)[source]¶ Exponential retry.
Constructs an Exponential retry object. The initial_backoff is used for the first retry. Subsequent retries are retried after initial_backoff + increment_power^retry_count seconds.
- Parameters
initial_backoff (int) – The initial backoff interval, in seconds, for the first retry.
increment_base (int) – The base, in seconds, to increment the initial_backoff by after the first retry.
max_attempts (int) – The maximum number of retry attempts.
retry_to_secondary (bool) – Whether the request should be retried to secondary, if able. This should only be enabled of RA-GRS accounts are used and potentially stale data can be handled.
random_jitter_range (int) – A number in seconds which indicates a range to jitter/randomize for the back-off interval. For example, a random_jitter_range of 3 results in the back-off interval x to vary between x+3 and x-3.
-
configure_retries
(request)¶
-
increment
(settings, request, response=None, error=None)¶ Increment the retry counters.
- Parameters
response – A pipeline response object.
error – An error encountered during the request, or None if the response was received successfully.
- Returns
Whether the retry attempts are exhausted.
-
send
(request)¶ Abstract send method for a synchronous pipeline. Mutates the request.
Context content is dependent on the HttpTransport.
- Parameters
request (PipelineRequest) – The pipeline request object
- Returns
The pipeline response object.
- Return type
-
sleep
(settings, transport)¶
-
class
azure.storage.filedatalake.
FileProperties
(**kwargs)[source]¶ - Variables
name (str) – name of the file
etag (str) – The ETag contains a value that you can use to perform operations conditionally.
deleted (bool) – if the current file marked as deleted
metadata (dict) – Name-value pairs associated with the file as metadata.
encryption_scope (str) – A predefined encryption scope used to encrypt the data on the service. An encryption scope can be created using the Management API and referenced here by name. If a default encryption scope has been defined at the file system, this value will override it if the file system level scope is configured to allow overrides. Otherwise an error will be raised.
lease (LeaseProperties) – Stores all the lease information for the file.
last_modified (datetime) – A datetime object representing the last time the file was modified.
creation_time (datetime) – Indicates when the file was created, in UTC.
size (int) – size of the file
remaining_retention_days (int) – The number of days that the file will be retained before being permanently deleted by the service.
content_settings (ContentSettings) –
-
get
(key, default=None)¶
-
has_key
(k)¶
-
items
()¶
-
keys
()¶
-
update
(*args, **kwargs)¶
-
values
()¶
-
class
azure.storage.filedatalake.
FileSasPermissions
(read=False, create=False, write=False, delete=False, **kwargs)[source]¶ FileSasPermissions class to be used with the
generate_file_sas()
function.- Parameters
- Keyword Arguments
add (bool) – Append data to the file.
move (bool) – Move any file in the directory to a new location. Note the move operation can optionally be restricted to the child file or directory owner or the parent directory owner if the saoid parameter is included in the token and the sticky bit is set on the parent directory.
execute (bool) – Get the status (system defined properties) and ACL of any file in the directory. If the caller is the owner, set access control on any file in the directory.
manage_ownership (bool) – Allows the user to set owner, owning group, or act as the owner when renaming or deleting a file or directory within a folder that has the sticky bit set.
manage_access_control (bool) – Allows the user to set permissions and POSIX ACLs on files and directories.
-
classmethod
from_string
(permission)[source]¶ Create a FileSasPermissions from a string.
To specify read, write, or delete permissions you need only to include the first letter of the word in the string. E.g. For read and write permissions, you would provide a string “rw”.
- Parameters
permission (str) – The string which dictates the read, add, create, write, or delete permissions.
- Returns
A FileSasPermissions object
- Return type
FileSasPermissions
-
class
azure.storage.filedatalake.
FileSystemClient
(account_url: str, file_system_name: str, credential: Optional[Union[str, Dict[str, str], AzureNamedKeyCredential, AzureSasCredential, "TokenCredential"]] # pylint: disable=line-too-long = None, **kwargs: Any)[source]¶ A client to interact with a specific file system, even if that file system may not yet exist.
For operations relating to a specific directory or file within this file system, a directory client or file client can be retrieved using the
get_directory_client()
orget_file_client()
functions.- Variables
url (str) – The full endpoint URL to the file system, including SAS token if used.
primary_endpoint (str) – The full primary endpoint URL.
primary_hostname (str) – The hostname of the primary endpoint.
- Parameters
account_url (str) – The URI to the storage account.
file_system_name (str) – The file system for the directory or files.
credential – The credentials with which to authenticate. This is optional if the account URL already has a SAS token. The value can be a SAS token string, an instance of a AzureSasCredential or AzureNamedKeyCredential from azure.core.credentials, an account shared access key, or an instance of a TokenCredentials class from azure.identity. If the resource URI already contains a SAS token, this will be ignored in favor of an explicit credential - except in the case of AzureSasCredential, where the conflicting SAS tokens will raise a ValueError. If using an instance of AzureNamedKeyCredential, “name” should be the storage account name, and “key” should be the storage account key.
- Keyword Arguments
api_version (str) – The Storage API version to use for requests. Default value is the most recent service version that is compatible with the current SDK. Setting to an older version may result in reduced feature compatibility.
Example:
# Instantiate a DataLakeServiceClient using a connection string from azure.storage.filedatalake import DataLakeServiceClient datalake_service_client = DataLakeServiceClient.from_connection_string(self.connection_string) # Instantiate a FileSystemClient file_system_client = datalake_service_client.get_file_system_client("mynewfilesystem")
-
acquire_lease
(lease_duration: int = - 1, lease_id: Optional[str] = None, **kwargs) → azure.storage.filedatalake._data_lake_lease.DataLakeLeaseClient[source]¶ Requests a new lease. If the file system does not have an active lease, the DataLake service creates a lease on the file system and returns a new lease ID.
- Parameters
lease_duration (int) – Specifies the duration of the lease, in seconds, or negative one (-1) for a lease that never expires. A non-infinite lease can be between 15 and 60 seconds. A lease duration cannot be changed using renew or change. Default is -1 (infinite lease).
lease_id (str) – Proposed lease ID, in a GUID string format. The DataLake service returns 400 (Invalid request) if the proposed lease ID is not in the correct format.
- Keyword Arguments
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
A DataLakeLeaseClient object, that can be run in a context manager.
- Return type
Example:
# Acquire a lease on the file system lease = file_system_client.acquire_lease() # Delete file system by passing in the lease file_system_client.delete_file_system(lease=lease)
-
close
() → None[source]¶ This method is to close the sockets opened by the client. It need not be used when using with a context manager.
-
create_directory
(directory: Union[azure.storage.filedatalake._models.DirectoryProperties, str], metadata: Optional[Dict[str, str]] = None, **kwargs) → azure.storage.filedatalake._data_lake_directory_client.DataLakeDirectoryClient[source]¶ Create directory
- Parameters
directory (str or DirectoryProperties) – The directory with which to interact. This can either be the name of the directory, or an instance of DirectoryProperties.
metadata (dict(str, str)) – Name-value pairs associated with the file as metadata.
- Keyword Arguments
content_settings (ContentSettings) – ContentSettings object used to set path properties.
lease (DataLakeLeaseClient or str) – Required if the file has an active lease. Value can be a DataLakeLeaseClient object or the lease ID as a string.
umask (str) – Optional and only valid if Hierarchical Namespace is enabled for the account. When creating a file or directory and the parent folder does not have a default ACL, the umask restricts the permissions of the file or directory to be created. The resulting permission is given by p & ^u, where p is the permission and u is the umask. For example, if p is 0777 and u is 0057, then the resulting permission is 0720. The default permission is 0777 for a directory and 0666 for a file. The default umask is 0027. The umask must be specified in 4-digit octal notation (e.g. 0766).
owner (str) – The owner of the file or directory.
group (str) – The owning group of the file or directory.
acl (str) – Sets POSIX access control rights on files and directories. The value is a comma-separated list of access control entries. Each access control entry (ACE) consists of a scope, a type, a user or group identifier, and permissions in the format “[scope:][type]:[id]:[permissions]”.
lease_id (str) – Proposed lease ID, in a GUID string format. The DataLake service returns 400 (Invalid request) if the proposed lease ID is not in the correct format.
lease_duration (int) – Specifies the duration of the lease, in seconds, or negative one (-1) for a lease that never expires. A non-infinite lease can be between 15 and 60 seconds. A lease duration cannot be changed using renew or change.
permissions (str) – Optional and only valid if Hierarchical Namespace is enabled for the account. Sets POSIX access permissions for the file owner, the file owning group, and others. Each class may be granted read, write, or execute permission. The sticky bit is also supported. Both symbolic (rwxrw-rw-) and 4-digit octal notation (e.g. 0766) are supported.
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
DataLakeDirectoryClient
Example:
directory_client = file_system_client.create_directory("mydirectory")
-
create_file
(file: Union[azure.storage.filedatalake._models.FileProperties, str], **kwargs) → azure.storage.filedatalake._data_lake_file_client.DataLakeFileClient[source]¶ Create file
- Parameters
file (str or FileProperties) – The file with which to interact. This can either be the name of the file, or an instance of FileProperties.
content_settings (ContentSettings) – ContentSettings object used to set path properties.
metadata (dict(str, str)) – Name-value pairs associated with the file as metadata.
- Keyword Arguments
lease (DataLakeLeaseClient or str) – Required if the file has an active lease. Value can be a DataLakeLeaseClient object or the lease ID as a string.
umask (str) – Optional and only valid if Hierarchical Namespace is enabled for the account. When creating a file or directory and the parent folder does not have a default ACL, the umask restricts the permissions of the file or directory to be created. The resulting permission is given by p & ^u, where p is the permission and u is the umask. For example, if p is 0777 and u is 0057, then the resulting permission is 0720. The default permission is 0777 for a directory and 0666 for a file. The default umask is 0027. The umask must be specified in 4-digit octal notation (e.g. 0766).
owner (str) – The owner of the file or directory.
group (str) – The owning group of the file or directory.
acl (str) – Sets POSIX access control rights on files and directories. The value is a comma-separated list of access control entries. Each access control entry (ACE) consists of a scope, a type, a user or group identifier, and permissions in the format “[scope:][type]:[id]:[permissions]”.
lease_id (str) – Proposed lease ID, in a GUID string format. The DataLake service returns 400 (Invalid request) if the proposed lease ID is not in the correct format.
lease_duration (int) – Specifies the duration of the lease, in seconds, or negative one (-1) for a lease that never expires. A non-infinite lease can be between 15 and 60 seconds. A lease duration cannot be changed using renew or change.
expires_on (datetime or int) – The time to set the file to expiry. If the type of expires_on is an int, expiration time will be set as the number of milliseconds elapsed from creation time. If the type of expires_on is datetime, expiration time will be set absolute to the time provided. If no time zone info is provided, this will be interpreted as UTC.
permissions (str) – Optional and only valid if Hierarchical Namespace is enabled for the account. Sets POSIX access permissions for the file owner, the file owning group, and others. Each class may be granted read, write, or execute permission. The sticky bit is also supported. Both symbolic (rwxrw-rw-) and 4-digit octal notation (e.g. 0766) are supported.
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
DataLakeFileClient
Example:
file_client = file_system_client.create_file("myfile")
-
create_file_system
(metadata: Optional[Dict[str, str]] = None, public_access: Optional[PublicAccess] = None, **kwargs) → Dict[str, Union[str, datetime]][source]¶ Creates a new file system under the specified account.
If the file system with the same name already exists, a ResourceExistsError will be raised. This method returns a client with which to interact with the newly created file system.
- Parameters
metadata (dict(str, str)) – A dict with name-value pairs to associate with the file system as metadata. Example: {‘Category’:’test’}
public_access (PublicAccess) – To specify whether data in the file system may be accessed publicly and the level of access.
- Keyword Arguments
encryption_scope_options (dict or EncryptionScopeOptions) –
Specifies the default encryption scope to set on the file system and use for all future writes.
New in version 12.9.0.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
A dictionary of response headers.
- Return type
Example:
file_system_client.create_file_system()
-
delete_directory
(directory: Union[azure.storage.filedatalake._models.DirectoryProperties, str], **kwargs) → azure.storage.filedatalake._data_lake_directory_client.DataLakeDirectoryClient[source]¶ Marks the specified path for deletion.
- Parameters
directory (str or DirectoryProperties) – The directory with which to interact. This can either be the name of the directory, or an instance of DirectoryProperties.
- Keyword Arguments
lease (DataLakeLeaseClient or str) – Required if the file has an active lease. Value can be a LeaseClient object or the lease ID as a string.
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
DataLakeDirectoryClient
Example:
file_system_client.delete_directory("mydirectory")
-
delete_file
(file: Union[azure.storage.filedatalake._models.FileProperties, str], **kwargs) → azure.storage.filedatalake._data_lake_file_client.DataLakeFileClient[source]¶ Marks the specified file for deletion.
- Parameters
file (str or FileProperties) – The file with which to interact. This can either be the name of the file, or an instance of FileProperties.
- Keyword Arguments
lease (DataLakeLeaseClient or str) – Required if the file has an active lease. Value can be a LeaseClient object or the lease ID as a string.
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
DataLakeFileClient
Example:
file_system_client.delete_file("myfile")
-
delete_file_system
(**kwargs: Any) → None[source]¶ Marks the specified file system for deletion.
The file system and any files contained within it are later deleted during garbage collection. If the file system is not found, a ResourceNotFoundError will be raised.
- Keyword Arguments
or ~azure.storage.filedatalake.DataLakeLeaseClient lease (str) – If specified, delete_file_system only succeeds if the file system’s lease is active and matches this ID. Required if the file system has an active lease.
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
timeout (int) – The timeout parameter is expressed in seconds.
- Return type
Example:
file_system_client.delete_file_system()
-
exists
(**kwargs: Any) → bool[source]¶ Returns True if a file system exists and returns False otherwise.
- Keyword Arguments
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
boolean
-
classmethod
from_connection_string
(conn_str: str, file_system_name: str, credential: Optional[Union[str, Dict[str, str], AzureNamedKeyCredential, AzureSasCredential, "TokenCredential"]] # pylint: disable=line-too-long = None, **kwargs: Any) → ClassType[source]¶ Create FileSystemClient from a Connection String.
- Parameters
conn_str (str) – A connection string to an Azure Storage account.
file_system_name (str) – The name of file system to interact with.
credential – The credentials with which to authenticate. This is optional if the account URL already has a SAS token, or the connection string already has shared access key values. The value can be a SAS token string, an instance of a AzureSasCredential or AzureNamedKeyCredential from azure.core.credentials, an account shared access key, or an instance of a TokenCredentials class from azure.identity. Credentials provided here will take precedence over those in the connection string. If using an instance of AzureNamedKeyCredential, “name” should be the storage account name, and “key” should be the storage account key.
:return a FileSystemClient :rtype ~azure.storage.filedatalake.FileSystemClient
Example:
from azure.storage.filedatalake import FileSystemClient file_system_client = FileSystemClient.from_connection_string(self.connection_string, "filesystem")
-
get_directory_client
(directory: Union[azure.storage.filedatalake._models.DirectoryProperties, str]) → azure.storage.filedatalake._data_lake_directory_client.DataLakeDirectoryClient[source]¶ Get a client to interact with the specified directory.
The directory need not already exist.
- Parameters
directory (str or DirectoryProperties) – The directory with which to interact. This can either be the name of the directory, or an instance of DirectoryProperties.
- Returns
A DataLakeDirectoryClient.
- Return type
Example:
# Get the DataLakeDirectoryClient from the FileSystemClient to interact with a specific file directory_client = file_system_client.get_directory_client("mynewdirectory")
-
get_file_client
(file_path: Union[azure.storage.filedatalake._models.FileProperties, str]) → azure.storage.filedatalake._data_lake_file_client.DataLakeFileClient[source]¶ Get a client to interact with the specified file.
The file need not already exist.
- Parameters
file_path (str or FileProperties) – The file with which to interact. This can either be the path of the file(from root directory), or an instance of FileProperties. eg. directory/subdirectory/file
- Returns
A DataLakeFileClient.
- Return type
Example:
# Get the FileClient from the FileSystemClient to interact with a specific file file_client = file_system_client.get_file_client("mynewfile")
-
get_file_system_access_policy
(**kwargs: Any) → Dict[str, Any][source]¶ Gets the permissions for the specified file system. The permissions indicate whether file system data may be accessed publicly.
- Keyword Arguments
lease (DataLakeLeaseClient or str) – If specified, the operation only succeeds if the file system’s lease is active and matches this ID.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
Access policy information in a dict.
- Return type
-
get_file_system_properties
(**kwargs: Any) → azure.storage.filedatalake._models.FileSystemProperties[source]¶ Returns all user-defined metadata and system properties for the specified file system. The data returned does not include the file system’s list of paths.
- Keyword Arguments
- Returns
Properties for the specified file system within a file system object.
- Return type
Example:
properties = file_system_client.get_file_system_properties()
-
get_paths
(path: Optional[str] = None, recursive: Optional[bool] = True, max_results: Optional[int] = None, **kwargs) → ItemPaged[PathProperties][source]¶ Returns a generator to list the paths(could be files or directories) under the specified file system. The generator will lazily follow the continuation tokens returned by the service.
- Parameters
- Keyword Arguments
upn – Optional. Valid only when Hierarchical Namespace is enabled for the account. If “true”, the user identity values returned in the x-ms-owner, x-ms-group, and x-ms-acl response headers will be transformed from Azure Active Directory Object IDs to User Principal Names. If “false”, the values will be returned as Azure Active Directory Object IDs. The default value is false. Note that group and application Object IDs are not translated because they do not have unique friendly names.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
An iterable (auto-paging) response of PathProperties.
- Return type
Example:
path_list = file_system_client.get_paths() for path in path_list: print(path.name + '\n')
-
list_deleted_paths
(**kwargs: Any) → azure.core.paging.ItemPaged[azure.storage.filedatalake._models.DeletedPathProperties][source]¶ Returns a generator to list the deleted (file or directory) paths under the specified file system. The generator will lazily follow the continuation tokens returned by the service.
New in version 12.4.0: This operation was introduced in API version ‘2020-06-12’.
- Keyword Arguments
path_prefix (str) – Filters the results to return only paths under the specified path.
results_per_page (int) – An optional value that specifies the maximum number of items to return per page. If omitted or greater than 5,000, the response will include up to 5,000 items per page.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
An iterable (auto-paging) response of DeletedPathProperties.
- Return type
-
set_file_system_access_policy
(signed_identifiers: Dict[str, AccessPolicy], public_access: Optional[Union[str, PublicAccess]] = None, **kwargs) → Dict[str, Union[str, datetime]][source]¶ Sets the permissions for the specified file system or stored access policies that may be used with Shared Access Signatures. The permissions indicate whether files in a file system may be accessed publicly.
- Parameters
signed_identifiers (dict[str, AccessPolicy]) – A dictionary of access policies to associate with the file system. The dictionary may contain up to 5 elements. An empty dictionary will clear the access policies set on the service.
public_access (PublicAccess) – To specify whether data in the file system may be accessed publicly and the level of access.
- Keyword Arguments
lease (DataLakeLeaseClient or str) – Required if the file system has an active lease. Value can be a DataLakeLeaseClient object or the lease ID as a string.
if_modified_since (datetime) – A datetime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified date/time.
if_unmodified_since (datetime) – A datetime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
File System-updated property dict (Etag and last modified).
- Return type
-
set_file_system_metadata
(metadata: Dict[str, str], **kwargs) → Dict[str, Union[str, datetime]][source]¶ Sets one or more user-defined name-value pairs for the specified file system. Each call to this operation replaces all existing metadata attached to the file system. To remove all metadata from the file system, call this operation with no metadata dict.
- Parameters
metadata (dict[str, str]) – A dict containing name-value pairs to associate with the file system as metadata. Example: {‘category’:’test’}
- Keyword Arguments
or ~azure.storage.filedatalake.DataLakeLeaseClient lease (str) – If specified, set_file_system_metadata only succeeds if the file system’s lease is active and matches this ID.
if_modified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has been modified since the specified time.
if_unmodified_since (datetime) – A DateTime value. Azure expects the date value passed in to be UTC. If timezone is included, any non-UTC datetimes will be converted to UTC. If a date is passed in without timezone info, it is assumed to be UTC. Specify this header to perform the operation only if the resource has not been modified since the specified date/time.
etag (str) – An ETag value, or the wildcard character (*). Used to check if the resource has changed, and act according to the condition specified by the match_condition parameter.
match_condition (MatchConditions) – The match condition to use upon the etag.
timeout (int) – The timeout parameter is expressed in seconds.
- Returns
filesystem-updated property dict (Etag and last modified).
Example:
# Create key, value pairs for metadata metadata = {'type': 'test'} # Set metadata on the file system file_system_client.set_file_system_metadata(metadata=metadata)
-
property
location_mode
¶ The location mode that the client is currently using.
By default this will be “primary”. Options include “primary” and “secondary”.
- Type
-
property
secondary_endpoint
¶ The full secondary endpoint URL if configured.
If not available a ValueError will be raised. To explicitly specify a secondary hostname, use the optional secondary_hostname keyword argument on instantiation.
- Type
- Raises
-
property
secondary_hostname
¶ The hostname of the secondary endpoint.
If not available this will be None. To explicitly specify a secondary hostname, use the optional secondary_hostname keyword argument on instantiation.
-
property
url
¶ The full endpoint URL to this entity, including SAS token if used.
This could be either the primary endpoint, or the secondary endpoint depending on the current
location_mode()
.
-
class
azure.storage.filedatalake.
FileSystemProperties
(**kwargs)[source]¶ File System properties class.
- Variables
name (str) – Name of the filesystem.
last_modified (datetime) – A datetime object representing the last time the file system was modified.
etag (str) – The ETag contains a value that you can use to perform operations conditionally.
lease (LeaseProperties) – Stores all the lease information for the file system.
public_access (str) – Specifies whether data in the file system may be accessed publicly and the level of access.
has_immutability_policy (bool) – Represents whether the file system has an immutability policy.
has_legal_hold (bool) – Represents whether the file system has a legal hold.
metadata (dict) – A dict with name-value pairs to associate with the file system as metadata.
encryption_scope (EncryptionScopeOptions) – The default encryption scope configuration for the file system.
deleted (bool) – Whether this file system was deleted.
deleted_version (str) – The version of a deleted file system.
Returned
FileSystemProperties
instances expose these values through a dictionary interface, for example:file_system_props["last_modified"]
. Additionally, the file system name is available asfile_system_props["name"]
.-
get
(key, default=None)¶
-
has_key
(k)¶
-
items
()¶
-
keys
()¶
-
update
(*args, **kwargs)¶
-
values
()¶
-
class
azure.storage.filedatalake.
FileSystemPropertiesPaged
(*args, **kwargs)[source]¶ An Iterable of File System properties.
- Variables
service_endpoint (str) – The service URL.
prefix (str) – A file system name prefix being used to filter the list.
marker (str) – The continuation token of the current page of results.
results_per_page (int) – The maximum number of results retrieved per API call.
continuation_token (str) – The continuation token to retrieve the next page of results.
location_mode (str) – The location mode being used to list results. The available options include “primary” and “secondary”.
current_page (list(FileSystemProperties)) – The current page of listed results.
- Parameters
command (callable) – Function to retrieve the next page of items.
prefix (str) – Filters the results to return only file systems whose names begin with the specified prefix.
results_per_page (int) – The maximum number of file system names to retrieve per call.
continuation_token (str) – An opaque continuation token.
Return an iterator of pages.
- Parameters
get_next – Callable that take the continuation token and return a HTTP response
extract_data – Callable that take an HTTP response and return a tuple continuation token, list of ReturnType
continuation_token (str) – The continuation token needed by get_next
-
next
() → Iterator[ReturnType]¶ Return the next item from the iterator. When exhausted, raise StopIteration
-
class
azure.storage.filedatalake.
FileSystemSasPermissions
(read=False, write=False, delete=False, list=False, **kwargs)[source]¶ FileSystemSasPermissions class to be used with the
generate_file_system_sas()
function.- Parameters
- Keyword Arguments
add (bool) – Append data to a file in the directory.
create (bool) – Write a new file, snapshot a file, or copy a file to a new file.
move (bool) – Move any file in the directory to a new location. Note the move operation can optionally be restricted to the child file or directory owner or the parent directory owner if the saoid parameter is included in the token and the sticky bit is set on the parent directory.
execute (bool) – Get the status (system defined properties) and ACL of any file in the directory. If the caller is the owner, set access control on any file in the directory.
manage_ownership (bool) – Allows the user to set owner, owning group, or act as the owner when renaming or deleting a file or directory within a folder that has the sticky bit set.
manage_access_control (bool) – Allows the user to set permissions and POSIX ACLs on files and directories.
-
classmethod
from_string
(permission)[source]¶ Create a FileSystemSasPermissions from a string.
To specify read, write, or delete permissions you need only to include the first letter of the word in the string. E.g. For read and write permissions, you would provide a string “rw”.
- Parameters
permission (str) – The string which dictates the read, add, create, write, or delete permissions.
- Returns
A FileSystemSasPermissions object
- Return type
FileSystemSasPermissions
-
class
azure.storage.filedatalake.
LeaseProperties
(**kwargs)[source]¶ DataLake Lease Properties.
- Variables
-
get
(key, default=None)¶
-
has_key
(k)¶
-
items
()¶
-
keys
()¶
-
update
(*args, **kwargs)¶
-
values
()¶
-
class
azure.storage.filedatalake.
LinearRetry
(backoff=15, retry_total=3, retry_to_secondary=False, random_jitter_range=3, **kwargs)[source]¶ Linear retry.
Constructs a Linear retry object.
- Parameters
backoff (int) – The backoff interval, in seconds, between retries.
max_attempts (int) – The maximum number of retry attempts.
retry_to_secondary (bool) – Whether the request should be retried to secondary, if able. This should only be enabled of RA-GRS accounts are used and potentially stale data can be handled.
random_jitter_range (int) – A number in seconds which indicates a range to jitter/randomize for the back-off interval. For example, a random_jitter_range of 3 results in the back-off interval x to vary between x+3 and x-3.
-
configure_retries
(request)¶
-
increment
(settings, request, response=None, error=None)¶ Increment the retry counters.
- Parameters
response – A pipeline response object.
error – An error encountered during the request, or None if the response was received successfully.
- Returns
Whether the retry attempts are exhausted.
-
send
(request)¶ Abstract send method for a synchronous pipeline. Mutates the request.
Context content is dependent on the HttpTransport.
- Parameters
request (PipelineRequest) – The pipeline request object
- Returns
The pipeline response object.
- Return type
-
sleep
(settings, transport)¶
-
class
azure.storage.filedatalake.
LocationMode
[source]¶ Specifies the location the request should be sent to. This mode only applies for RA-GRS accounts which allow secondary read access. All other account types must use PRIMARY.
-
PRIMARY
= 'primary'¶ Requests should be sent to the primary location.
-
SECONDARY
= 'secondary'¶ Requests should be sent to the secondary location, if possible.
-
-
class
azure.storage.filedatalake.
Metrics
(**kwargs)[source]¶ A summary of request statistics grouped by API in hour or minute aggregates.
- Keyword Arguments
version (str) – The version of Storage Analytics to configure. The default value is 1.0.
enabled (bool) – Indicates whether metrics are enabled for the Datalake service. The default value is False.
include_apis (bool) – Indicates whether metrics should generate summary statistics for called API operations.
retention_policy (RetentionPolicy) – Determines how long the associated data should persist. If not specified the retention policy will be disabled by default.
version – The version of Storage Analytics to configure.
enabled – Indicates whether metrics are enabled for the Blob service. Required.
include_apis – Indicates whether metrics should generate summary statistics for called API operations.
retention_policy – the retention policy which determines how long the associated data should persist.
-
as_dict
(keep_readonly=True, key_transformer=<function attribute_transformer>, **kwargs)¶ Return a dict that can be JSONify using json.dump.
Advanced usage might optionally use a callback as parameter:
Key is the attribute name used in Python. Attr_desc is a dict of metadata. Currently contains ‘type’ with the msrest type and ‘key’ with the RestAPI encoded key. Value is the current value in this object.
The string returned will be used to serialize the key. If the return type is a list, this is considered hierarchical result dict.
See the three examples in this file:
attribute_transformer
full_restapi_key_transformer
last_restapi_key_transformer
If you want XML serialization, you can pass the kwargs is_xml=True.
- Parameters
key_transformer (function) – A key transformer function.
- Returns
A dict JSON compatible object
- Return type
-
classmethod
deserialize
(data, content_type=None)¶ Parse a str using the RestAPI syntax and return a model.
-
classmethod
enable_additional_properties_sending
()¶
-
classmethod
from_dict
(data, key_extractors=None, content_type=None)¶ Parse a dict using given key extractor return a model.
By default consider key extractors (rest_key_case_insensitive_extractor, attribute_key_case_insensitive_extractor and last_rest_key_case_insensitive_extractor)
-
classmethod
is_xml_model
()¶
-
serialize
(keep_readonly=False, **kwargs)¶ Return the JSON that would be sent to azure from this model.
This is an alias to as_dict(full_restapi_key_transformer, keep_readonly=False).
If you want XML serialization, you can pass the kwargs is_xml=True.
-
class
azure.storage.filedatalake.
PathProperties
(**kwargs)[source]¶ Path properties listed by get_paths api.
- Variables
name (str) – The full path for a file or directory.
owner (str) – The owner of the file or directory.
group (str) – The owning group of the file or directory.
permissions (str) – Sets POSIX access permissions for the file owner, the file owning group, and others. Each class may be granted read, write, or execute permission. The sticky bit is also supported. Both symbolic (rwxrw-rw-) and 4-digit octal notation (e.g. 0766) are supported.
last_modified (datetime) – A datetime object representing the last time the directory/file was modified.
is_directory (bool) – Is the path a directory or not.
etag (str) – The ETag contains a value that you can use to perform operations conditionally.
content_length (int) – The size of file if the path is a file.
creation_time (datetime) – The creation time of the file/directory.
expiry_time (datetime) – The expiry time of the file/directory.
encryption_scope (str) – A predefined encryption scope used to encrypt the data on the service. An encryption scope can be created using the Management API and referenced here by name. If a default encryption scope has been defined at the file system, this value will override it if the file system level scope is configured to allow overrides. Otherwise an error will be raised.
-
get
(key, default=None)¶
-
has_key
(k)¶
-
items
()¶
-
keys
()¶
-
update
(*args, **kwargs)¶
-
values
()¶
-
class
azure.storage.filedatalake.
PublicAccess
(value)[source]¶ Specifies whether data in the file system may be accessed publicly and the level of access.
-
FILE
= 'blob'¶ Specifies public read access for files. file data within this file system can be read via anonymous request, but file system data is not available. Clients cannot enumerate files within the container via anonymous request.
-
FILESYSTEM
= 'container'¶ Specifies full public read access for file system and file data. Clients can enumerate files within the file system via anonymous request, but cannot enumerate file systems within the storage account.
-
-
class
azure.storage.filedatalake.
QuickQueryDialect
(value)[source]¶ Specifies the quick query input/output dialect.
-
DELIMITEDJSON
= 'DelimitedJsonDialect'¶
-
DELIMITEDTEXT
= 'DelimitedTextDialect'¶
-
PARQUET
= 'ParquetDialect'¶
-
-
class
azure.storage.filedatalake.
ResourceTypes
(service=False, file_system=False, object=False)[source]¶ Specifies the resource types that are accessible with the account SAS.
- Parameters
-
classmethod
from_string
(string)[source]¶ Create a ResourceTypes from a string.
To specify service, container, or object you need only to include the first letter of the word in the string. E.g. service and container, you would provide a string “sc”.
- Parameters
string (str) – Specify service, container, or object in in the string with the first letter of the word.
- Returns
A ResourceTypes object
- Return type
ResourceTypes
-
class
azure.storage.filedatalake.
RetentionPolicy
(enabled=False, days=None)[source]¶ The retention policy which determines how long the associated data should persist.
- Parameters
enabled (bool) – Indicates whether a retention policy is enabled for the storage service. The default value is False.
days (int) – Indicates the number of days that metrics or logging or soft-deleted data should be retained. All data older than this value will be deleted. If enabled=True, the number of days must be specified.
- Keyword Arguments
enabled (bool) – Indicates whether a retention policy is enabled for the storage service. Required.
days (int) – Indicates the number of days that metrics or logging or soft-deleted data should be retained. All data older than this value will be deleted.
allow_permanent_delete (bool) – Indicates whether permanent delete is allowed on this storage account.
-
as_dict
(keep_readonly=True, key_transformer=<function attribute_transformer>, **kwargs)¶ Return a dict that can be JSONify using json.dump.
Advanced usage might optionally use a callback as parameter:
Key is the attribute name used in Python. Attr_desc is a dict of metadata. Currently contains ‘type’ with the msrest type and ‘key’ with the RestAPI encoded key. Value is the current value in this object.
The string returned will be used to serialize the key. If the return type is a list, this is considered hierarchical result dict.
See the three examples in this file:
attribute_transformer
full_restapi_key_transformer
last_restapi_key_transformer
If you want XML serialization, you can pass the kwargs is_xml=True.
- Parameters
key_transformer (function) – A key transformer function.
- Returns
A dict JSON compatible object
- Return type
-
classmethod
deserialize
(data, content_type=None)¶ Parse a str using the RestAPI syntax and return a model.
-
classmethod
enable_additional_properties_sending
()¶
-
classmethod
from_dict
(data, key_extractors=None, content_type=None)¶ Parse a dict using given key extractor return a model.
By default consider key extractors (rest_key_case_insensitive_extractor, attribute_key_case_insensitive_extractor and last_rest_key_case_insensitive_extractor)
-
classmethod
is_xml_model
()¶
-
serialize
(keep_readonly=False, **kwargs)¶ Return the JSON that would be sent to azure from this model.
This is an alias to as_dict(full_restapi_key_transformer, keep_readonly=False).
If you want XML serialization, you can pass the kwargs is_xml=True.
-
class
azure.storage.filedatalake.
StaticWebsite
(**kwargs)[source]¶ The properties that enable an account to host a static website.
- Keyword Arguments
enabled (bool) – Indicates whether this account is hosting a static website. The default value is False.
index_document (str) – The default name of the index page under each directory.
error_document404_path (str) – The absolute path of the custom 404 page.
default_index_document_path (str) – Absolute path of the default index page.
enabled – Indicates whether this account is hosting a static website. Required.
index_document – The default name of the index page under each directory.
error_document404_path – The absolute path of the custom 404 page.
default_index_document_path – Absolute path of the default index page.
-
as_dict
(keep_readonly=True, key_transformer=<function attribute_transformer>, **kwargs)¶ Return a dict that can be JSONify using json.dump.
Advanced usage might optionally use a callback as parameter:
Key is the attribute name used in Python. Attr_desc is a dict of metadata. Currently contains ‘type’ with the msrest type and ‘key’ with the RestAPI encoded key. Value is the current value in this object.
The string returned will be used to serialize the key. If the return type is a list, this is considered hierarchical result dict.
See the three examples in this file:
attribute_transformer
full_restapi_key_transformer
last_restapi_key_transformer
If you want XML serialization, you can pass the kwargs is_xml=True.
- Parameters
key_transformer (function) – A key transformer function.
- Returns
A dict JSON compatible object
- Return type
-
classmethod
deserialize
(data, content_type=None)¶ Parse a str using the RestAPI syntax and return a model.
-
classmethod
enable_additional_properties_sending
()¶
-
classmethod
from_dict
(data, key_extractors=None, content_type=None)¶ Parse a dict using given key extractor return a model.
By default consider key extractors (rest_key_case_insensitive_extractor, attribute_key_case_insensitive_extractor and last_rest_key_case_insensitive_extractor)
-
classmethod
is_xml_model
()¶
-
serialize
(keep_readonly=False, **kwargs)¶ Return the JSON that would be sent to azure from this model.
This is an alias to as_dict(full_restapi_key_transformer, keep_readonly=False).
If you want XML serialization, you can pass the kwargs is_xml=True.
-
class
azure.storage.filedatalake.
StorageErrorCode
(value)[source]¶ An enumeration.
-
ACCOUNT_ALREADY_EXISTS
= 'AccountAlreadyExists'¶
-
ACCOUNT_BEING_CREATED
= 'AccountBeingCreated'¶
-
ACCOUNT_IS_DISABLED
= 'AccountIsDisabled'¶
-
APPEND_POSITION_CONDITION_NOT_MET
= 'AppendPositionConditionNotMet'¶
-
AUTHENTICATION_FAILED
= 'AuthenticationFailed'¶
-
AUTHORIZATION_FAILURE
= 'AuthorizationFailure'¶
-
BLOB_ALREADY_EXISTS
= 'BlobAlreadyExists'¶
-
BLOB_ARCHIVED
= 'BlobArchived'¶
-
BLOB_BEING_REHYDRATED
= 'BlobBeingRehydrated'¶
-
BLOB_NOT_ARCHIVED
= 'BlobNotArchived'¶
-
BLOB_NOT_FOUND
= 'BlobNotFound'¶
-
BLOB_OVERWRITTEN
= 'BlobOverwritten'¶
-
BLOB_TIER_INADEQUATE_FOR_CONTENT_LENGTH
= 'BlobTierInadequateForContentLength'¶
-
BLOCK_COUNT_EXCEEDS_LIMIT
= 'BlockCountExceedsLimit'¶
-
BLOCK_LIST_TOO_LONG
= 'BlockListTooLong'¶
-
CANNOT_CHANGE_TO_LOWER_TIER
= 'CannotChangeToLowerTier'¶
-
CANNOT_DELETE_FILE_OR_DIRECTORY
= 'CannotDeleteFileOrDirectory'¶
-
CANNOT_VERIFY_COPY_SOURCE
= 'CannotVerifyCopySource'¶
-
CLIENT_CACHE_FLUSH_DELAY
= 'ClientCacheFlushDelay'¶
-
CONDITION_HEADERS_NOT_SUPPORTED
= 'ConditionHeadersNotSupported'¶
-
CONDITION_NOT_MET
= 'ConditionNotMet'¶
-
CONTAINER_ALREADY_EXISTS
= 'ContainerAlreadyExists'¶
-
CONTAINER_BEING_DELETED
= 'ContainerBeingDeleted'¶
-
CONTAINER_DISABLED
= 'ContainerDisabled'¶
-
CONTAINER_NOT_FOUND
= 'ContainerNotFound'¶
-
CONTAINER_QUOTA_DOWNGRADE_NOT_ALLOWED
= 'ContainerQuotaDowngradeNotAllowed'¶
-
CONTENT_LENGTH_LARGER_THAN_TIER_LIMIT
= 'ContentLengthLargerThanTierLimit'¶
-
CONTENT_LENGTH_MUST_BE_ZERO
= 'ContentLengthMustBeZero'¶
-
COPY_ACROSS_ACCOUNTS_NOT_SUPPORTED
= 'CopyAcrossAccountsNotSupported'¶
-
COPY_ID_MISMATCH
= 'CopyIdMismatch'¶
-
DELETE_PENDING
= 'DeletePending'¶
-
DESTINATION_PATH_IS_BEING_DELETED
= 'DestinationPathIsBeingDeleted'¶
-
DIRECTORY_NOT_EMPTY
= 'DirectoryNotEmpty'¶
-
EMPTY_METADATA_KEY
= 'EmptyMetadataKey'¶
-
FEATURE_VERSION_MISMATCH
= 'FeatureVersionMismatch'¶
-
FILE_LOCK_CONFLICT
= 'FileLockConflict'¶
-
FILE_SYSTEM_ALREADY_EXISTS
= 'FilesystemAlreadyExists'¶
-
FILE_SYSTEM_BEING_DELETED
= 'FilesystemBeingDeleted'¶
-
FILE_SYSTEM_NOT_FOUND
= 'FilesystemNotFound'¶
-
INCREMENTAL_COPY_BLOB_MISMATCH
= 'IncrementalCopyBlobMismatch'¶
-
INCREMENTAL_COPY_OF_ERALIER_VERSION_SNAPSHOT_NOT_ALLOWED
= 'IncrementalCopyOfEralierVersionSnapshotNotAllowed'¶
-
INCREMENTAL_COPY_SOURCE_MUST_BE_SNAPSHOT
= 'IncrementalCopySourceMustBeSnapshot'¶
-
INFINITE_LEASE_DURATION_REQUIRED
= 'InfiniteLeaseDurationRequired'¶
-
INSUFFICIENT_ACCOUNT_PERMISSIONS
= 'InsufficientAccountPermissions'¶
-
INTERNAL_ERROR
= 'InternalError'¶
-
INVALID_AUTHENTICATION_INFO
= 'InvalidAuthenticationInfo'¶
-
INVALID_BLOB_OR_BLOCK
= 'InvalidBlobOrBlock'¶
-
INVALID_BLOB_TIER
= 'InvalidBlobTier'¶
-
INVALID_BLOB_TYPE
= 'InvalidBlobType'¶
-
INVALID_BLOCK_ID
= 'InvalidBlockId'¶
-
INVALID_BLOCK_LIST
= 'InvalidBlockList'¶
-
INVALID_DESTINATION_PATH
= 'InvalidDestinationPath'¶
-
INVALID_FILE_OR_DIRECTORY_PATH_NAME
= 'InvalidFileOrDirectoryPathName'¶
-
INVALID_FLUSH_POSITION
= 'InvalidFlushPosition'¶
-
INVALID_HEADER_VALUE
= 'InvalidHeaderValue'¶
-
INVALID_HTTP_VERB
= 'InvalidHttpVerb'¶
-
INVALID_INPUT
= 'InvalidInput'¶
-
INVALID_MARKER
= 'InvalidMarker'¶
-
INVALID_MD5
= 'InvalidMd5'¶
-
INVALID_METADATA
= 'InvalidMetadata'¶
-
INVALID_OPERATION
= 'InvalidOperation'¶
-
INVALID_PAGE_RANGE
= 'InvalidPageRange'¶
-
INVALID_PROPERTY_NAME
= 'InvalidPropertyName'¶
-
INVALID_QUERY_PARAMETER_VALUE
= 'InvalidQueryParameterValue'¶
-
INVALID_RANGE
= 'InvalidRange'¶
-
INVALID_RENAME_SOURCE_PATH
= 'InvalidRenameSourcePath'¶
-
INVALID_RESOURCE_NAME
= 'InvalidResourceName'¶
-
INVALID_SOURCE_BLOB_TYPE
= 'InvalidSourceBlobType'¶
-
INVALID_SOURCE_BLOB_URL
= 'InvalidSourceBlobUrl'¶
-
INVALID_SOURCE_OR_DESTINATION_RESOURCE_TYPE
= 'InvalidSourceOrDestinationResourceType'¶
-
INVALID_SOURCE_URI
= 'InvalidSourceUri'¶
-
INVALID_URI
= 'InvalidUri'¶
-
INVALID_VERSION_FOR_PAGE_BLOB_OPERATION
= 'InvalidVersionForPageBlobOperation'¶
-
INVALID_XML_DOCUMENT
= 'InvalidXmlDocument'¶
-
INVALID_XML_NODE_VALUE
= 'InvalidXmlNodeValue'¶
-
LEASE_ALREADY_BROKEN
= 'LeaseAlreadyBroken'¶
-
LEASE_ALREADY_PRESENT
= 'LeaseAlreadyPresent'¶
-
LEASE_ID_MISMATCH_WITH_BLOB_OPERATION
= 'LeaseIdMismatchWithBlobOperation'¶
-
LEASE_ID_MISMATCH_WITH_CONTAINER_OPERATION
= 'LeaseIdMismatchWithContainerOperation'¶
-
LEASE_ID_MISMATCH_WITH_LEASE_OPERATION
= 'LeaseIdMismatchWithLeaseOperation'¶
-
LEASE_ID_MISSING
= 'LeaseIdMissing'¶
-
LEASE_IS_ALREADY_BROKEN
= 'LeaseIsAlreadyBroken'¶
-
LEASE_IS_BREAKING_AND_CANNOT_BE_ACQUIRED
= 'LeaseIsBreakingAndCannotBeAcquired'¶
-
LEASE_IS_BREAKING_AND_CANNOT_BE_CHANGED
= 'LeaseIsBreakingAndCannotBeChanged'¶
-
LEASE_IS_BROKEN_AND_CANNOT_BE_RENEWED
= 'LeaseIsBrokenAndCannotBeRenewed'¶
-
LEASE_LOST
= 'LeaseLost'¶
-
LEASE_NAME_MISMATCH
= 'LeaseNameMismatch'¶
-
LEASE_NOT_PRESENT_WITH_BLOB_OPERATION
= 'LeaseNotPresentWithBlobOperation'¶
-
LEASE_NOT_PRESENT_WITH_CONTAINER_OPERATION
= 'LeaseNotPresentWithContainerOperation'¶
-
LEASE_NOT_PRESENT_WITH_LEASE_OPERATION
= 'LeaseNotPresentWithLeaseOperation'¶
-
MAX_BLOB_SIZE_CONDITION_NOT_MET
= 'MaxBlobSizeConditionNotMet'¶
-
MD5_MISMATCH
= 'Md5Mismatch'¶
-
MESSAGE_NOT_FOUND
= 'MessageNotFound'¶
-
MESSAGE_TOO_LARGE
= 'MessageTooLarge'¶
-
METADATA_TOO_LARGE
= 'MetadataTooLarge'¶
-
MISSING_CONTENT_LENGTH_HEADER
= 'MissingContentLengthHeader'¶
-
MISSING_REQUIRED_HEADER
= 'MissingRequiredHeader'¶
-
MISSING_REQUIRED_QUERY_PARAMETER
= 'MissingRequiredQueryParameter'¶
-
MISSING_REQUIRED_XML_NODE
= 'MissingRequiredXmlNode'¶
-
MULTIPLE_CONDITION_HEADERS_NOT_SUPPORTED
= 'MultipleConditionHeadersNotSupported'¶
-
NO_AUTHENTICATION_INFORMATION
= 'NoAuthenticationInformation'¶
-
NO_PENDING_COPY_OPERATION
= 'NoPendingCopyOperation'¶
-
OPERATION_NOT_ALLOWED_ON_INCREMENTAL_COPY_BLOB
= 'OperationNotAllowedOnIncrementalCopyBlob'¶
-
OPERATION_TIMED_OUT
= 'OperationTimedOut'¶
-
OUT_OF_RANGE_INPUT
= 'OutOfRangeInput'¶
-
OUT_OF_RANGE_QUERY_PARAMETER_VALUE
= 'OutOfRangeQueryParameterValue'¶
-
PARENT_NOT_FOUND
= 'ParentNotFound'¶
-
PATH_ALREADY_EXISTS
= 'PathAlreadyExists'¶
-
PATH_CONFLICT
= 'PathConflict'¶
-
PATH_NOT_FOUND
= 'PathNotFound'¶
-
PENDING_COPY_OPERATION
= 'PendingCopyOperation'¶
-
POP_RECEIPT_MISMATCH
= 'PopReceiptMismatch'¶
-
PREVIOUS_SNAPSHOT_CANNOT_BE_NEWER
= 'PreviousSnapshotCannotBeNewer'¶
-
PREVIOUS_SNAPSHOT_NOT_FOUND
= 'PreviousSnapshotNotFound'¶
-
PREVIOUS_SNAPSHOT_OPERATION_NOT_SUPPORTED
= 'PreviousSnapshotOperationNotSupported'¶
-
QUEUE_ALREADY_EXISTS
= 'QueueAlreadyExists'¶
-
QUEUE_BEING_DELETED
= 'QueueBeingDeleted'¶
-
QUEUE_DISABLED
= 'QueueDisabled'¶
-
QUEUE_NOT_EMPTY
= 'QueueNotEmpty'¶
-
QUEUE_NOT_FOUND
= 'QueueNotFound'¶
-
READ_ONLY_ATTRIBUTE
= 'ReadOnlyAttribute'¶
-
RENAME_DESTINATION_PARENT_PATH_NOT_FOUND
= 'RenameDestinationParentPathNotFound'¶
-
REQUEST_BODY_TOO_LARGE
= 'RequestBodyTooLarge'¶
-
REQUEST_URL_FAILED_TO_PARSE
= 'RequestUrlFailedToParse'¶
-
RESOURCE_ALREADY_EXISTS
= 'ResourceAlreadyExists'¶
-
RESOURCE_NOT_FOUND
= 'ResourceNotFound'¶
-
RESOURCE_TYPE_MISMATCH
= 'ResourceTypeMismatch'¶
-
SEQUENCE_NUMBER_CONDITION_NOT_MET
= 'SequenceNumberConditionNotMet'¶
-
SEQUENCE_NUMBER_INCREMENT_TOO_LARGE
= 'SequenceNumberIncrementTooLarge'¶
-
SERVER_BUSY
= 'ServerBusy'¶
-
SHARE_ALREADY_EXISTS
= 'ShareAlreadyExists'¶
-
SHARE_BEING_DELETED
= 'ShareBeingDeleted'¶
-
SHARE_DISABLED
= 'ShareDisabled'¶
-
SHARE_HAS_SNAPSHOTS
= 'ShareHasSnapshots'¶
-
SHARE_NOT_FOUND
= 'ShareNotFound'¶
-
SHARE_SNAPSHOT_COUNT_EXCEEDED
= 'ShareSnapshotCountExceeded'¶
-
SHARE_SNAPSHOT_IN_PROGRESS
= 'ShareSnapshotInProgress'¶
-
SHARE_SNAPSHOT_OPERATION_NOT_SUPPORTED
= 'ShareSnapshotOperationNotSupported'¶
-
SHARING_VIOLATION
= 'SharingViolation'¶
-
SNAPHOT_OPERATION_RATE_EXCEEDED
= 'SnaphotOperationRateExceeded'¶
-
SNAPSHOTS_PRESENT
= 'SnapshotsPresent'¶
-
SNAPSHOT_COUNT_EXCEEDED
= 'SnapshotCountExceeded'¶
-
SOURCE_CONDITION_NOT_MET
= 'SourceConditionNotMet'¶
-
SOURCE_PATH_IS_BEING_DELETED
= 'SourcePathIsBeingDeleted'¶
-
SOURCE_PATH_NOT_FOUND
= 'SourcePathNotFound'¶
-
SYSTEM_IN_USE
= 'SystemInUse'¶
-
TARGET_CONDITION_NOT_MET
= 'TargetConditionNotMet'¶
-
UNAUTHORIZED_BLOB_OVERWRITE
= 'UnauthorizedBlobOverwrite'¶
-
UNSUPPORTED_HEADER
= 'UnsupportedHeader'¶
-
UNSUPPORTED_HTTP_VERB
= 'UnsupportedHttpVerb'¶
-
UNSUPPORTED_QUERY_PARAMETER
= 'UnsupportedQueryParameter'¶
-
UNSUPPORTED_REST_VERSION
= 'UnsupportedRestVersion'¶
-
UNSUPPORTED_XML_NODE
= 'UnsupportedXmlNode'¶
-
-
class
azure.storage.filedatalake.
StorageStreamDownloader
(downloader)[source]¶ A streaming object to download from Azure Storage.
- Variables
name (str) – The name of the file being downloaded.
properties (FileProperties) – The properties of the file being downloaded. If only a range of the data is being downloaded, this will be reflected in the properties.
size (int) – The size of the total data in the stream. This will be the byte range if speficied, otherwise the total size of the file.
-
chunks
() → Iterator[bytes][source]¶ Iterate over chunks in the download stream.
- Return type
Iterator[bytes]
-
read
(size: Optional[int] = - 1) → bytes[source]¶ Read up to size bytes from the stream and return them. If size is unspecified or is -1, all bytes will be read.
- Parameters
size – The number of bytes to download from the stream. Leave unsepcified or set to -1 to download all bytes.
- Returns
The requsted data as bytes. If the return value is empty, there is no more data to read.
- Return type
-
readall
() → bytes[source]¶ Download the contents of this file.
This operation is blocking until all data is downloaded. :rtype: bytes
-
readinto
(stream) → int[source]¶ Download the contents of this file to a stream.
- Parameters
stream – The stream to download to. This can be an open file-handle, or any writable stream. The stream must be seekable if the download uses more than one parallel connection.
- Returns
The number of bytes read.
- Return type
-
class
azure.storage.filedatalake.
UserDelegationKey
[source]¶ Represents a user delegation key, provided to the user by Azure Storage based on their Azure Active Directory access token.
The fields are saved as simple strings since the user does not have to interact with this object; to generate an identify SAS, the user can simply pass it to the right API.
- Variables
signed_oid (str) – Object ID of this token.
signed_tid (str) – Tenant ID of the tenant that issued this token.
signed_start (str) – The datetime this token becomes valid.
signed_expiry (str) – The datetime this token expires.
signed_service (str) – What service this key is valid for.
signed_version (str) – The version identifier of the REST service that created this token.
value (str) – The user delegation key.
-
azure.storage.filedatalake.
generate_account_sas
(account_name: str, account_key: str, resource_types: Union[ResourceTypes, str], permission: Union[AccountSasPermissions, str], expiry: Optional[Union[datetime, str]], **kwargs: Any) → str[source]¶ Generates a shared access signature for the DataLake service.
Use the returned signature as the credential parameter of any DataLakeServiceClient, FileSystemClient, DataLakeDirectoryClient or DataLakeFileClient.
- Parameters
account_name (str) – The storage account name used to generate the shared access signature.
account_key (str) – The access key to generate the shared access signature.
resource_types (str or ResourceTypes) – Specifies the resource types that are accessible with the account SAS.
permission (str or AccountSasPermissions) – The permissions associated with the shared access signature. The user is restricted to operations allowed by the permissions. Required unless an id is given referencing a stored access policy which contains this field. This field must be omitted if it has been specified in an associated stored access policy.
expiry (datetime or str) – The time at which the shared access signature becomes invalid. Required unless an id is given referencing a stored access policy which contains this field. This field must be omitted if it has been specified in an associated stored access policy. Azure will always convert values to UTC. If a date is passed in without timezone info, it is assumed to be UTC.
- Keyword Arguments
start (datetime or str) – The time at which the shared access signature becomes valid. If omitted, start time for this call is assumed to be the time when the storage service receives the request. Azure will always convert values to UTC. If a date is passed in without timezone info, it is assumed to be UTC.
ip (str) – Specifies an IP address or a range of IP addresses from which to accept requests. If the IP address from which the request originates does not match the IP address or address range specified on the SAS token, the request is not authenticated. For example, specifying ip=168.1.5.65 or ip=168.1.5.60-168.1.5.70 on the SAS restricts the request to those IP addresses.
protocol (str) – Specifies the protocol permitted for a request made. The default value is https.
encryption_scope (str) – Specifies the encryption scope for a request made so that all write operations will be service encrypted.
- Returns
A Shared Access Signature (sas) token.
- Return type
-
azure.storage.filedatalake.
generate_directory_sas
(account_name: str, file_system_name: str, directory_name: str, credential: Union[str, UserDelegationKey], permission: Optional[Union[FileSasPermissions, str]] = None, expiry: Optional[Union[datetime, str]] = None, **kwargs: Any) → str[source]¶ Generates a shared access signature for a directory.
Use the returned signature with the credential parameter of any DataLakeServiceClient, FileSystemClient, DataLakeDirectoryClient or DataLakeFileClient.
- Parameters
account_name (str) – The storage account name used to generate the shared access signature.
file_system_name (str) – The name of the file system.
directory_name (str) – The name of the directory.
credential (str or UserDelegationKey) – Credential could be either account key or user delegation key. If use account key is used as credential, then the credential type should be a str. Instead of an account key, the user could also pass in a user delegation key. A user delegation key can be obtained from the service by authenticating with an AAD identity; this can be accomplished by calling
get_user_delegation_key()
. When present, the SAS is signed with the user delegation key instead.permission (str or FileSasPermissions) – The permissions associated with the shared access signature. The user is restricted to operations allowed by the permissions. Permissions must be ordered racwdlmeop. Required unless an id is given referencing a stored access policy which contains this field. This field must be omitted if it has been specified in an associated stored access policy.
expiry (datetime or str) – The time at which the shared access signature becomes invalid. Required unless an id is given referencing a stored access policy which contains this field. This field must be omitted if it has been specified in an associated stored access policy. Azure will always convert values to UTC. If a date is passed in without timezone info, it is assumed to be UTC.
- Keyword Arguments
start (datetime or str) – The time at which the shared access signature becomes valid. If omitted, start time for this call is assumed to be the time when the storage service receives the request. Azure will always convert values to UTC. If a date is passed in without timezone info, it is assumed to be UTC.
ip (str) – Specifies an IP address or a range of IP addresses from which to accept requests. If the IP address from which the request originates does not match the IP address or address range specified on the SAS token, the request is not authenticated. For example, specifying ip=168.1.5.65 or ip=168.1.5.60-168.1.5.70 on the SAS restricts the request to those IP addresses.
protocol (str) – Specifies the protocol permitted for a request made. The default value is https.
cache_control (str) – Response header value for Cache-Control when resource is accessed using this shared access signature.
content_disposition (str) – Response header value for Content-Disposition when resource is accessed using this shared access signature.
content_encoding (str) – Response header value for Content-Encoding when resource is accessed using this shared access signature.
content_language (str) – Response header value for Content-Language when resource is accessed using this shared access signature.
content_type (str) – Response header value for Content-Type when resource is accessed using this shared access signature.
preauthorized_agent_object_id (str) – The AAD object ID of a user assumed to be authorized by the owner of the user delegation key to perform the action granted by the SAS token. The service will validate the SAS token and ensure that the owner of the user delegation key has the required permissions before granting access but no additional permission check for the agent object id will be performed.
agent_object_id (str) – The AAD object ID of a user assumed to be unauthorized by the owner of the user delegation key to perform the action granted by the SAS token. The service will validate the SAS token and ensure that the owner of the user delegation key has the required permissions before granting access and the service will perform an additional POSIX ACL check to determine if this user is authorized to perform the requested operation.
correlation_id (str) – The correlation id to correlate the storage audit logs with the audit logs used by the principal generating and distributing the SAS.
encryption_scope (str) – Specifies the encryption scope for a request made so that all write operations will be service encrypted.
- Returns
A Shared Access Signature (sas) token.
- Return type
-
azure.storage.filedatalake.
generate_file_sas
(account_name: str, file_system_name: str, directory_name: str, file_name: str, credential: Union[str, UserDelegationKey], permission: Optional[Union[FileSasPermissions, str]] = None, expiry: Optional[Union[datetime, str]] = None, **kwargs: Any) → str[source]¶ Generates a shared access signature for a file.
Use the returned signature with the credential parameter of any BDataLakeServiceClient, FileSystemClient, DataLakeDirectoryClient or DataLakeFileClient.
- Parameters
account_name (str) – The storage account name used to generate the shared access signature.
file_system_name (str) – The name of the file system.
directory_name (str) – The name of the directory.
file_name (str) – The name of the file.
credential (str or UserDelegationKey) – Credential could be either account key or user delegation key. If use account key is used as credential, then the credential type should be a str. Instead of an account key, the user could also pass in a user delegation key. A user delegation key can be obtained from the service by authenticating with an AAD identity; this can be accomplished by calling
get_user_delegation_key()
. When present, the SAS is signed with the user delegation key instead.permission (str or FileSasPermissions) – The permissions associated with the shared access signature. The user is restricted to operations allowed by the permissions. Permissions must be ordered racwdmeop. Required unless an id is given referencing a stored access policy which contains this field. This field must be omitted if it has been specified in an associated stored access policy.
expiry (datetime or str) – The time at which the shared access signature becomes invalid. Required unless an id is given referencing a stored access policy which contains this field. This field must be omitted if it has been specified in an associated stored access policy. Azure will always convert values to UTC. If a date is passed in without timezone info, it is assumed to be UTC.
- Keyword Arguments
start (datetime or str) – The time at which the shared access signature becomes valid. If omitted, start time for this call is assumed to be the time when the storage service receives the request. Azure will always convert values to UTC. If a date is passed in without timezone info, it is assumed to be UTC.
ip (str) – Specifies an IP address or a range of IP addresses from which to accept requests. If the IP address from which the request originates does not match the IP address or address range specified on the SAS token, the request is not authenticated. For example, specifying ip=168.1.5.65 or ip=168.1.5.60-168.1.5.70 on the SAS restricts the request to those IP addresses.
protocol (str) – Specifies the protocol permitted for a request made. The default value is https.
cache_control (str) – Response header value for Cache-Control when resource is accessed using this shared access signature.
content_disposition (str) – Response header value for Content-Disposition when resource is accessed using this shared access signature.
content_encoding (str) – Response header value for Content-Encoding when resource is accessed using this shared access signature.
content_language (str) – Response header value for Content-Language when resource is accessed using this shared access signature.
content_type (str) – Response header value for Content-Type when resource is accessed using this shared access signature.
preauthorized_agent_object_id (str) – The AAD object ID of a user assumed to be authorized by the owner of the user delegation key to perform the action granted by the SAS token. The service will validate the SAS token and ensure that the owner of the user delegation key has the required permissions before granting access but no additional permission check for the agent object id will be performed.
agent_object_id (str) – The AAD object ID of a user assumed to be unauthorized by the owner of the user delegation key to perform the action granted by the SAS token. The service will validate the SAS token and ensure that the owner of the user delegation key has the required permissions before granting access and the service will perform an additional POSIX ACL check to determine if this user is authorized to perform the requested operation.
correlation_id (str) – The correlation id to correlate the storage audit logs with the audit logs used by the principal generating and distributing the SAS. This can only be used when to generate sas with delegation key.
encryption_scope (str) – Specifies the encryption scope for a request made so that all write operations will be service encrypted.
- Returns
A Shared Access Signature (sas) token.
- Return type
-
azure.storage.filedatalake.
generate_file_system_sas
(account_name: str, file_system_name: str, credential: Union[str, UserDelegationKey], permission: Optional[Union[FileSystemSasPermissions, str]] = None, expiry: Optional[Union[datetime, str]] = None, **kwargs: Any) → str[source]¶ Generates a shared access signature for a file system.
Use the returned signature with the credential parameter of any DataLakeServiceClient, FileSystemClient, DataLakeDirectoryClient or DataLakeFileClient.
- Parameters
account_name (str) – The storage account name used to generate the shared access signature.
file_system_name (str) – The name of the file system.
credential (str or UserDelegationKey) – Credential could be either account key or user delegation key. If use account key is used as credential, then the credential type should be a str. Instead of an account key, the user could also pass in a user delegation key. A user delegation key can be obtained from the service by authenticating with an AAD identity; this can be accomplished by calling
get_user_delegation_key()
. When present, the SAS is signed with the user delegation key instead.permission (str or FileSystemSasPermissions) – The permissions associated with the shared access signature. The user is restricted to operations allowed by the permissions. Permissions must be ordered racwdlmeop. Required unless an id is given referencing a stored access policy which contains this field. This field must be omitted if it has been specified in an associated stored access policy.
expiry (datetime or str) – The time at which the shared access signature becomes invalid. Required unless an id is given referencing a stored access policy which contains this field. This field must be omitted if it has been specified in an associated stored access policy. Azure will always convert values to UTC. If a date is passed in without timezone info, it is assumed to be UTC.
- Keyword Arguments
start (datetime or str) – The time at which the shared access signature becomes valid. If omitted, start time for this call is assumed to be the time when the storage service receives the request. Azure will always convert values to UTC. If a date is passed in without timezone info, it is assumed to be UTC.
ip (str) – Specifies an IP address or a range of IP addresses from which to accept requests. If the IP address from which the request originates does not match the IP address or address range specified on the SAS token, the request is not authenticated. For example, specifying ip=168.1.5.65 or ip=168.1.5.60-168.1.5.70 on the SAS restricts the request to those IP addresses.
protocol (str) – Specifies the protocol permitted for a request made. The default value is https.
cache_control (str) – Response header value for Cache-Control when resource is accessed using this shared access signature.
content_disposition (str) – Response header value for Content-Disposition when resource is accessed using this shared access signature.
content_encoding (str) – Response header value for Content-Encoding when resource is accessed using this shared access signature.
content_language (str) – Response header value for Content-Language when resource is accessed using this shared access signature.
content_type (str) – Response header value for Content-Type when resource is accessed using this shared access signature.
preauthorized_agent_object_id (str) – The AAD object ID of a user assumed to be authorized by the owner of the user delegation key to perform the action granted by the SAS token. The service will validate the SAS token and ensure that the owner of the user delegation key has the required permissions before granting access but no additional permission check for the agent object id will be performed.
agent_object_id (str) – The AAD object ID of a user assumed to be unauthorized by the owner of the user delegation key to perform the action granted by the SAS token. The service will validate the SAS token and ensure that the owner of the user delegation key has the required permissions before granting access and the service will perform an additional POSIX ACL check to determine if this user is authorized to perform the requested operation.
correlation_id (str) – The correlation id to correlate the storage audit logs with the audit logs used by the principal generating and distributing the SAS.
encryption_scope (str) – Specifies the encryption scope for a request made so that all write operations will be service encrypted.
- Returns
A Shared Access Signature (sas) token.
- Return type