azure.eventhub.extensions.checkpointstoreblob package

class azure.eventhub.extensions.checkpointstoreblob.BlobCheckpointStore(blob_account_url, container_name, credential=None, **kwargs)[source]

A CheckpointStore that uses Azure Blob Storage to store the partition ownership and checkpoint data.

This class implements methods list_ownership, claim_ownership, update_checkpoint and list_checkpoints that are defined in class azure.eventhub.aio.CheckpointStore of package azure-eventhub.

Parameters

  • account_url (str) – The URI to the storage account. In order to create a client given the full URI to the container, use the from_container_url() classmethod.

  • container_name (str) – The name of the container for the blob.

  • credential – The credentials with which to authenticate. This is optional if the account URL already has a SAS token. The value can be a SAS token string, an account shared access key, or an instance of a TokenCredentials class from azure.identity. If the URL already has a SAS token, specifying an explicit credential will take priority.

Keyword Arguments

  • api_version (str) – The Storage API version to use for requests. Default value is ‘2019-07-07’.

  • secondary_hostname (str) – The hostname of the secondary endpoint.

claim_ownership(ownership_list: Iterable[Dict[str, Any]], **kwargs: Any) → Iterable[Dict[str, Any]][source]

Tries to claim ownership for a list of specified partitions.

Parameters

ownership_list (Iterable[Dict[str,Any]]) – Iterable of dictionaries containing all the ownerships to claim.

Return type

Iterable[Dict[str,Any]], Iterable of dictionaries containing partition ownership information:

  • fully_qualified_namespace (str): The fully qualified namespace that the Event Hub belongs to. The format is like “<namespace>.servicebus.windows.net”.

  • eventhub_name (str): The name of the specific Event Hub the checkpoint is associated with, relative to the Event Hubs namespace that contains it.

  • consumer_group (str): The name of the consumer group the ownership are associated with.

  • partition_id (str): The partition ID which the checkpoint is created for.

  • owner_id (str): A UUID representing the owner attempting to claim this partition.

  • last_modified_time (UTC datetime.datetime): The last time this ownership was claimed.

  • etag (str): The Etag value for the last time this ownership was modified. Optional depending on storage implementation.

close()[source]
classmethod from_connection_string(conn_str: str, container_name: str, credential: Optional[Any] = None, **kwargs: Any) → azure.eventhub.extensions.checkpointstoreblob._blobstoragecs.BlobCheckpointStore[source]

Create BlobCheckpointStore from a storage connection string.

Parameters

  • conn_str (str) – A connection string to an Azure Storage account.

  • container_name (str) – The container name for the blob.

  • credential – The credentials with which to authenticate. This is optional if the account URL already has a SAS token, or the connection string already has shared access key values. The value can be a SAS token string, an account shared access key, or an instance of a TokenCredentials class from azure.identity. Credentials provided here will take precedence over those in the connection string.

Keyword Arguments

  • api_version (str) – The Storage API version to use for requests. Default value is ‘2019-07-07’.

  • secondary_hostname (str) – The hostname of the secondary endpoint.

Returns

A blob checkpoint store.

Return type

BlobCheckpointStore

list_checkpoints(fully_qualified_namespace: str, eventhub_name: str, consumer_group: str, **kwargs: Any) → Iterable[Dict[str, Any]][source]

List the updated checkpoints from the storage blob.

Parameters

  • fully_qualified_namespace (str) – The fully qualified namespace that the Event Hub belongs to. The format is like “<namespace>.servicebus.windows.net”.

  • eventhub_name (str) – The name of the specific Event Hub the checkpoints are associated with, relative to the Event Hubs namespace that contains it.

  • consumer_group (str) – The name of the consumer group the checkpoints are associated with.

Return type

Iterable[Dict[str,Any]], Iterable of dictionaries containing partition checkpoint information:

  • fully_qualified_namespace (str): The fully qualified namespace that the Event Hub belongs to. The format is like “<namespace>.servicebus.windows.net”.

  • eventhub_name (str): The name of the specific Event Hub the checkpoints are associated with, relative to the Event Hubs namespace that contains it.

  • consumer_group (str): The name of the consumer group the checkpoints are associated with.

  • partition_id (str): The partition ID which the checkpoint is created for.

  • sequence_number (int): The sequence number of the EventData.

  • offset (str): The offset of the EventData.

list_ownership(fully_qualified_namespace: str, eventhub_name: str, consumer_group: str, **kwargs: Any) → Iterable[Dict[str, Any]][source]

Retrieves a complete ownership list from the storage blob.

Parameters

  • fully_qualified_namespace (str) – The fully qualified namespace that the Event Hub belongs to. The format is like “<namespace>.servicebus.windows.net”.

  • eventhub_name (str) – The name of the specific Event Hub the partition ownerships are associated with, relative to the Event Hubs namespace that contains it.

  • consumer_group (str) – The name of the consumer group the ownerships are associated with.

Return type

Iterable[Dict[str, Any]], Iterable of dictionaries containing partition ownership information:

  • fully_qualified_namespace (str): The fully qualified namespace that the Event Hub belongs to. The format is like “<namespace>.servicebus.windows.net”.

  • eventhub_name (str): The name of the specific Event Hub the checkpoint is associated with, relative to the Event Hubs namespace that contains it.

  • consumer_group (str): The name of the consumer group the ownership are associated with.

  • partition_id (str): The partition ID which the checkpoint is created for.

  • owner_id (str): A UUID representing the current owner of this partition.

  • last_modified_time (UTC datetime.datetime): The last time this ownership was claimed.

  • etag (str): The Etag value for the last time this ownership was modified. Optional depending on storage implementation.

update_checkpoint(checkpoint: Dict[str, Union[str, int, None]], **kwargs: Any) → None[source]

Updates the checkpoint using the given information for the offset, associated partition and consumer group in the storage blob.

Note: If you plan to implement a custom checkpoint store with the intention of running between cross-language EventHubs SDKs, it is recommended to persist the offset value as an integer.

Parameters

checkpoint (Dict[str,Any]) –

A dict containing checkpoint information:

  • fully_qualified_namespace (str): The fully qualified namespace that the Event Hub belongs to. The format is like “<namespace>.servicebus.windows.net”.

  • eventhub_name (str): The name of the specific Event Hub the checkpoint is associated with, relative to the Event Hubs namespace that contains it.

  • consumer_group (str): The name of the consumer group the checkpoint is associated with.

  • partition_id (str): The partition ID which the checkpoint is created for.

  • sequence_number (int): The sequence number of the EventData the new checkpoint will be associated with.

  • offset (str): The offset of the EventData the new checkpoint will be associated with.

Return type

None