Azure Cognitive Search client library for Python ================================================ `Azure Cognitive Search `_ is a search-as-a-service cloud solution that gives developers APIs and tools for adding a rich search experience over private, heterogeneous content in web, mobile, and enterprise applications. The Azure Cognitive Search service is well suited for the following application scenarios: * Consolidate varied content types into a single searchable index. To populate an index, you can push JSON documents that contain your content, or if your data is already in Azure, create an indexer to pull in data automatically. * Attach skillsets to an indexer to create searchable content from images and large text documents. A skillset leverages AI from Cognitive Services for built-in OCR, entity recognition, key phrase extraction, language detection, text translation, and sentiment analysis. You can also add custom skills to integrate external processing of your content during data ingestion. * In a search client application, implement query logic and user experiences similar to commercial web search engines. Use the Azure.Search.Documents client library to: * Submit queries for simple and advanced query forms that include fuzzy search, wildcard search, regular expressions. * Implement filtered queries for faceted navigation, geospatial search, or to narrow results based on filter criteria. * Create and manage search indexes. * Upload and update documents in the search index. * Create and manage indexers that pull data from Azure into an index. * Create and manage skillsets that add AI enrichment to data ingestion. * Create and manage analyzers for advanced text analysis or multi-lingual content. * Optimize results through scoring profiles to factor in business logic or freshness. `Source code `_ | `Package (PyPI) `_ | `API reference documentation `_ | `Product documentation `_ | `Samples `_ *Disclaimer* ---------------- *Azure SDK Python packages support for Python 2.7 has ended 01 January 2022. For more information and questions, please refer to https://github.com/Azure/azure-sdk-for-python/issues/20691* Getting started --------------- Install the package ^^^^^^^^^^^^^^^^^^^ Install the Azure Cognitive Search client library for Python with `pip `_\ : .. code-block:: bash pip install azure-search-documents Prerequisites ^^^^^^^^^^^^^ * Python 3.7 or later is required to use this package. * You need an `Azure subscription `_ and a `Azure Cognitive Search service `_ to use this package. To create a new search service, you can use the `Azure portal `_\ , `Azure PowerShell `_\ , or the `Azure CLI `_. .. code-block:: Powershell az search service create --name --resource-group --sku free --location westus See `choosing a pricing tier `_ for more information about available options. Authenticate the client ^^^^^^^^^^^^^^^^^^^^^^^ All requests to a search service need an api-key that was generated specifically for your service. `The api-key is the sole mechanism for authenticating access to your search service endpoint. `_ You can obtain your api-key from the `Azure portal `_ or via the Azure CLI: .. code-block:: Powershell az search admin-key show --service-name --resource-group There are two types of keys used to access your search service: **admin** *(read-write)* and **query** *(read-only)* keys. Restricting access and operations in client apps is essential to safeguarding the search assets on your service. Always use a query key rather than an admin key for any query originating from a client app. *Note: The example Azure CLI snippet above retrieves an admin key so it's easier to get started exploring APIs, but it should be managed carefully.* We can use the api-key to create a new ``SearchClient``. .. code-block:: python import os from azure.core.credentials import AzureKeyCredential from azure.search.documents import SearchClient index_name = "nycjobs" # Get the service endpoint and API key from the environment endpoint = os.environ["SEARCH_ENDPOINT"] key = os.environ["SEARCH_API_KEY"] # Create a client credential = AzureKeyCredential(key) client = SearchClient(endpoint=endpoint, index_name=index_name, credential=credential) Key concepts ------------ An Azure Cognitive Search service contains one or more indexes that provide persistent storage of searchable data in the form of JSON documents. *(If you're brand new to search, you can make a very rough analogy between indexes and database tables.)* The Azure.Search.Documents client library exposes operations on these resources through two main client types. * ``SearchClient`` helps with: * `Searching `_ your indexed documents using `rich queries `_ and `powerful data shaping `_ * `Autocompleting `_ partially typed search terms based on documents in the index * `Suggesting `_ the most likely matching text in documents as a user types * `Adding, Updating or Deleting Documents `_ documents from an index * ``SearchIndexClient`` allows you to: * `Create, delete, update, or configure a search index `_ * `Declare custom synonym maps to expand or rewrite queries `_ * Most of the ``SearchServiceClient`` functionality is not yet available in our current preview * ``SearchIndexerClient`` allows you to: * `Start indexers to automatically crawl data sources `_ * `Define AI powered Skillsets to transform and enrich your data `_ *The ``Azure.Search.Documents`` client library (v1) is a brand new offering for Python developers who want to use search technology in their applications. There is an older, fully featured ``Microsoft.Azure.Search`` client library (v10) with many similar looking APIs, so please be careful to avoid confusion when exploring online resources.* Examples -------- The following examples all use a simple `Hotel data set `_ that you can `import into your own index from the Azure portal. `_ These are just a few of the basics - please `check out our Samples `_ for much more. * `Querying <#querying>`_ * `Creating an index <#creating-an-index>`_ * `Adding documents to your index <#adding-documents-to-your-index>`_ * `Retrieving a specific document from your index <#retrieving-a-specific-document-from-your-index>`_ * `Async APIs <#async-apis>`_ Querying ^^^^^^^^ Let's start by importing our namespaces. .. code-block:: python import os from azure.core.credentials import AzureKeyCredential from azure.search.documents import SearchClient We'll then create a ``SearchClient`` to access our hotels search index. .. code-block:: python index_name = "hotels" # Get the service endpoint and API key from the environment endpoint = os.environ["SEARCH_ENDPOINT"] key = os.environ["SEARCH_API_KEY"] # Create a client credential = AzureKeyCredential(key) client = SearchClient(endpoint=endpoint, index_name=index_name, credential=credential) Let's search for a "luxury" hotel. .. code-block:: python results = client.search(search_text="luxury") for result in results: print("{}: {})".format(result["hotelId"], result["hotelName"])) Creating an index ^^^^^^^^^^^^^^^^^ You can use the ``SearchIndexClient`` to create a search index. Fields can be defined using convenient ``SimpleField``\ , ``SearchableField``\ , or ``ComplexField`` models. Indexes can also define suggesters, lexical analyzers, and more. .. code-block:: python import os from azure.core.credentials import AzureKeyCredential from azure.search.documents.indexes import SearchIndexClient from azure.search.documents.indexes.models import ( ComplexField, CorsOptions, SearchIndex, ScoringProfile, SearchFieldDataType, SimpleField, SearchableField ) endpoint = os.environ["SEARCH_ENDPOINT"] key = os.environ["SEARCH_API_KEY"] # Create a service client client = SearchIndexClient(endpoint, AzureKeyCredential(key)) # Create the index name = "hotels" fields = [ SimpleField(name="hotelId", type=SearchFieldDataType.String, key=True), SimpleField(name="baseRate", type=SearchFieldDataType.Double), SearchableField(name="description", type=SearchFieldDataType.String), ComplexField(name="address", fields=[ SimpleField(name="streetAddress", type=SearchFieldDataType.String), SimpleField(name="city", type=SearchFieldDataType.String), ]) ] cors_options = CorsOptions(allowed_origins=["*"], max_age_in_seconds=60) scoring_profiles = [] index = SearchIndex( name=name, fields=fields, scoring_profiles=scoring_profiles, cors_options=cors_options) result = client.create_index(index) Adding documents to your index ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ You can ``Upload``\ , ``Merge``\ , ``MergeOrUpload``\ , and ``Delete`` multiple documents from an index in a single batched request. There are `a few special rules for merging `_ to be aware of. .. code-block:: python import os from azure.core.credentials import AzureKeyCredential from azure.search.documents import SearchClient index_name = "hotels" endpoint = os.environ["SEARCH_ENDPOINT"] key = os.environ["SEARCH_API_KEY"] DOCUMENT = { 'Category': 'Hotel', 'hotelId': '1000', 'rating': 4.0, 'rooms': [], 'hotelName': 'Azure Inn', } search_client = SearchClient(endpoint, index_name, AzureKeyCredential(key)) result = search_client.upload_documents(documents=[DOCUMENT]) print("Upload of new document succeeded: {}".format(result[0].succeeded)) Authenticate in a National Cloud ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ To authenticate in a `National Cloud `_\ , you will need to make the following additions to your client configuration: * Set the ``AuthorityHost`` in the credential options or via the ``AZURE_AUTHORITY_HOST`` environment variable * Set the ``audience`` in ``SearchClient``\ , ``SearchIndexClient``\ , or ``SearchIndexerClient`` .. code-block:: python # Create a SearchClient that will authenticate through AAD in the China national cloud. import os from azure.identity import DefaultAzureCredential, AzureAuthorityHosts from azure.search.documents import SearchClient index_name = "hotels" endpoint = os.environ["SEARCH_ENDPOINT"] key = os.environ["SEARCH_API_KEY"] credential = DefaultAzureCredential(authority=AzureAuthorityHosts.AZURE_CHINA) search_client = SearchClient(endpoint, index_name, crdential=credential, audience="https://search.azure.cn") Retrieving a specific document from your index ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ In addition to querying for documents using keywords and optional filters, you can retrieve a specific document from your index if you already know the key. You could get the key from a query, for example, and want to show more information about it or navigate your customer to that document. .. code-block:: python import os from azure.core.credentials import AzureKeyCredential from azure.search.documents import SearchClient index_name = "hotels" endpoint = os.environ["SEARCH_ENDPOINT"] key = os.environ["SEARCH_API_KEY"] client = SearchClient(endpoint, index_name, AzureKeyCredential(key)) result = client.get_document(key="1") print("Details for hotel '1' are:") print(" Name: {}".format(result["HotelName"])) print(" Rating: {}".format(result["Rating"])) print(" Category: {}".format(result["Category"])) Async APIs ^^^^^^^^^^ This library includes a complete async API. To use it, you must first install an async transport, such as `aiohttp `_. See `azure-core documentation `_ for more information. .. code-block:: py from azure.core.credentials import AzureKeyCredential from azure.search.documents.aio import SearchClient client = SearchClient(endpoint, index_name, AzureKeyCredential(api_key)) async with client: results = await client.search(search_text="hotel") async for result in results: print("{}: {})".format(result["hotelId"], result["hotelName"])) ... Troubleshooting --------------- General ^^^^^^^ The Azure Cognitive Search client will raise exceptions defined in `Azure Core `_. Logging ^^^^^^^ This library uses the standard `logging `_ library for logging. Basic information about HTTP sessions (URLs, headers, etc.) is logged at INFO level. Detailed DEBUG level logging, including request/response bodies and unredacted headers, can be enabled on a client with the ``logging_enable`` keyword argument: .. code-block:: python import sys import logging from azure.core.credentials import AzureKeyCredential from azure.search.documents import SearchClient # Create a logger for the 'azure' SDK logger = logging.getLogger('azure') logger.setLevel(logging.DEBUG) # Configure a console output handler = logging.StreamHandler(stream=sys.stdout) logger.addHandler(handler) # This client will log detailed information about its HTTP sessions, at DEBUG level client = SearchClient("", "", AzureKeyCredential(""), logging_enable=True) Similarly, ``logging_enable`` can enable detailed logging for a single operation, even when it isn't enabled for the client: .. code-block:: python result = client.search(search_text="spa", logging_enable=True) Next steps ---------- * Go further with Azure.Search.Documents and our `https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/search/azure-search-documents/samples `_ * Watch a `demo or deep dive video `_ * Read more about the `Azure Cognitive Search service `_ Contributing ------------ See our `Search CONTRIBUTING.md `_ for details on building, testing, and contributing to this library. This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit `cla.microsoft.com `_. This project has adopted the `Microsoft Open Source Code of Conduct `_. For more information see the `Code of Conduct FAQ `_ or contact `opencode@microsoft.com `_ with any additional questions or comments. .. image:: https://azure-sdk-impressions.azurewebsites.net/api/impressions/azure-sdk-for-net%2Fsdk%2Fsearch%2FAzure.Search.Documents%2FREADME.png :target: https://azure-sdk-impressions.azurewebsites.net/api/impressions/azure-sdk-for-net%2Fsdk%2Fsearch%2FAzure.Search.Documents%2FREADME.png :alt: Impressions Related projects ---------------- * `Microsoft Azure SDK for Python `_ .. raw:: html .. image:: https://azure-sdk-impressions.azurewebsites.net/api/impressions/azure-sdk-for-python%2Fsdk%2Fsearch%2Fazure-search-documents%2FREADME.png :target: https://azure-sdk-impressions.azurewebsites.net/api/impressions/azure-sdk-for-python%2Fsdk%2Fsearch%2Fazure-search-documents%2FREADME.png :alt: Impressions Indices and tables ------------------ * :ref:`genindex` * :ref:`modindex` * :ref:`search` .. toctree:: :maxdepth: 5 :glob: :caption: Developer Documentation azure.search.documents.rst