Package com.azure.ai.formrecognizer

package com.azure.ai.formrecognizer

Azure Form Recognizer is a cloud-based service provided by Microsoft Azure that utilizes machine learning to extract information from various types of forms. It is designed to automate the process of form recognition, data extraction, and form understanding. Azure Form Recognizer can handle structured forms, such as invoices, receipts, and surveys, as well as unstructured form data, such as contracts, agreements, and financial reports.

The service uses advanced optical character recognition (OCR) technology to extract text and key-value pairs from custom forms, enabling organizations to automate data entry tasks that would otherwise require manual effort. It can recognize and extract information like dates, addresses, invoice numbers, line items, and other relevant data points from forms.

The Azure Form Recognizer client library allows Java developers to interact with the Azure Form Recognizer service. It provides a set of classes and methods that abstract the underlying RESTful API of Azure Form Recognizer, making it easier to integrate the service into Java applications.

The Azure Form Recognizer client library provides the following capabilities:

  1. Form recognizing: It allows you to submit forms to extract information like text, key-value pairs, tables, and form fields. You can analyze both structured and unstructured documents.
  2. Model Management: It enables you to train custom models by providing labeled training data. You can also list and delete existing models.
  3. Recognize Results: It provides methods to retrieve and interpret analysis results, including extracted text and field values, confidence scores, and form layout information.
  4. Polling and Callbacks: It includes mechanisms for polling the service to check the status of an analysis operation or registering callbacks to receive notifications when the analysis is complete.

Getting Started

The Azure Form Recognizer library provides analysis clients like FormRecognizerAsyncClient and FormRecognizerClient to connect to the Form Recognizer Azure Cognitive Service to analyze information from forms and extract it into structured data. It also provides training clients like FormTrainingClient and FormTrainingAsyncClient to build and manage models from custom forms.

Note:This client only supports FormRecognizerServiceVersion.V2_1 and lower. Recommended to use a newer service version, DocumentAnalysisClient and DocumentModelAdministrationClient.

Refer to the Migration guide to use API versions 2022-08-31 and up.

Service clients are the point of interaction for developers to use Azure Form Recognizer. FormRecognizerClient is the synchronous service client and FormRecognizerAsyncClient is the asynchronous service client. The examples shown in this document use a credential object named DefaultAzureCredential for authentication, which is appropriate for most scenarios, including local development and production environments. Additionally, we recommend using managed identity for authentication in production environments. You can find more information on different ways of authenticating and their corresponding credential types in the Azure Identity documentation".

Sample: Construct a FormRecognizerClient with DefaultAzureCredential

The following code sample demonstrates the creation of a FormRecognizerClient, using the `DefaultAzureCredentialBuilder` to configure it.

 FormRecognizerClient formRecognizerClient = new FormRecognizerClientBuilder()
     .endpoint("{endpoint}")
     .credential(new DefaultAzureCredentialBuilder().build())
     .buildClient();

Further, see the code sample below to use AzureKeyCredential for client creation.

 FormRecognizerClient formRecognizerClient = new FormRecognizerClientBuilder()
     .credential(new AzureKeyCredential("{key}"))
     .endpoint("{endpoint}")
     .buildClient();

Let's take a look at the analysis client scenarios and their respective usage below.


Analyzing forms with prebuilt models

Form Recognizer models and their associated output to help you choose the best model to address your document scenario needs.

You can use domain specific models or train a custom model tailored to your specific business needs and use cases.

Sample: Recognize data from receipts using a url source

The following code sample demonstrates how to detect and extract data from receipts using optical character recognition (OCR).

 String receiptUrl = "https://raw.githubusercontent.com/Azure/azure-sdk-for-java/main/sdk/formrecognizer"
         + "/azure-ai-formrecognizer/src/samples/resources/sample-forms/receipts/contoso-allinone.jpg";
 SyncPoller<FormRecognizerOperationResult, List<RecognizedForm>> syncPoller =
     formRecognizerClient.beginRecognizeReceiptsFromUrl(receiptUrl);
 List<RecognizedForm> receiptPageResults = syncPoller.getFinalResult();

 for (int i = 0; i < receiptPageResults.size(); i++) {
     RecognizedForm recognizedForm = receiptPageResults.get(i);
     Map<String, FormField> recognizedFields = recognizedForm.getFields();
     System.out.printf("----------- Recognizing receipt info for page %d -----------%n", i);
     FormField merchantNameField = recognizedFields.get("MerchantName");
     if (merchantNameField != null) {
         if (FieldValueType.STRING == merchantNameField.getValue().getValueType()) {
             String merchantName = merchantNameField.getValue().asString();
             System.out.printf("Merchant Name: %s, confidence: %.2f%n",
                 merchantName, merchantNameField.getConfidence());
         }
     }

     FormField merchantPhoneNumberField = recognizedFields.get("MerchantPhoneNumber");
     if (merchantPhoneNumberField != null) {
         if (FieldValueType.PHONE_NUMBER == merchantPhoneNumberField.getValue().getValueType()) {
             String merchantAddress = merchantPhoneNumberField.getValue().asPhoneNumber();
             System.out.printf("Merchant Phone number: %s, confidence: %.2f%n",
                 merchantAddress, merchantPhoneNumberField.getConfidence());
         }
     }

     FormField transactionDateField = recognizedFields.get("TransactionDate");
     if (transactionDateField != null) {
         if (FieldValueType.DATE == transactionDateField.getValue().getValueType()) {
             LocalDate transactionDate = transactionDateField.getValue().asDate();
             System.out.printf("Transaction Date: %s, confidence: %.2f%n",
                 transactionDate, transactionDateField.getConfidence());
         }
     }

     FormField receiptItemsField = recognizedFields.get("Items");
     if (receiptItemsField != null) {
         System.out.printf("Receipt Items: %n");
         if (FieldValueType.LIST == receiptItemsField.getValue().getValueType()) {
             List<FormField> receiptItems = receiptItemsField.getValue().asList();
             receiptItems.stream()
                 .filter(receiptItem -> FieldValueType.MAP == receiptItem.getValue().getValueType())
                 .map(formField -> formField.getValue().asMap())
                 .forEach(formFieldMap -> formFieldMap.forEach((key, formField) -> {
                     if ("Quantity".equals(key)) {
                         if (FieldValueType.FLOAT == formField.getValue().getValueType()) {
                             Float quantity = formField.getValue().asFloat();
                             System.out.printf("Quantity: %f, confidence: %.2f%n",
                                 quantity, formField.getConfidence());
                         }
                     }
                 }));
         }
     }
 }

You can also extract data from a local receipt with prebuilt models using the beginRecognizeReceipts method.

For more information on which supported model you should use refer to models usage documentation.


Analyze a custom form with a model trained with or without labels.

Analyze a custom form with a model trained with or without labels. Custom models are trained with your own data, so they're tailored to your documents.

For more information, see train a model with labels.

Sample: Analyze a custom form with a model trained with labels

This sample demonstrates how to recognize form fields and other content from your custom forms, using models you trained with your own form types.

 String trainingFilesUrl = "{SAS_URL_of_your_container_in_blob_storage}";
 boolean useTrainingLabels = true;

 SyncPoller<FormRecognizerOperationResult, CustomFormModel> trainingPoller =
     formTrainingClient.beginTraining(trainingFilesUrl,
         useTrainingLabels,
         new TrainingOptions()
             .setModelName("my model trained with labels"),
         Context.NONE);

 CustomFormModel customFormModel = trainingPoller.getFinalResult();

 // Model Info
 System.out.printf("Model Id: %s%n", customFormModel.getModelId());

 String customFormUrl = "customFormUrl";
 String modelId = customFormModel.getModelId();
 SyncPoller<FormRecognizerOperationResult, List<RecognizedForm>> recognizeFormPoller =
     formRecognizerClient.beginRecognizeCustomFormsFromUrl(modelId, customFormUrl);

 List<RecognizedForm> recognizedForms = recognizeFormPoller.getFinalResult();

 for (int i = 0; i < recognizedForms.size(); i++) {
     RecognizedForm form = recognizedForms.get(i);
     System.out.printf("----------- Recognized custom form info for page %d -----------%n", i);
     System.out.printf("Form type: %s%n", form.getFormType());
     System.out.printf("Form type confidence: %.2f%n", form.getFormTypeConfidence());
     form.getFields().forEach((label, formField) ->
         System.out.printf("Field %s has value %s with confidence score of %f.%n", label,
             formField.getValueData().getText(),
             formField.getConfidence())
     );
 }

For a suggested approach to extracting information from custom forms with known fields, see strongly-typing a recognized form.

See Also: