Azure Schema Registry Avro Serializer/Deserializer client library for Java
Azure Schema Registry Avro is a serializer/deserializer library for Avro data format that is integrated with Azure Schema Registry hosted in Azure Event Hubs, providing schema storage, versioning, and management. This package provides an Avro serializer capable of serializing and deserializing payloads containing Schema Registry schema identifiers and Avro-encoded data. This library uses Apache Avro implementation for Avro serialization and deserialization.
Source code | Package (Maven) | API reference documentation | Product Documentation | [Samples][sample_readme]
Getting started
Prerequisites
- Java Development Kit (JDK) with version 8 or above
- Azure Subscription
- An Event Hubs namespace
Include the Package
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-data-schemaregistry-avro</artifactId>
<version>1.0.0-beta.3</version>
</dependency>
Create SchemaRegistryAvroSerializer instance
The SchemaRegistryAvroSerializer instance is the main class that provides APIs for serializing and
deserializing avro data format. The avro schema is stored and retrieved from the Schema Registry service
through the SchemaRegistryClient. So, before we create the serializer, we should create the client.
Create SchemaRegistryClient with Azure Active Directory Credential
In order to interact with the Azure Schema Registry service, you'll need to create an instance of the
SchemaRegistryClient class through the SchemaRegistryClientBuilder. You will need an endpoint and an
API key to instantiate a client object.
You can authenticate with Azure Active Directory using the Azure Identity library. Note that regional endpoints do not support AAD authentication. Create a [custom subdomain][custom_subdomain] for your resource in order to use this type of authentication.
To use the DefaultAzureCredential provider shown below, or other credential providers provided
with the Azure SDK, please include the azure-identity package:
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-identity</artifactId>
<version>1.1.2</version>
</dependency>
You will also need to [register a new AAD application][registeraadapp] and [grant access][aadgrantaccess] to Schema Registry service.
```java TokenCredential tokenCredential = new DefaultAzureCredentialBuilder().build();
SchemaRegistryAsyncClient schemaRegistryAsyncClient = new SchemaRegistryClientBuilder() .endpoint("{schema-registry-endpoint") .credential(tokenCredential) .buildAsyncClient(); ```
Create SchemaRegistryAvroSerializer through the builder
SchemaRegistryAvroSerializer schemaRegistryAvroSerializer = new SchemaRegistryAvroSerializerBuilder()
.schemaRegistryAsyncClient(schemaRegistryAsyncClient)
.schemaGroup("{schema-group}")
.buildSerializer();
Key concepts
ObjectSerializer
This library provides a serializer, SchemaRegistryAvroSerializer, that implements the ObjectSerializer interface.
This allows a developer to use this serializer in any Java Azure SDKs that utilize ObjectSerializer. The
SchemaRegistryAvroSerializer utilitizes a SchemaRegistryClient to construct messages using a wire format containing
schema information such as a schema ID.
This serializer requires the Apache Avro library. The payload types accepted by this serializer include GenericRecord and SpecificRecord.
Wire Format
The serializer in this library creates messages in a wire format. The format is the following:
- Bytes [0-3] – record format indicator – currently is \x00\x00\x00\x00
- Bytes [4-35] – UTF-8 GUID, identifying the schema in a Schema Registry instance
- Bytes [36-end] – serialized payload bytes
Examples
- [Serialize][schema_serialize]
- [Deserialize][schema_deserialize]
Serialize
Serialize a strongly-typed object into Schema Registry-compatible avro payload.
PlayingCard playingCard = new PlayingCard();
playingCard.setPlayingCardSuit(PlayingCardSuit.SPADES);
playingCard.setIsFaceCard(false);
playingCard.setCardValue(5);
// write serialized data to byte array outputstream
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
schemaRegistryAvroSerializer.serialize(outputStream, playingCard);
The avro type PlayinCard is available in samples package
com.azure.data.schemaregistry.avro.generatedtestsources.
Deserialize
Deserialize a Schema Registry-compatible avro payload into a strongly-type object.
SchemaRegistryAvroSerializer schemaRegistryAvroSerializer = createAvroSchemaRegistrySerializer();
InputStream inputStream = getSchemaRegistryAvroData();
PlayingCard playingCard = schemaRegistryAvroSerializer.deserialize(inputStream,
TypeReference.createInstance(PlayingCard.class));
Troubleshooting
Enabling Logging
Azure SDKs for Java offer a consistent logging story to help aid in troubleshooting application errors and expedite their resolution. The logs produced will capture the flow of an application before reaching the terminal state to help locate the root issue. View the [logging][logging] wiki for guidance about enabling logging.
Next steps
More samples can be found here.
Contributing
This project welcomes contributions and suggestions. Most contributions require you to agree to a [Contributor License Agreement (CLA)][cla] declaring that you have the right to, and actually do, grant us the rights to use your contribution.
When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the [Microsoft Open Source Code of Conduct][coc]. For more information see the [Code of Conduct FAQ][cocfaq] or contact [opencode@microsoft.com][coccontact] with any additional questions or comments.
