Registering Apache Avro schemas with the Confluent Schema Registry

This topic describes the process of Apache Avro schema registration with the Confluent Schema Registry.

Data Bridge generates an Avro schema for each view that is configured to publish steaming updates using Apache Avro format. In order for Data Bridge to be able to publish messages to a Kafka topic using an Avro schema, it is first necessary to register it with the Schema Registry. The schema registration can be done automatically or manually, depending on your environment settings.

Schema subject naming convention

In both automatic and manual schema registration options, Data Bridge uses the following naming pattern for the Avro schema subject name:

<TopicName-NameSpace.SchemaName>

Where:

• TopicName is the name of the Kafka topic specified in the view configuration
  
• NameSpace is the Kafka namespace specified in the view configuration
  
• SchemaName is the Kafka schema name specified in the view configuration
  

Values for these variables are configured for each view in the Kafka Configuration section in the Publishing Options tab in the Data Bridge UI. For more information about which characters are supported for these variables, see Creating views.

Example:

contacts-nexj.crm.crmcontacts

Automatic schema registration

When Data Bridge attempts to publish messages serialized in Apache Avro format to a Kafka topic, it first attempts to retrieve a schema from the Schema Registry for the subject name, following the naming convention described above. If the retrieval is not successful, a new Avro schema is registered with the same subject name. Data Bridge then proceeds to serialize and publish messages using the registered schema.

Registering a schema manually

Your environment settings may potentially prevent automatic schema registration from occurring successfully. For example, this could happen if a Data Bridge view is updated and schema compatibility settings selected in the Schema Registry do not allow it. In such cases, an Avro schema can be registered manually.

To register an Avro schema generated by Data Bridge: 

1. Open or edit the required view in the View Explorer in the Data Bridge UI.
2. Select the Publishing Options tab, and ensure that Avro is selected in the Kafka Configuration section.
3. Save the view, select the Preview tab, select the Message subtab, and click Export Avro Schema to download the schema from the UI. 
   The following screenshot shows an example message and Export Avro Schema:
   

   

   
   
4. Using the Schema Registry API, register it to the topic with the subject name following the naming pattern described in Schema subject naming convention. For more information, see https://docs.confluent.io/platform/current/schema-registry/develop/api.html.