Part 3: Configuring Clients Earlier, we introduced Kafka Serializers and Deserializers that are capable of writing and reading Kafka records in Avro format. In this part we will going to see how to configure producers and consumers to use them. Setting up a Kafka Topic for use as a Schema Store KafkaTopicSchemaProvider works with a […]
Implementing a Schema Store In Part 1, we saw the need for an Apache Avro schema provider but did not implement one. In this part we will implement a schema provider that works with Apache Kafka as storage. In-Memory SchemaStore First we can implement an in-memory store for schemas. This is useful to understand the […]
Our thanks to Don Drake (@dondrake), an independent technology consultant who is currently working at Allstate Insurance, for the guest post below about his experiences comparing use of the Apache Avro and Apache Parquet file formats with Apache Spark. Over the last few months, numerous hallway conversations, informal discussions, and meetings have occurred at Allstate […]
