In the world of big data, efficient streaming across platforms is critical. That is why Winton has developed a new component to stream data over Apache Kafka from Python - work that we are making available to the open source community, as we announced on July 12 at the EuroPython 2017 conference in Rimini, Italy.
Winton’s Andreas Heider and Robert Wall, who implemented the API with Steffen Knollmann, gave a presentation at EuroPython. The slides are on the EuroPython site and the talk, entitled ‘Taking the Hipster Out of Streaming’, can be watched below:
The ‘winton-kafka-streams code, which is still at an early stage of development, is available at Winton’s GitHub page.
Apache Kafka is the pre-eminent solution for streaming data, and our Apache Kafka’s Streams API adds an easy-to-use layer to the base Kafka model. This enables the development of elastic, highly-scalable and fault-tolerant streaming applications, without the complexity of a separate processing cluster.
This Streams API had only previously been available to developers using Java. But Winton’s application for Python has now opened up the power of Apache Kafka’s Streams to Python developers and users who want to avoid introducing Java into their technical stack, or for whom Java implementation is unnecessary.
While the release of our Kafka Streams API is our most significant open source contribution to date, Winton is increasingly collaborating with the community, both by engaging with ongoing projects and by sharing our knowledge and expertise via our Tech Blog. We are involved in several other exciting initatives currently. To find out more, watch this space.
This is also Winton’s second year sponsoring EuroPython, which we view as a natural fit since Python is the premier language for a core part of our business: the analysis of large datasets. As Johan Ditmar, Winton’s Head of Software Development, puts it: “Winton were delighted to sponsor EuroPython for the second year running. We are committed users of Python across all divisions of the company and look forward to getting increasingly involved with the community.”
Last year’s talk in Bilbao was also written up on our tech blog.