Frequently Asked Questions
What is stream processing?
Stream processing is an alternative method of data processing. In contrast to
batch processing, data is processed in real time and without intermediate
storage. This avoids large accumulations of data, which can be particularly
helpful in the area of big data. When companies depend on processing data as
quickly as possible, batch processing quickly reaches its limits. In the case of
analysis results, for example, which become less meaningful due to the delay. In
addition, the resource requirements for storing data increase as more data is
generated. Stream processing circumvents this problem.
How does stream processing work?
With stream processing, data is forwarded and processed with minimal delay
immediately after it is created. A data stream is generated from a data source,
which consists of a large number of individual data items in a specific format.
It is received by a recipient and processed further. A forwarded data stream can
trigger certain actions, such as updating data analyzes and statistics or
creating new data streams.
How is the data processed in stream processing?
When it comes to processing, a distinction is made between native streaming and
micro batching. With native streaming, a data stream is processed directly,
while with micro batching, smaller units of data streams are collected in order
to then process them further. The advantage of native streaming is further
processing in real time and without intermediate storage.
What open source platforms do you recommend?
Due to our expertise and as a confluent partner, we are happy to recommend our
customers to use the streaming platform Apache Kafka and Flink.