Projections in event sourcing

Event sourcing is becoming increasingly popular. Eventsourcing is mostly combined with Domain-Driven Design (DDD) and Command-Query-Responsibility-Segregation (CQRS).

Today’s data volumes are constantly growing and decisions have to be made faster and faster. Real-time processing is therefore a top priority in many digital transformation strategies of large companies. This is where projections come into play. Learn exactly what projections are, their benefits, and how to implement them.

‍What is a projection?

To put it simply: With projections, data can be effectively split into contexts. To do this, data is taken from one or more event streams and processed using a projection. The events are passed through the projections (event processors), which then store parts of the event in a “read model” (in most cases this is a database) using the previously aggregated states. Projections help to group the large amount of information generated during event sourcing and make it available for visual reports, for example. To do this, they map part of the persistent data that comes together in the aggregated state of a system and based on the event streams stored in the event store. Depending on the purpose for which a projection is used, it can be modeled differently and thus depict different information and states. For example, all historical and previously collected information can be called up and integrated into the projection.

Representation of a projection in the Event Stream.‍

Where does the term projection come from?

The term comes from the Event Sourcing and CQRS (Command and Query Responsibility Segregation) environment. It is not only used when there is talk of a report. It also designates the transformation process of the information stored in the streams into the desired data models - in this way any event can be transformed into a new data model. In the stream processing environment, projections are also referred to simply as stream processors.

Where are projections used?

Projections can be used in many ways, whether in reporting, in process optimization, in the display of user interfaces (UI) or in the preparation of data required in the backend. A popular example is the reporting of warehouse stocks: The storage quantity, storage time and delivery status can be recorded at a glance using the compact visual representation. Another example is the projection of digital purchasing behavior: All online purchases made so far via an account can be listed chronologically via the shopping cart. In addition to the goods that were finally purchased, products are also displayed that were added to the shopping cart but were not purchased. In this way, they provide the providers of online shops with a deeper insight into purchasing behavior.

Event streaming can be used to map digital purchasing behavior.

‍The advantages of projections

It is inherently clear that thanks to projections, the entire event store/stream does not have to be evaluated with a replay of the events for each query. For large event stores, this would be a tedious task. By means of projections, the data can always be displayed in an up-to-date manner, since they always show the status quo of a system and the desired information due to constant updates. The updates run synchronously or asynchronously. Synchronous means that the projection is also adjusted with a new event, while asynchronous means that the event is first recorded in the event store and only then included in the projection, since not every event has to be stored in the event store.

Simple data query

The event stores are usually not designed to allow simple queries, and if they do, these are often very time-consuming. Projections solve this problem by releasing the data in a structured way for the use case and making data queries from the event stores superfluous.

Separation of reads and writes

Another advantage of projections? The separation of write and read models (CQRS). Projections support the separation of the two object models, since the collection of events and event streams and the associated storage in the stores is completely detached from the subsequent projection. This means that write and read models can be processed with the appropriate technologies. For example, a No-SQL database is suitable for the read models, while another solution is usually better for the event store.

Separation of write and read models and localization of projections.

‍Implement projections and exploit potential

In individual cases, the implementation of projections sometimes depends on certain prerequisites – in particular which goals are being pursued. Therefore, the existing infrastructure or the existing software stack should always be taken into account. The most important thing here is to understand how the data will later be read from the Read Models in which it is stored.

Which solution to which problem?

• If the query to the read model is always static, for example only the “number of items in the shopping cart” is queried, a no-SQL solution is suitable. In this way, user experiences (UI) can be mapped well in the web shop.
• If it is initially unclear which tables/collections and which fields need to be combined, an RDBMS is a more suitable choice.
• For some events, an in-memory approach is a sensible choice.
• A search engine is useful if the use case is, for example, a search of all purchases made in the past.

Conclusion

In summary, it can be said that projections help to use the appropriate read models and databases for each use case. This allows applications to be scaled and the user experience to be raised to the best possible level.

📝 Like what you read?

Then let’s get in touch! Our engineering team will get back to you as soon as possible.