# Overview ## Use Cases for Stream Processing #flashcard - Fraud detection systems need to determine if the usage patterns of a credit card have unexpectedly changed, and block the card if it is likely to have been stolen. - Trading systems need to examine price changes in a financial market and execute trades according to specified rules. - Manufacturing systems need to monitor the status of machines in a factory, and quickly identify the problem if there is a malfunction. - Military and intelligence systems need to track the activities of a potential aggressor, and raise the alarm if there are signs of an attack. <!--ID: 1751507776990--> # Key Considerations ## Types of Stream Processing - [[Complex Event Processing (CEP)]] - an approach for analyzing event streams geared toward identifying specific event patterns - [[Stream Analytics]] ## Types of Stream Joins [[Stream Data Joins]] - **Stream-stream joins** - Both input streams consist of activity events, and the join operator searches for related events that occur within some window of time. For example, it may match two actions taken by the same user within 30 minutes of each other. The two join inputs may in fact be the same stream (a self-join) if you want to find related events within that one stream. - **Stream-table joins** - One input stream consists of activity events, while the other is a database changelog. The changelog keeps a local copy of the database up to date. For each activity event, the join operator queries the database and outputs an enriched activity event. - **Table-table joins** - Both input streams are database changelogs. In this case, every change on one side is joined with the latest state of the other side. The result is a stream of changes to the materialized view of the join between the two tables # Implementation Details ## [[Fault Tolerance]] Approaches - [[Microbatching]] - [[Checkpointing]] - [[Database Transactions]] - [[idempotency]] # Useful Links ## Stream Processing Products - [[Apache Flink]] - [[Spark Streaming]] - [[Tez]] - [[Storm]] - [[Upsolver]] - [[Materialize]] # Related Topics ## Reference #### Working Notes #### Sources