Complex Event Processing
------------------------
It is essential to have a clear idea about ' Event' before proceeding to Complex Event Processing.
What is an Event ?
"An event may be observed as a change of state with any physical or logical or otherwise discriminated condition of and in a technical or economical system, each state information with an attached time stamp defining the order of occurrence and a topology mark defining the location of occurrence"
What is Complex Event Processing ?
"Complex Event Processing, or CEP, is basically an event processing concept that can be used in identifying the most meaningful events within the event cloud, analyzing their impact, and taking subsequent action in real time. Complex event processing refers to process states, the changes of state exceeding a defined threshold of level, time, or value increment or just of a count as the event. It requires the respective event monitoring, event reporting, event recording and event filtering. "
According to the Complex Event Processing (http://complexevents.com/?page_id=3) website, by Research Professor David Luckham:
“Complex Event Processing (CEP) is an emerging technology for building and managing information systems including:
* Business Activity Monitoring
* Business Process Management
* Enterprise Application Integration
* Event-Driven Architectures
* Network and business level Security
* Real time conformance to regulations and policies.”
What is expected from a CEP Engine ?
There can be found three most common things expected from a CEP engine by most of above applications.
1. High throughput - process large volumes of messages (between 1,000 to 100k messages per second)
2. Low latency - React in real-time to conditions that occur (from a few milliseconds to a few seconds)
3. Complex computations - Detect patterns among events (event correlation), filter events, aggregate time or length windows of events, join event streams, trigger based on absence of events etc.
What are available complex event processing engines ?
It can be found many complex event processing engines and most of them are developed to address the requirements of above mentioned applications which analyze and reacts to events.
Some of them are :
Drools Fusion Complex Event Processing Engine (http://www.jboss.org/drools/drools-fusion.html)
Esper Fusion Complex Event Processing Engine (http://esper.codehaus.org/)
Yahoo Complex Event Processing Engine (http://www.infoq.com/news/2010/11/yahoo-releases-s4)
Example:
Lets consider the weather department of a particular area. It has so many sensors which are generating event streams. But as all of us know that each and every
event generated by these sensors are not useful. Therefor it is needed to filter out useful events from these event streams.
Lets say the weather department has three sensor types to detect
* Speed of the wind
* Humidity
* Light intensity
When it is processing the events generated with these sensors, There may be a time :
* Speed of wind value hits a threshold value of raining condition
* Humidity value hits a threshold value of raining condition
* Light intensity value hits a threshold value of raining condition
When the above three conditions occurs simultaneously , It can be a new event generated on 'Rain within next hour'.
