Complex Event Processing (CEP)

Complex Event Processing
------------------------
It is essential to have a clear idea about ' Event' before proceeding to Complex Event Processing.

What is an Event ?

"An event may be observed as a change of state with any physical or logical or otherwise discriminated condition of and in a technical or economical system, each state information with an attached time stamp defining the order of occurrence and a topology mark defining the location of occurrence"

What is Complex Event Processing ?

"Complex Event Processing, or CEP, is basically an event processing concept that can be used in identifying the most meaningful events within the event cloud, analyzing their impact, and taking subsequent action in real time. Complex event processing refers to process states, the changes of state exceeding a defined threshold of level, time, or value increment or just of a count as the event. It requires the respective event monitoring, event reporting, event recording and event filtering. "

According to the Complex Event Processing (http://complexevents.com/?page_id=3) website, by Research Professor David Luckham:

“Complex Event Processing (CEP) is an emerging technology for building and managing information systems including:

* Business Activity Monitoring
* Business Process Management
* Enterprise Application Integration
* Event-Driven Architectures
* Network and business level Security
* Real time conformance to regulations and policies.”

What is expected from a CEP Engine ?

There can be found three most common things expected from a CEP engine by most of above applications.

1. High throughput - process large volumes of messages (between 1,000 to 100k messages per second)


2. Low latency - React in real-time to conditions that occur (from a few milliseconds to a few seconds)


3. Complex computations - Detect patterns among events (event correlation), filter events, aggregate time or length windows of events, join event streams, trigger based on absence of events etc.



What are available complex event processing engines ?

It can be found many complex event processing engines and most of them are developed to address the requirements of above mentioned applications which analyze and reacts to events.
Some of them are :

Drools Fusion Complex Event Processing Engine (http://www.jboss.org/drools/drools-fusion.html)
Esper Fusion Complex Event Processing Engine (http://esper.codehaus.org/)
Yahoo Complex Event Processing Engine (http://www.infoq.com/news/2010/11/yahoo-releases-s4)



Example:
Lets consider the weather department of a particular area. It has so many sensors which are generating event streams. But as all of us know that each and every
event generated by these sensors are not useful. Therefor it is needed to filter out useful events from these event streams.

Lets say the weather department has three sensor types to detect
* Speed of the wind
* Humidity
* Light intensity

When it is processing the events generated with these sensors, There may be a time :

* Speed of wind value hits a threshold value of raining condition
* Humidity value hits a threshold value of raining condition
* Light intensity value hits a threshold value of raining condition

When the above three conditions occurs simultaneously , It can be a new event generated on 'Rain within next hour'.

Business Rules

What are business rules? Where we use them ?

Business Rule is a some kind of a rule that relates to a particular business. From time to time this rules may change. We know there is a logic behind each and every decision we get. Normaly we are getting some decision by considering some facts. Same thing apply to business. In business we get some decision , after considering some kind of facts. These facts may change from time to time according to the nature of the business.

Lets take a simple example : A Mobile Service Provider

With the experience we know , Mobile Service Providers most interested party is their customers. So they do many kind of promotions to attract customers to them. So from time to time they change their call rates and freely available features , etc. Here What they are doing is changing the logic from time to time according to their requirement. But they do not change the way they operate.

Basically we can separate this in to two parts;
1. What is to be done (Logic)
2. How is to be done (Implementation)

So in our example, from time to time , Business development team of the Mobile Service provide change the logic. We know Mobile Service Providers are running there business with some kind of a software. So software is developed by developers when the logic is clearly identified. Lets take a simple example:

if the season is new year festival season , all the call rates must be reduced by 10%

So this is the logic that software should perform actions behind. This logic may change from time to time. So if we are going to implement software using if else statements, We have to implement it each and every time logic changes. In Software world, This is a mess. So we need to have a solution to this problem, which we can change the logic with out changing the software.

This is the point Business Rules comes in to play.

With the use of business rules we can change the logic, with out affecting to the implemented software. There are many kind of rule engines. JBOSS Drools is the most common language we can use to write these business rules.

Since business rules are decide by business management people, business rules should be easy to understand by a person who have very little knowledge in java, c++ or what ever. Drools provide this simplicity.

you can get clear idea on drools by refering to the JBOSS Drools web site

Java Messaging Service (JMS)

What is Java Message Service ?

Before jumping in to the JMS directly , it is worth to get a back ground knowledge on where we use JMS. Lets get a bit understanding on messaging system.

We know people use email to communicate with each other. When we send an email to an address of another person , we are pretty sure that , the message will be delivered to that person , since all the stuff handled by email system. So email is commonly used human to human messaging system.

Here we basically consider about how different software applications can communicate with each other using messaging systems. In many business systems, it can be found application - to - application messaging systems and they are generally identified as Enterprise Messaging Systems or Message Oriented Middleware. (MOM)

Message-Oriented Middleware, can be used to transmit messages from one application to another across a network. These Enterprise Messaging Systems ensure that messages are properly distributed among applications. Apart from that Enterprise Messaging Systems provide ;

• Fault tolerance
• Load balancing
• Scalability
• Transactional support

How to create a patch file and how to apply that to the source code

Here i m going to tell you how we can easily create a patch file and how we can apply that patch file in to our source code;

Creating the patch file:

To create a patch file you need to have SVN access. When you want to create a patch file
move to the location that you have a difference with svn . Then use the bellow command to create the patch file.

svn diff > patchFileToBeCreated.txt

this will create the patch file called "patchFileToBeCreated.txt" in the same location.

Then when any other person wants to apply your patch to his source , he should have the same SVN revision that you created your patch file.

to apply the created patch file , you have to go to the same folder path that you created your patch file
Then use the bellow command to apply the patch to your source

patch -p0 < patchFileToBeCreated.txt

Mostly used svn commands

Adding Files to SVN :
svn add theFileNeedToBeAdded

Check in added Files or modified files :
svn ci theFileModifiedOrAdded -m 'Comment about the modification done'

Check out the files from svn :
svn co -uri http://location.of.checking out files

Check the difference with svn
svn diff fileToBeCheckedWithTheServer


Check the status of files with svn
svn status

Get latest version from SVN
svn up

Adding patched files to code :
patch -p0 < theFileToBePatched

How to do find and delete in command prompt

Here is the terminal command to do find and delete

Eg: to find all files with extension ".svn" we can use

find -iname '*.svn'

to delete the founded file we can use :

find -iname '*.svn' | xargs rm -rf