Friday, October 14, 2016

Configure WSO2 products for Continuous JFR Recordings



Production systems are daily exposing to huge traffic and sometimes these production systems are not serving requests as expected. In those kind of scenarios, We need a way to figure out what went wrong in the past few hours.

With Java Flight Recorder, If we have already enabled it, It is very easy to figure out what went wrong with information on

  • Memory Usage
  • CPU Usage 
  • Thread Usage 
  • Etc.

In order to configure a WSO2 server to continuously record this information, You can change the startup script of wso2 server as follows.

For that, you need to open wso2server.sh file which is located in the bin directory of the WSO2 server and add following lines to it. 

 -XX:+UnlockCommercialFeatures \  
 -XX:+FlightRecorder \  
 -XX:FlightRecorderOptions=defaultrecording=true,disk=true,maxage=60m,repository=./tmp,dumponexit=true,dumponexitpath=./ \  

Once you add them, It will look like follows.
  -Dfile.encoding=UTF8 \   
  -XX:+UnlockCommercialFeatures \   
  -XX:+FlightRecorder \   
  -XX:FlightRecorderOptions=defaultrecording=true,disk=true,maxage=60m,repository=./tmp,dumponexit=true,dumponexitpath=./ \   
 org.wso2.carbon.bootstrap.Bootstrap $  

Then you can save it and restart the server. It will automatically, Dump records to the tmp directory in  your WSO2 server. 

Acquire Heap Dump of Java Process

When it comes to production troubleshooting with Java Servers, It is a main fact that we need to analyze the memory consumption.  Normally if we have configured following properties , It will automatically create the heap dump when there is an OutOfMemory exception.

 -XX:+HeapDumpOnOutOfMemoryError  
 -XX:HeapDumpPath=<path>  

There are some OOM exceptions which is not creating heap dump since it is not related to memory. You can understand on various OutOfMemory Exceptions in the article [1].


But, If you need to get a heap dump at a time which server is consuming memory but not yet throwing OOM Exception, You can use following command to do that.

Assuming that you have set JAVA_HOME and PID is the Process ID.

In a Linux based system.
 jmap -dump:format=b,file=./heap.hprof <PID>  

In a windows based system

 <JAVA_HOME>/bin/jmap -dump:format=b,file=c:\temp\heap.hprof <PID>  



Once you obtain the heap dump, You can analyze it using tools like MAT, VisualVM.



[1] https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/memleaks002.html

Friday, April 22, 2016

Applying security for ESB proxy services...


Security is a major factor we consider when it comes to each and every deployment. WSO2 Enterprise Service Bus also capable of securing services.

WSO2 ESB 4.8 or previous versions were having the capability of applying the security for a proxy service from Admin Console as in [1]

However, From ESB 4.9.0 , we can no longer apply security for a proxy service from Admin Console of the ESB. We need to use WSO2 Developer Studio version 3.8 for this requirement for ESB 4.9.0.


You can find the documentation on  applying security to ESB 4.9.0 based proxy service here[2].  However, i would like to add a small modification to the doc in [2] at the end.

After securing the proxy according to the document, We need to create the Composite Application Project and export the CAR file. When exporting the CAR file, by default the server role of the Registry project is being selected as GovernanceRegistry as in the bellow image.




When we deploy that CAR file in ESB, We are getting following exception [3] due to above Server Role.

In order to fix the problem, we need to change the server role to ESB as bellow since we are going to deploy it in ESB.






[1] https://docs.wso2.com/display/ESB481/Securing+Proxy+Services
[2] https://docs.wso2.com/display/ESB490/Applying+Security+to+a+Proxy+Service
[3]

 [2016-04-12 14:34:48,658] INFO - ApplicationManager Deploying Carbon Application : MySecondCarProject1_1.0.1.car...  
 [2016-04-12 14:34:48,669] INFO - EndpointDeployer Endpoint named 'SimpleStockQuote' has been deployed from file : /Users/shammi/wso2/Support-Issues/MOTOROLAMOBPROD-44/wso2esb-4.9.0/tmp/carbonapps/-1234/1460496888659MySecondCarProject1_1.0.1.car/SimpleStockQuote_1.0.0/SimpleStockQuote-1.0.0.xml  
 [2016-04-12 14:34:48,670] INFO - ProxyService Building Axis service for Proxy service : myTestProxy  
 [2016-04-12 14:34:48,671] WARN - SynapseConfigUtils Cannot convert null to a StreamSource  
 [2016-04-12 14:34:48,671] ERROR - ProxyServiceDeployer ProxyService Deployment from the file : /Users/shammi/wso2/Support-Issues/MOTOROLAMOBPROD-44/wso2esb-4.9.0/tmp/carbonapps/-1234/1460496888659MySecondCarProject1_1.0.1.car/myTestProxy_1.0.0/myTestProxy-1.0.0.xml : Failed.  
 org.apache.synapse.SynapseException: Cannot convert null to a StreamSource  
      at org.apache.synapse.config.SynapseConfigUtils.handleException(SynapseConfigUtils.java:578)  
      at org.apache.synapse.config.SynapseConfigUtils.getStreamSource(SynapseConfigUtils.java:79)  
      at org.apache.synapse.core.axis2.ProxyService.getPolicyFromKey(ProxyService.java:822)  
      at org.apache.synapse.core.axis2.ProxyService.buildAxisService(ProxyService.java:608)  
      at org.apache.synapse.deployers.ProxyServiceDeployer.deploySynapseArtifact(ProxyServiceDeployer.java:80)  
      at org.wso2.carbon.proxyadmin.ProxyServiceDeployer.deploySynapseArtifact(ProxyServiceDeployer.java:46)  
      at org.apache.synapse.deployers.AbstractSynapseArtifactDeployer.deploy(AbstractSynapseArtifactDeployer.java:194)  
      at org.wso2.carbon.application.deployer.synapse.SynapseAppDeployer.deployArtifacts(SynapseAppDeployer.java:130)  
      at org.wso2.carbon.application.deployer.internal.ApplicationManager.deployCarbonApp(ApplicationManager.java:263)  
      at org.wso2.carbon.application.deployer.CappAxis2Deployer.deploy(CappAxis2Deployer.java:72)  
      at org.apache.axis2.deployment.repository.util.DeploymentFileData.deploy(DeploymentFileData.java:136)  
      at org.apache.axis2.deployment.DeploymentEngine.doDeploy(DeploymentEngine.java:807)  
      at org.apache.axis2.deployment.repository.util.WSInfoList.update(WSInfoList.java:144)  
      at org.apache.axis2.deployment.RepositoryListener.update(RepositoryListener.java:377)  
      at org.apache.axis2.deployment.RepositoryListener.checkServices(RepositoryListener.java:254)  
      at org.apache.axis2.deployment.RepositoryListener.startListener(RepositoryListener.java:371)  
      at org.apache.axis2.deployment.scheduler.SchedulerTask.checkRepository(SchedulerTask.java:59)  
      at org.apache.axis2.deployment.scheduler.SchedulerTask.run(SchedulerTask.java:67)  
      at org.wso2.carbon.core.deployment.CarbonDeploymentSchedulerTask.runAxisDeployment(CarbonDeploymentSchedulerTask.java:93)  
      at org.wso2.carbon.core.deployment.CarbonDeploymentSchedulerTask.run(CarbonDeploymentSchedulerTask.java:138)  
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)  
      at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)  
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)  
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)  
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
      at java.lang.Thread.run(Thread.java:745)  
 [2016-04-12 14:34:48,672] ERROR - AbstractSynapseArtifactDeployer Deployment of the Synapse Artifact from file : /Users/shammi/wso2/Support-Issues/MOTOROLAMOBPROD-44/wso2esb-4.9.0/tmp/carbonapps/-1234/1460496888659MySecondCarProject1_1.0.1.car/myTestProxy_1.0.0/myTestProxy-1.0.0.xml : Failed!  
 org.apache.synapse.deployers.SynapseArtifactDeploymentException: ProxyService Deployment from the file : /Users/shammi/wso2/Support-Issues/MOTOROLAMOBPROD-44/wso2esb-4.9.0/tmp/carbonapps/-1234/1460496888659MySecondCarProject1_1.0.1.car/myTestProxy_1.0.0/myTestProxy-1.0.0.xml : Failed.  
      at org.apache.synapse.deployers.AbstractSynapseArtifactDeployer.handleSynapseArtifactDeploymentError(AbstractSynapseArtifactDeployer.java:475)  
      at org.apache.synapse.deployers.ProxyServiceDeployer.deploySynapseArtifact(ProxyServiceDeployer.java:112)  
      at org.wso2.carbon.proxyadmin.ProxyServiceDeployer.deploySynapseArtifact(ProxyServiceDeployer.java:46)  
      at org.apache.synapse.deployers.AbstractSynapseArtifactDeployer.deploy(AbstractSynapseArtifactDeployer.java:194)  
      at org.wso2.carbon.application.deployer.synapse.SynapseAppDeployer.deployArtifacts(SynapseAppDeployer.java:130)  
      at org.wso2.carbon.application.deployer.internal.ApplicationManager.deployCarbonApp(ApplicationManager.java:263)  
      at org.wso2.carbon.application.deployer.CappAxis2Deployer.deploy(CappAxis2Deployer.java:72)  
      at org.apache.axis2.deployment.repository.util.DeploymentFileData.deploy(DeploymentFileData.java:136)  
      at org.apache.axis2.deployment.DeploymentEngine.doDeploy(DeploymentEngine.java:807)  
      at org.apache.axis2.deployment.repository.util.WSInfoList.update(WSInfoList.java:144)  
      at org.apache.axis2.deployment.RepositoryListener.update(RepositoryListener.java:377)  
      at org.apache.axis2.deployment.RepositoryListener.checkServices(RepositoryListener.java:254)  
      at org.apache.axis2.deployment.RepositoryListener.startListener(RepositoryListener.java:371)  
      at org.apache.axis2.deployment.scheduler.SchedulerTask.checkRepository(SchedulerTask.java:59)  
      at org.apache.axis2.deployment.scheduler.SchedulerTask.run(SchedulerTask.java:67)  
      at org.wso2.carbon.core.deployment.CarbonDeploymentSchedulerTask.runAxisDeployment(CarbonDeploymentSchedulerTask.java:93)  
      at org.wso2.carbon.core.deployment.CarbonDeploymentSchedulerTask.run(CarbonDeploymentSchedulerTask.java:138)  
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)  
      at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)  
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)  
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)  
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
      at java.lang.Thread.run(Thread.java:745)  
 Caused by: org.apache.synapse.SynapseException: Cannot convert null to a StreamSource  
      at org.apache.synapse.config.SynapseConfigUtils.handleException(SynapseConfigUtils.java:578)  
      at org.apache.synapse.config.SynapseConfigUtils.getStreamSource(SynapseConfigUtils.java:79)  
      at org.apache.synapse.core.axis2.ProxyService.getPolicyFromKey(ProxyService.java:822)  
      at org.apache.synapse.core.axis2.ProxyService.buildAxisService(ProxyService.java:608)  
      at org.apache.synapse.deployers.ProxyServiceDeployer.deploySynapseArtifact(ProxyServiceDeployer.java:80)  
      ... 22 more  
 [2016-04-12 14:34:48,673] INFO - AbstractSynapseArtifactDeployer The file has been backed up into : NO_BACKUP_ON_WORKER.INFO  
 [2016-04-12 14:34:48,673] ERROR - AbstractSynapseArtifactDeployer Deployment of synapse artifact failed. Error reading /Users/shammi/wso2/Support-Issues/MOTOROLAMOBPROD-44/wso2esb-4.9.0/tmp/carbonapps/-1234/1460496888659MySecondCarProject1_1.0.1.car/myTestProxy_1.0.0/myTestProxy-1.0.0.xml : ProxyService Deployment from the file : /Users/shammi/wso2/Support-Issues/MOTOROLAMOBPROD-44/wso2esb-4.9.0/tmp/carbonapps/-1234/1460496888659MySecondCarProject1_1.0.1.car/myTestProxy_1.0.0/myTestProxy-1.0.0.xml : Failed.  
 org.apache.axis2.deployment.DeploymentException: ProxyService Deployment from the file : /Users/shammi/wso2/Support-Issues/MOTOROLAMOBPROD-44/wso2esb-4.9.0/tmp/carbonapps/-1234/1460496888659MySecondCarProject1_1.0.1.car/myTestProxy_1.0.0/myTestProxy-1.0.0.xml : Failed.  
      at org.apache.synapse.deployers.AbstractSynapseArtifactDeployer.deploy(AbstractSynapseArtifactDeployer.java:201)  
      at org.wso2.carbon.application.deployer.synapse.SynapseAppDeployer.deployArtifacts(SynapseAppDeployer.java:130)  
      at org.wso2.carbon.application.deployer.internal.ApplicationManager.deployCarbonApp(ApplicationManager.java:263)  
      at org.wso2.carbon.application.deployer.CappAxis2Deployer.deploy(CappAxis2Deployer.java:72)  
      at org.apache.axis2.deployment.repository.util.DeploymentFileData.deploy(DeploymentFileData.java:136)  
      at org.apache.axis2.deployment.DeploymentEngine.doDeploy(DeploymentEngine.java:807)  
      at org.apache.axis2.deployment.repository.util.WSInfoList.update(WSInfoList.java:144)  
      at org.apache.axis2.deployment.RepositoryListener.update(RepositoryListener.java:377)  
      at org.apache.axis2.deployment.RepositoryListener.checkServices(RepositoryListener.java:254)  
      at org.apache.axis2.deployment.RepositoryListener.startListener(RepositoryListener.java:371)  
      at org.apache.axis2.deployment.scheduler.SchedulerTask.checkRepository(SchedulerTask.java:59)  
      at org.apache.axis2.deployment.scheduler.SchedulerTask.run(SchedulerTask.java:67)  
      at org.wso2.carbon.core.deployment.CarbonDeploymentSchedulerTask.runAxisDeployment(CarbonDeploymentSchedulerTask.java:93)  
      at org.wso2.carbon.core.deployment.CarbonDeploymentSchedulerTask.run(CarbonDeploymentSchedulerTask.java:138)  
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)  
      at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)  
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)  
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)  
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
      at java.lang.Thread.run(Thread.java:745)  
 Caused by: org.apache.synapse.deployers.SynapseArtifactDeploymentException: ProxyService Deployment from the file : /Users/shammi/wso2/Support-Issues/MOTOROLAMOBPROD-44/wso2esb-4.9.0/tmp/carbonapps/-1234/1460496888659MySecondCarProject1_1.0.1.car/myTestProxy_1.0.0/myTestProxy-1.0.0.xml : Failed.  
      at org.apache.synapse.deployers.AbstractSynapseArtifactDeployer.handleSynapseArtifactDeploymentError(AbstractSynapseArtifactDeployer.java:475)  
      at org.apache.synapse.deployers.ProxyServiceDeployer.deploySynapseArtifact(ProxyServiceDeployer.java:112)  
      at org.wso2.carbon.proxyadmin.ProxyServiceDeployer.deploySynapseArtifact(ProxyServiceDeployer.java:46)  
      at org.apache.synapse.deployers.AbstractSynapseArtifactDeployer.deploy(AbstractSynapseArtifactDeployer.java:194)  
      ... 20 more  
 Caused by: org.apache.synapse.SynapseException: Cannot convert null to a StreamSource  
      at org.apache.synapse.config.SynapseConfigUtils.handleException(SynapseConfigUtils.java:578)  
      at org.apache.synapse.config.SynapseConfigUtils.getStreamSource(SynapseConfigUtils.java:79)  
      at org.apache.synapse.core.axis2.ProxyService.getPolicyFromKey(ProxyService.java:822)  
      at org.apache.synapse.core.axis2.ProxyService.buildAxisService(ProxyService.java:608)  
      at org.apache.synapse.deployers.ProxyServiceDeployer.deploySynapseArtifact(ProxyServiceDeployer.java:80)  
      ... 22 more  
 [2016-04-12 14:34:48,674] ERROR - ApplicationManager Error occurred while deploying Carbon Application  
 org.apache.axis2.deployment.DeploymentException: ProxyService Deployment from the file : /Users/shammi/wso2/Support-Issues/MOTOROLAMOBPROD-44/wso2esb-4.9.0/tmp/carbonapps/-1234/1460496888659MySecondCarProject1_1.0.1.car/myTestProxy_1.0.0/myTestProxy-1.0.0.xml : Failed.  
      at org.apache.synapse.deployers.AbstractSynapseArtifactDeployer.deploy(AbstractSynapseArtifactDeployer.java:213)  
      at org.wso2.carbon.application.deployer.synapse.SynapseAppDeployer.deployArtifacts(SynapseAppDeployer.java:130)  
      at org.wso2.carbon.application.deployer.internal.ApplicationManager.deployCarbonApp(ApplicationManager.java:263)  
      at org.wso2.carbon.application.deployer.CappAxis2Deployer.deploy(CappAxis2Deployer.java:72)  
      at org.apache.axis2.deployment.repository.util.DeploymentFileData.deploy(DeploymentFileData.java:136)  
      at org.apache.axis2.deployment.DeploymentEngine.doDeploy(DeploymentEngine.java:807)  
      at org.apache.axis2.deployment.repository.util.WSInfoList.update(WSInfoList.java:144)  
      at org.apache.axis2.deployment.RepositoryListener.update(RepositoryListener.java:377)  
      at org.apache.axis2.deployment.RepositoryListener.checkServices(RepositoryListener.java:254)  
      at org.apache.axis2.deployment.RepositoryListener.startListener(RepositoryListener.java:371)  
      at org.apache.axis2.deployment.scheduler.SchedulerTask.checkRepository(SchedulerTask.java:59)  
      at org.apache.axis2.deployment.scheduler.SchedulerTask.run(SchedulerTask.java:67)  
      at org.wso2.carbon.core.deployment.CarbonDeploymentSchedulerTask.runAxisDeployment(CarbonDeploymentSchedulerTask.java:93)  
      at org.wso2.carbon.core.deployment.CarbonDeploymentSchedulerTask.run(CarbonDeploymentSchedulerTask.java:138)  
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)  
      at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)  
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)  
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)  
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
      at java.lang.Thread.run(Thread.java:745)  
 Caused by: org.apache.axis2.deployment.DeploymentException: ProxyService Deployment from the file : /Users/shammi/wso2/Support-Issues/MOTOROLAMOBPROD-44/wso2esb-4.9.0/tmp/carbonapps/-1234/1460496888659MySecondCarProject1_1.0.1.car/myTestProxy_1.0.0/myTestProxy-1.0.0.xml : Failed.  
      at org.apache.synapse.deployers.AbstractSynapseArtifactDeployer.deploy(AbstractSynapseArtifactDeployer.java:201)  
      ... 20 more  
 Caused by: org.apache.synapse.deployers.SynapseArtifactDeploymentException: ProxyService Deployment from the file : /Users/shammi/wso2/Support-Issues/MOTOROLAMOBPROD-44/wso2esb-4.9.0/tmp/carbonapps/-1234/1460496888659MySecondCarProject1_1.0.1.car/myTestProxy_1.0.0/myTestProxy-1.0.0.xml : Failed.  
      at org.apache.synapse.deployers.AbstractSynapseArtifactDeployer.handleSynapseArtifactDeploymentError(AbstractSynapseArtifactDeployer.java:475)  
      at org.apache.synapse.deployers.ProxyServiceDeployer.deploySynapseArtifact(ProxyServiceDeployer.java:112)  
      at org.wso2.carbon.proxyadmin.ProxyServiceDeployer.deploySynapseArtifact(ProxyServiceDeployer.java:46)  
      at org.apache.synapse.deployers.AbstractSynapseArtifactDeployer.deploy(AbstractSynapseArtifactDeployer.java:194)  
      ... 20 more  
 Caused by: org.apache.synapse.SynapseException: Cannot convert null to a StreamSource  
      at org.apache.synapse.config.SynapseConfigUtils.handleException(SynapseConfigUtils.java:578)  
      at org.apache.synapse.config.SynapseConfigUtils.getStreamSource(SynapseConfigUtils.java:79)  
      at org.apache.synapse.core.axis2.ProxyService.getPolicyFromKey(ProxyService.java:822)  
      at org.apache.synapse.core.axis2.ProxyService.buildAxisService(ProxyService.java:608)  
      at org.apache.synapse.deployers.ProxyServiceDeployer.deploySynapseArtifact(ProxyServiceDeployer.java:80)  
      ... 22 more  



Thursday, April 14, 2016

How to get the Client's IP Address in WSO2 API Manager/ WSO2 ESB

Middleware solutions are designed to communicate with multiple parties and most of them are integrations. While integration different systems, It is required to validate the requests and collect statistics. When it comes to collecting statistics, Client's / Request Originator's IP Address plays a vital role.

In order to publish the client's IP to the stat collector, We need to extract the client's IP from the request received to the server.

When the deployment contains WSO2 API Manager or WSO2 Enterprise Service Bus, We can obtain the client's IP address using a property mediator in the InSequence.

If the deployment has a Load Balancer in front of ESB/APIManager, We can use X-Forwarded-For Header property as explained in the blog post of Firzhan.

In a deployment which doest not has Load Balancer in front of WSO2 ESB / API Manager, We can use REMOTE_ADDR to obtain the client's IP Address.

We can extract it as follows with using a property mediator.


 <property name="api.ut.REMOTE_ADDR"
                            expression="get-property('axis2','REMOTE_ADDR')"/&gt

Then we can use it in the sequence. As an example, if we extract the IP Address as above and log it, synapse configuration for it will look like bellow.


<property name="api.ut.REMOTE_ADDR"
                            expression="get-property('axis2','REMOTE_ADDR')"/>
                  <log level="full">
                     <property name="Actual Remote Address"
                               expression="get-property('api.ut.REMOTE_ADDR')"/>
                  </log>

You can use this in the InSequence of ESB or API Manager to obtain the client's IP Address.

Saturday, April 9, 2016

Customize HTTP Server Response Header in WSO2 API Manager / WSO2 ESB


You may know that in the response header from WSO2 ESB invocations or WSO2 API Manager invocations, You are getting "Server" header as bellow.

HTTP/1.1 200 OK
Access-Control-Allow-Headers: authorization,Access-Control-Allow-Origin,Content-Type,SOAPAction
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: POST,GET,DELETE,PUT,HEAD
Content-Type: application/json
Access-Control-Allow-Credentials: true
Date: Sat, 09 Apr 2016 20:02:58 GMT
Server: WSO2-PassThrough-HTTP
Transfer-Encoding: chunked
Connection: Keep-Alive

{
  "origin": "50.185.34.119"
}


You can see that Server Header Contains WSO2 as bellow.

Server: WSO2-PassThrough-HTTP

Sometimes there are situations like It needs to customize this header.

Eg: If we need to customize this as bellow.

Server: PassThrough-HTTP

What we need to do is , Add the http.origin-server to passthru-http.properties file located in ESB_HOME/repository/conf/ directory with customized value as bellow.


http.origin-server=PassThrough-HTTP

Once you restart the server, Above response will be changed as bellow.

HTTP/1.1 200 OK
Access-Control-Allow-Headers: authorization,Access-Control-Allow-Origin,Content-Type,SOAPAction
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: POST,GET,DELETE,PUT,HEAD
Content-Type: application/json
Access-Control-Allow-Credentials: true
Date: Sat, 09 Apr 2016 20:11:47 GMT
Server: PassThrough-HTTP
Transfer-Encoding: chunked
Connection: Keep-Alive

{
  "origin": "50.185.34.119"
}

Friday, April 8, 2016

ActiveMQ - WSO2 ESB - Redelivery Delay - MaximumRedeliveries Configuration


There are use cases which we need to configure Redelivery Delay and the Maximum Redeliveries for Message Consumers of Activemq.

When consuming ActiveMq Queue, we can configure these parameters as mentioned in [1]

WSO2 ESB also can act as a message consumer and a message producer. Information of configuring that can be found in ESB documentation. [2]  Then we can find Consumer / Producer configurations in [3]

In ESB JMS proxy, we can configure Redelivery Delay and MaximumRedeliveries with using following parameters.

Eg:  By default Redelivery Delay in ActiveMq is one second and MaximumRedelivery count is 6. If you need to change the as bellow, you can do it with following parameters in the proxy service.

Redelivery Delay - 3 Seconds
MaximumRedelivery Count - 2


   <parameter name="redeliveryPolicy.maximumRedeliveries">2</parameter>
   <parameter name="transport.jms.SessionTransacted">true</parameter>
   <parameter name="redeliveryPolicy.redeliveryDelay">3000</parameter>
   <parameter name="transport.jms.CacheLevel">consumer</parameter>

Other than enabling the default configurations for JMS transport receiver, you dont need to add any other parameters to the axis2.xml to achive this.

Here is a sample proxy service which i have added above parameters.

Tested Versions : ESB 4.9.0 / Apache ActiveMQ 5.10.0


<proxy xmlns="http://ws.apache.org/ns/synapse"
       name="JMStoHTTPStockQuoteProxy"
       transports="jms"
       statistics="disable"
       trace="disable"
       startOnLoad="true">
    <target>
        <inSequence>
            <log level="full">
                <property name="Status" value=" Consuming the message"/>
                <property name="transactionID" expression="get-property('MessageID')"/>
            </log>
            <property name="SET_ROLLBACK_ONLY" value="true" scope="axis2"/>
            <drop/>
        </inSequence>
    </target>
    <parameter name="redeliveryPolicy.maximumRedeliveries">2</parameter>
    <parameter name="transport.jms.DestinationType">queue</parameter>
    <parameter name="transport.jms.SessionTransacted">true</parameter>
    <parameter name="transport.jms.Destination">JMStoHTTPStockQuoteProxy</parameter>
    <parameter name="redeliveryPolicy.redeliveryDelay">3000</parameter>
    <parameter name="transport.jms.CacheLevel">consumer</parameter>
    <description/>
</proxy>


When we configure these redelivery parameters, we need to make sure that we have enabled transactions for the proxy. We have done it using following parameter.

<parameter name="transport.jms.SessionTransacted">true</parameter>

Once we enable transactions, If the transaction is successful there is no redelivery happens.  So, in order to test the redelivery functionality, After consuming the message, we need to RollBack the transaction. In order to do that we need add following parameter inside the InSequence of the proxy service.


<property name="SET_ROLLBACK_ONLY" value="true" scope="axis2"/>

With above property, we notify the server that, The tranaction is Rollbacked.

All these properties are passed in to the server when we make the connection from ESB to the Message Broker. So, at that time we need to specify all these parameters.


[1] http://activemq.apache.org/redelivery-policy.html
[2] https://docs.wso2.com/display/ESB490/Configure+with+ActiveMQ
[3] https://docs.wso2.com/display/ESB490/ESB+as+a+JMS+Consumer



Thursday, January 7, 2016

Introduction to BITCOIN

 What is the price of a Lamborghini in BitCoins ???








You should contact Lamborghini to find the today's price :). But you have to believe that this is real and world is moving towards digital currency. [1]

In this post, I would like to provide you a brief introduction to BitCoins after attending to a Meetup on introduction to BitCoins held on Silicon Valley yesterday.

There were plenty of questions I had in my mind before I attend to this meetup. So I thought many people will have the same and sharing the answers I got will help everyone.

What is a BitCoin?

BitCoin is a decentralized digital currency. It is virtual and you can not touch and feel. But you can transfer bitcoins from person to person or one organization to another organization. Basically, it is like normal cash or gold. But no any government or organization controls it.  As we have USD for US Dollars, BitCoins are identified as BTC. and its symbol is as in the bellow image.





In order to identify anything as a currency it should have four properties


  1. Divisible (Broken into small amounts)
  2. Durable ( Stands for a long time)
  3. Fungible 
  4. Verifiable (Verify that it is real)
All the currencies we trade for goods and services have all these properties. BitCoin also has the same four properties. The difference is, the currencies we use subjected to inflation. But BitCoins are not and it automatically controls the inflation.

As we know, US government can print US Dollars and issue to the market. There is no limit in how many bills it going to print over time. It makes the inflation. But in BitCoins, there is a limit of BitCoins which can be exchanged ever in the market. Currently, it has not reached that limit and that limit is 21 Million BitCoins.

How many BitCoins we have in the market at the moment?

At the moment I am writing this blog post, it has the amount in the following image.


You can get the most updated information on the count in [2]. BitCoins are not coming into the market automatically. BitCoins are mined by BitCoin miners. Approximately, in every ~10 mins, 25 new BitCoins are introduced to the market in the time I am writing this blog post. Reason behind saying that "at the time I am writing is", after another few years or months this rate of mining BitCoins become half of the current value.

Who Invented BitCoin?

Satoshi Nakamoto
In 2008 satoshi published a whitepaper on Bitcoin: A Peer-to-Peer Electronic Cash System[3]. Then he released the first client version of the bitcoin software in 2009. You can read more on satoshi with [4].

Who are BitCoin Miners?  

As I mentioned previously, BitCoins are not come into circulation automatically. They are mined by BitCoin miners. Satoshi was the first BitCoin miner and now there are huge amount of BitCoin miners in the world.

What is the task of a BitCoin Miner? Do they dig the soil to mine?

It is more than Digging :).  BitCoin mining comes into the picture with a transaction. Lets assume as following.



Alice has 50 BTC in her wallet. Then she needs to give 30 BTC to Bob and She needs to get 19 BTC as change and 1 BTC as transaction fee.  This is a particular transaction as we do in normal currency. Lets assume Alice has a 50$ bill. She gives it to intermediate party to give which Alice and Bob trust to give 30$ to Bob and get 19$ back to Alice as change. Then she asks intermediate party to keep 1$ as transaction fee.

So, in this digital currency transaction it is as same. But here this 50BTC is kept in an address which has a public key and a private key. This address is encrypted and the public key of this address can be used to decrypt and find the amount of BTC has in this address. But only the private key of this address can be used to send BitCoins from this address to another address.

So, it is important that this private key is stored very securely since once a hacker has this private key, it is like a thief got the wallet. When Alice wants to send money to Bob, what she does is three steps.

She uses a BitCoin Client application. She provides the address of Bob's wallet which she wants to send money. Then she provides the amount she expects to send to Bob, change she expects and the transaction fee. Once she confirms the transaction with her private key, this transaction is done.

Then this transaction is going to all the miners in the world through a distributed communication system. Like this transaction, there are plenty of transactions happening in the BitCoin world and all those transactions are received to the minors.

When there is some number of transactions received by the Miners,  Then they start to validate those transactions first. After validating all those transactions, they need to use a hashing algorithm to hash these set of transactions into a 10 digit number. When hashing these set of transactions to into a 10 digit number, they have to use a random number to make it happens. So it is not an easy task.

As I mentioned previously there are plenty of BitCoin miners in the world. It is like a contest and everybody tries to come up with this 10 digit number and the first one to come up with that number gets the reward.

The reward is 25BTC and all the transaction fees in this transaction. At the moment, the value of a BitCoin in USD is $456.77. So, what do you think? How much you can earn?

Ok, From where this 25BTC comes from?

It comes from Nowhere. That is called mining. When the minor starts to hash and find this 10 digit number, He adds this as a transaction like bellow.  This address 1axxas... is the public address of this miner. So this 25BTC is mined with this transaction.

25 BTC To Address 1axxas...

After finding 10 Digit number What happens?

Once a minor found this 10 digit number for the set of transactions, he broadcasts this number to the entire world.  These is common ledger called "Block Chain". Every minor has the same copy of this common ledger in the world and it has each and every transaction happen from the first BitCoin.
So, once the above 10 Digit number is broadcasted, everybody who mines adds this set of transactions with this number to their ledger. This set of transactions and the number is called a "Block".



The above mentioned 10 digit number is the link from one block to another block. As everybody agrees to use the same key over the entire world, this block is getting saved in everyone's block chain.
After this block is get saved in every ones ledger and it was added three more blocks, the transaction is confirmed.

You can read more information about block chains and how it works in [5].

Can I be a Miner?

Of course, you can. But you will get 25BTC only if you win the contest. You 'll need huge computational power to be the winner. Currently, there is huge computation power used to my miners to mine. Following is a BitCoin mining farm.




Where can I buy BitCoins?

You can buy BitCoins from BitCoin ATMs who are selling BitCoins[6]. You can pay with USDs if you are in USA and you can pay in Yens if you are in Japan.



Who accept BitCoins?

As I mentioned in the beginning, world is moving to the digital currency. So there will be most of the people accepting BitCoins for transactions like buying a coffee to buying a Lamborghini.

Conclusion
This is just an introduction to BitCoins and how it works. There are plenty of resources in the web which you can learn about BitCoins and how it works. Here I am listing few of them for your convenience.

Finally, I would like to thank to Donn Lee who held this meetup and Prabath Siriwardene who asked me to come to this Meetup.

[1] https://www.cryptocoinsnews.com/buying-cars-bitcoin-lamborghini-jeep-everything/
[2] http://realtimebitcoin.info
[3] http://www.coindesk.com/bitcoin-peer-to-peer-electronic-cash-system/
[4] http://www.coindesk.com/information/who-is-satoshi-nakamoto/
[5] http://www.coindesk.com/information/how-do-bitcoin-transactions-work/
[6] http://www.coindesk.com/bitcoin-atm-map/

Additional Resources