Wednesday, August 5, 2015

How to setup networking for Virtualbox VMs in Mac

By default when we configure Virtual box VM for  Mac we are getting networking with NAT. The configuration looks like bellow.




But with that i could not PING from my Mac to the Windows VM or vice-versa.  As usual i changed the connection to Bridge mode but no success. Then contacted my Networking friend "Dhanushka Ranasinge" and with his guide i could get it done.

Following is the TIP.

For the VM
1. Change the  network interface to "Host-Only Adapter" as in the bellow image.


2. Then start the VM and disable to windows firewall.



Now you are in a STATE that you can PING from your MAC to WINDOWS.

Now we need to set up internet for the VM.

3. Shutdown the VM and add second Network adaptor for NAT



With this NAT interface , it will connect your VM to internet. Now it is all done. Start your VM and you should be able to connect to INTERNET and you should be able to PING from MAC to WINDOWS and vice-versa.

Thursday, July 9, 2015

How to deal with "java.nio.charset.MalformedInputException: Input length = 1" WSO2 ESB

Some times we are receiving unusual characters in our responses from various back ends. Then WSO2 ESB is facing problems in understanding those characters from the response and it tends to throw following exception.

[2015-07-09 12:42:49,651] ERROR - TargetHandler I/O error: Input length = 1
java.nio.charset.MalformedInputException: Input length = 1
    at java.nio.charset.CoderResult.throwException(CoderResult.java:277)
    at org.apache.http.impl.nio.reactor.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:193)
    at org.apache.http.impl.nio.codecs.AbstractMessageParser.parse(AbstractMessageParser.java:171)
    at org.apache.synapse.transport.http.conn.LoggingNHttpClientConnection$LoggingNHttpMessageParser.parse(LoggingNHttpClientConnection.java:210)
    at org.apache.synapse.transport.http.conn.LoggingNHttpClientConnection$LoggingNHttpMessageParser.parse(LoggingNHttpClientConnection.java:192)
    at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:171)
    at org.apache.synapse.transport.http.conn.LoggingNHttpClientConnection.consumeInput(LoggingNHttpClientConnection.java:106)
    at org.apache.synapse.transport.passthru.ClientIODispatch.onInputReady(ClientIODispatch.java:83)
    at org.apache.synapse.transport.passthru.ClientIODispatch.onInputReady(ClientIODispatch.java:41)
    at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:119)
    at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:160)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:342)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:320)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:280)
    at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:106)
    at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:604)
    at java.lang.Thread.run(Thread.java:745)


Most of the case,  by adding following property to the "passthru-http.properties" file will solve the case.

Eg:  once i was getting a response like following

[2015-07-09 12:47:55,236] DEBUG - wire >> "HTTP/1.1 100 Continue[\r][\n]"
[2015-07-09 12:47:55,237] DEBUG - wire >> "[\r][\n]"
[2015-07-09 12:47:55,274] DEBUG - wire >> "HTTP/1.1 201 Cr[0xe9]e[\r][\n]"
[2015-07-09 12:47:55,274] DEBUG - wire >> "Set-Cookie: JSESSIONID=87101C43FCABBB97D049C0F0C8DD216D; Path=/api/; Secure; HttpOnly[\r][\n]"
[2015-07-09 12:47:55,274] DEBUG - wire >> "Location: https://abc.foo.com/api/office/files/2038[\r][\n]"
[2015-07-09 12:47:55,274] DEBUG - wire >> "Content-Type: application/json[\r][\n]"
[2015-07-09 12:47:55,274] DEBUG - wire >> "Transfer-Encoding: chunked[\r][\n]"
[2015-07-09 12:47:55,275] DEBUG - wire >> "Vary: Accept-Encoding[\r][\n]"
[2015-07-09 12:47:55,275] DEBUG - wire >> "Date: Thu, 09 Jul 2015 16:47:55 GMT[\r][\n]"
[2015-07-09 12:47:55,275] DEBUG - wire >> "Server: qa-byp[\r][\n]"
[2015-07-09 12:47:55,275] DEBUG - wire >> "[\r][\n]"
[2015-07-09 12:47:55,275] DEBUG - wire >> "1f[\r][\n]"
[2015-07-09 12:47:55,275] DEBUG - wire >> "{"person":john}[\r][\n]"
[2015-07-09 12:47:55,275] DEBUG - wire >> "0[\r][\n]"
[2015-07-09 12:47:55,275] DEBUG - wire >> "[\r][\n]"


Then it thew the above exception.

By adding the above entry as follows, i could get rid of that.

http.protocol.element-charset=iso-8859-1

සිතුවිල්ල





එදා සුදු ගවුම ඇඳන් බස් එකෙන් ඇවිත් බහිනකල්
නොඉවසිල්ලෙන් බස් ස්ටැන්ඩ් එකට වෙලා බලන් උන්නු මම...
අද..
පුංචි පුතාගෙ හුරතල් හිනාව බලන්න
ගෙදර යන්න ඔරලොසුවෙ කටුව කැරකෙනකල් බලන් ඉන්නවා..
කාලය... නුඹේ අරුමය..

Wednesday, July 8, 2015

WSO2 ESB API with JMS Queues - HTTP GET

We are going to discuss on how we can handle HTTP GET methods with JMS Proxy services in WSO2. With this blog we are going to discuss following message flow.


Flow is like follows;


  • Client invokes an API defined in ESB
  • From the API it sends the message to a JMS queue specifying a "ReplyTo" queue
  • There is a JMS proxy consume message from the above queue 
  • After consuming message, it will invoke the backend and response will be sent back to JMS "Reply To" queue
  • Rest API will consume the message from that "Reply To" queue and send back the response to the client.


In order to have this flow working, you need to setup ActiveMQ with wso2 ESB. You can find the documentation to do it in this documentation page.

Lets see the API


<api xmlns="http://ws.apache.org/ns/synapse" name="WeatherAPI" context="/getQuote">
   <resource methods="POST GET" uri-template="/details?*">
      <inSequence>
         <property name="transport.jms.ContentTypeProperty" value="Content-Type" scope="axis2"></property>
         <log>
            <property name="httpMethod" expression="get-property('axis2','HTTP_METHOD')"></property>
         </log>
         <property name="HTTP_METHOD" expression="get-property('axis2','HTTP_METHOD')" scope="transport" type="STRING"></property>
         <send>
            <endpoint>
               <address uri="jms:/SMSStore?transport.jms.ConnectionFactoryJNDIName=QueueConnectionFactory&java.naming.factory.initial=org.apache.activemq.jndi.ActiveMQInitialContextFactory&java.naming.provider.url=tcp://localhost:61616&transport.jms.DestinationType=queue&transport.jms.ContentTypeProperty=Content-Type&transport.jms.ReplyDestination=SMSReceiveNotificationStore"></address>
            </endpoint>
         </send>
      </inSequence>
      <outSequence>
         <property name="TRANSPORT_HEADERS" scope="axis2" action="remove"></property>
         <send></send>
      </outSequence>
   </resource>
</api>

In the above API , there are few things to be focused on.
  • You can see that we are setting the HTTP_METHOD to a property extracted from the incoming message as follows.  By default, for HTTP POST method, API invocation works perfectly with out setting this property at the end. But If we are doing a HTTP GET , we need to obtain the HTTP method at the consumer side of the queue to invoke the back end with a http GET. So we are setting it here. 

<property name="HTTP_METHOD" expression="get-property('axis2','HTTP_METHOD')" scope="transport" type="STRING"></property>


  • In the endpoint of the send mediator we are setting up following information which you should have 
    • transport.jms.ContentTypeProperty=Content-Type  - We are saying that we are passing the content type in JMS transport
    • ransport.jms.ReplyDestination=SMSReceiveNotificationStore - We are saying that ESB is expecting a response to this queue for this request 

Then in the JMS proxy which we consume the message, it will look like follows.


<proxy xmlns="http://ws.apache.org/ns/synapse"
       name="SMSForwardProxy"
       transports="jms"
       statistics="disable"
       trace="disable"
       startOnLoad="true">
   <target faultSequence="fault">
      <inSequence>
         <property name="HTTP_METHOD" expression="$trp:HTTP_METHOD" scope="axis2"/>
         <send>
            <endpoint>
               <address uri="http://localhost:8080/foo">
                  <suspendOnFailure>
                     <errorCodes>-1</errorCodes>
                     <progressionFactor>1.0</progressionFactor>
                  </suspendOnFailure>
                  <markForSuspension>
                     <errorCodes>-1</errorCodes>
                  </markForSuspension>
               </address>
            </endpoint>
         </send>
      </inSequence>
      <outSequence>
         <send/>
      </outSequence>
   </target>
   <parameter name="transport.jms.ContentType">
      <rules>
         <jmsProperty>contentType</jmsProperty>
         <default>text/xml</default>
      </rules>
   </parameter>
   <parameter name="transport.jms.ConnectionFactory">myQueueConnectionFactory</parameter>
   <parameter name="transport.jms.DestinationType">queue</parameter>
   <parameter name="transport.jms.Destination">SMSStore</parameter>
   <description/>
</proxy>


In the above proxy configuration, you can see following line


 <property name="HTTP_METHOD" expression="$trp:HTTP_METHOD" scope="axis2"/>

In that , what we do is , we read the HTTP_METHOD we already set in previous API before sending the message to JMS Queue and now we are reading it from transport level property.  Then we are setting it to "HTTP_METHOD" property with the "axis2" scope.

With that it will be able to figure out the HTTP method to be used on invoking the actual endpoint.


After getting a response, this Proxy will send the reply to the "SMSReceiveNotificationStore" queue. Then it will consumed from the previous API and send back the response to the client.

This will be working perfectly for HTTP POST requests. If it is a HTTP GET Request , you need to do some additional changes from the client side and the server side.

Client Side Changes


  • We need to pass the content type from the client side as follows. You can set it as a header in SOAP UI
       contentType:  application/x-www-form-urlencoded




Server Side Changes
  • You need to change the builder and formatter for above content type in the axis2.xml file located in WSO2ESB/repository/conf/axis2 directory as follows. 

.....

<messageFormatter contentType="application/x-www-form-urlencoded"
                        class="org.wso2.carbon.relay.ExpandingMessageFormatter"/>

.......

<messageBuilder contentType="application/x-www-form-urlencoded"
                        class="org.wso2.carbon.relay.BinaryRelayBuilder"/>

......

If you does not set above , there will be message building and formatting errors.


With this way you ll be able to get this working for GET http methods.

This is tested with WSO2 ESB 4.8.1 with it's latest patches and Apache Activemq 5.10.0


Monday, June 29, 2015

How to generate a custom Error Message with Custom HTTP Status Code for unavailable Resources in WSO2 ESB


WSO2 ESB 4.8.1  does not throw any exception or error message when an API defined is access with incorrect HTTP method and it will just respond with 202.  In this blog post , i am explaining on how we can get a custom HTTP status code for the above.


In order to get a custom error message , you need to add following sequence to ESB which is not there by default.


<?xml version="1.0" encoding="UTF-8"?>
<sequence xmlns="http://ws.apache.org/ns/synapse" name="_resource_mismatch_handler_">
   <payloadFactory media-type="xml">
      <format>
         <tp:fault xmlns:tp="http://test.com">
            <tp:code>405</tp:code>
            <tp:type>Status report</tp:type>
            <tp:message>Method not allowed</tp:message>
            <tp:description>The requested HTTP method for resource (/$1) is not allowed.</tp:description>
         </tp:fault>
      </format>
      <args>
         <arg xmlns:ns="http://org.apache.synapse/xsd"
              xmlns:ns3="http://org.apache.synapse/xsd"
              evaluator="xml"
              expression="$axis2:REST_URL_POSTFIX"/>
      </args>
   </payloadFactory>
   <property name="NO_ENTITY_BODY" scope="axis2" action="remove"/>
   <property name="HTTP_SC" value="405" scope="axis2"/>
   <respond/>
   <drop/>
</sequence>

In ESB documentation [1] , it has explained that in order to handle non-matching resources, it is needed to define this sequence _resource_mismatch_handler_


[1] https://docs.wso2.com/display/ESB481/Configuring+Specific+Use+Cases#ConfiguringSpecificUseCases-Handlingnon-matchingresources


How to generate a custom Error Message with Custom HTTP Status Code for unavailable Resources in WSO2 API Manager

We are going to explain on how we can generate a custom HTTP Status code for a request which is addressed to a un-matching resource of an API.

Problem :

When an API exposed with resource "GET" , if the client invoke the API with "POST","PUT" or any other which is not "GET", By default API manager returns following.

{  
   "fault":{  
      "code":"900906",
      "type":"Status report",
      "message":"Runtime Error",
      "description":"No matching resource found in the API for the given request"
   }
}

In the RAW level you ll see it as follows

HTTP/1.1 403 Forbidden
Access-Control-Allow-Headers: authorization,Access-Control-Allow-Origin,Content-Type
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET,PUT,POST,DELETE,OPTIONS
Content-Type: application/xml; charset=UTF-8
Date: Mon, 29 Jun 2015 14:46:29 GMT
Server: WSO2-PassThrough-HTTP
Transfer-Encoding: chunked
Connection: Keep-Alive

<ams:fault xmlns:ams="http://wso2.org/apimanager/security">
   <ams:code>900906</ams:code>
   <ams:message>No matching resource found in the API for the given request</ams:message>
   <ams:description>Access failure for API: /sss, version: v1 with key: 4a33dc81be68d1b7a5b48aeffebe7e</ams:description>
</ams:fault>



Expected Solution :

We need to change this HTTP Response code 405 [1] with a custom error message.



Solution :

We need to create a sequence which builds the custom error message and the error code and deploy it in API manager's default sequences folder.


<?xml version="1.0" encoding="UTF-8"?>
<sequence xmlns="http://ws.apache.org/ns/synapse" name="converter">
    <payloadFactory media-type="xml">
        <format>
            <am:fault xmlns:am="http://wso2.org/apimanager">
                <am:message>Resource not found</am:message>
                <am:description>Wrong http method</am:description>
            </am:fault>
        </format>
    </payloadFactory>
    <property name="RESPONSE" value="true"/>
    <header name="To" action="remove"/>
    <property name="HTTP_SC" value="405" scope="axis2"/>
    <property name="messageType" value="application/xml" scope="axis2"/>
    <send/>
</sequence>

You can save this as converter.xml in wso2am-1.8.0/repository/deployment/server/synapse-configs/default/sequences folder.

Then we need to invoke this sequence in _auth_failure_handler_.xml which is located in the above sequences folder. In order to do that , we need to change it as follows.


<?xml version="1.0" encoding="UTF-8"?>
<sequence xmlns="http://ws.apache.org/ns/synapse" name="_auth_failure_handler_">
   <property name="error_message_type" value="application/xml"/>
    <filter source="get-property('ERROR_CODE')" regex="900906">
      <then>
          <sequence key="converter"/>
          <drop/>
      </then>
      <else>
      </else>
    </filter>
    <sequence key="_build_"/>
</sequence>


Once you done the above changes, save them. Then you can test your scenario. If you are successful with this , you ll be able see following response


HTTP/1.1 405 Method Not Allowed
Host: 10.210.1.202:8243
Content-Type: application/xml
Date: Mon, 29 Jun 2015 14:59:12 GMT
Server: WSO2-PassThrough-HTTP
Transfer-Encoding: chunked
Connection: Keep-Alive

<am:fault xmlns:am="http://wso2.org/apimanager">
   <am:message>Resource not found</am:message>
   <am:description>Wrong http method</am:description>
</am:fault>

Explanation : 

By default, when we invoke an non-existing resource it will send the default 403 error code with the message "No matching resource found in the API for the given request". If you check the log of the WSO2 AM, you can see that it has thrown following exception in the backend.


[2015-06-29 10:59:12,103] ERROR - APIAuthenticationHandler API authentication failure
org.wso2.carbon.apimgt.gateway.handlers.security.APISecurityException: Access failure for API: /sss, version: v1 with key: 4a33dc81be68d1b7a5b48aeffebe7e
    at org.wso2.carbon.apimgt.gateway.handlers.security.oauth.OAuthAuthenticator.authenticate(OAuthAuthenticator.java:212)
    at org.wso2.carbon.apimgt.gateway.handlers.security.APIAuthenticationHandler.handleRequest(APIAuthenticationHandler.java:94)
    at org.apache.synapse.rest.API.process(API.java:284)
    at org.apache.synapse.rest.RESTRequestHandler.dispatchToAPI(RESTRequestHandler.java:83)
    at org.apache.synapse.rest.RESTRequestHandler.process(RESTRequestHandler.java:64)
    at org.apache.synapse.core.axis2.Axis2SynapseEnvironment.injectMessage(Axis2SynapseEnvironment.java:220)
    at org.apache.synapse.core.axis2.SynapseMessageReceiver.receive(SynapseMessageReceiver.java:83)
    at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180)
    at org.apache.synapse.transport.passthru.ServerWorker.processNonEntityEnclosingRESTHandler(ServerWorker.java:344)
    at org.apache.synapse.transport.passthru.ServerWorker.processEntityEnclosingRequest(ServerWorker.java:385)
    at org.apache.synapse.transport.passthru.ServerWorker.run(ServerWorker.java:183)
    at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:744)

When it throws above exception, the flow will hit the  _auth_failure_handler_.xml
sequence. So what we have done in this sequence, with using the filter mediator, we have filtered the error code "900906" and for that error code, we invoke our custom sequence and drop the message then.

In the custom sequence , we have used the payload factory mediator to create the payload and added required properties to make it as response. You can find the information further on each of those properties from [2][3][4]

Then after invoking the custom sequence, it will invoke the "_build_" sequence in the same folder which invoke the message builders to build the message.

I have used resources [4] on creating this blog post.


[1] http://www.checkupdown.com/status/E405.html#
[2] https://docs.wso2.com/display/ESB481/Generic+Properties#GenericProperties-RESPONSE
[3] https://docs.wso2.com/display/ESB481/Generic+Properties#GenericProperties-messageType
[4] http://sanjeewamalalgoda.blogspot.com/2015/04/how-to-generate-custom-error-message.html

Thursday, June 25, 2015

How to add a thread sleep to a Proxy Service

Here i am going to provide you a example on how we can create a mock service with WSO2 ESB and adding a sleep to that service.

In order to do that we need to use ;

  1. Payload Factory mediator to create the mock response
  2. script mediator to do a thread sleep
Here is the simple mock service proxy with a thread sleep.

<proxy xmlns="http://ws.apache.org/ns/synapse"
       name="MyMockProxy"
       transports="https,http"
       statistics="disable"
       trace="disable"
       startOnLoad="true">
   <target>
      <inSequence>
         <log>
            <property name="===Before sleep===" value="===Before sleep==="/>
         </log>
         <script language="js">java.lang.Thread.sleep(75000);</script>
         <log>
            <property name="===After sleep===" value="===After sleep==="/>
         </log>
         <payloadFactory media-type="xml">
            <format>
               <Response xmlns="">
                  <status>OK</status>
                  <code>1</code>
               </Response>
            </format>
            <args/>
         </payloadFactory>
         <header name="To" action="remove"/>
         <property name="RESPONSE" value="true" scope="default" type="STRING"/>
         <property name="NO_ENTITY_BODY" scope="axis2" action="remove"/>
         <send/>
      </inSequence>
   </target>
   <description/>
</proxy>

I have used the blog of miyuru [1] to create this.

[1] http://miyurudw.blogspot.com/2012/08/how-to-create-simple-mock-services.html