WSO2 API Manager with Consul for Dynamic endpoints in distributed deployment

WSO2 API Manager 2.1.0 + is fully capable of supporting dynamic endpoints. As you see in the documentation [1], In order to use the dynamic endpoint, You need to us TO header

The dynamic endpoint sends the message to the address specified in the To header. You can configure dynamic endpoints by setting mediation extensions with a set of conditions to dynamically change the To header. 

As you may already understand, with the above implementation, We are setting the to header in the mediation extension as a hardcoded value. If you want to change the endpoint, you need to set the correct mediation extension and re-publish the API.


QUESTION:

How can we change the endpoint dynamically, without re-publishing the API?

- You can read the endpoint from a database
- You can read the endpoint from the file system
- You can use a service registry like Consul

When it comes to production deployment, Calling a DB for each every API request is pretty expensive. What can we do?

Can't we cache the DB response?

Yes, We can. But when the cache is expiring. What if I want to change the endpoint before cache expires?

Same thing with the file system also. Reading from the file for each and every API call is pretty expensive.

Apart from the above, When it comes to distributed deployment in a production environment which multi-data centers,  We can not use a database or filesystem until they are shared.

We can overcome all of the above issues with using a multi data center supported service discovery tool which is Consul.

How to use consul with WSO2 API Manager

• Downloaded consul binary from https://www.consul.io/downloads.html
• Added the consul to the $PATH

  export CONSUL_HOME=/Users/shammijayasinghe/wso2/tools/consul;
  export PATH=$PATH:$CONSUL_HOME
  

• Created a directory for data of consul /Users/shammijayasinghe/wso2/tools/consul/data
• Started the consul with the command consul agent -server -bind=0.0.0.0 -data-dir=/Users/shammijayasinghe/wso2/tools/consul/data1 -bootstrap

Shammis-MacBook-Pro:consul shammijayasinghe$ consul agent -server -bind=0.0.0.0 -data-dir=/Users/shammijayasinghe/wso2/tools/consul/data1 -bootstrap
==> WARNING: Bootstrap mode enabled! Do not enable unless necessary
==> Starting Consul agent...
==> Consul agent running!
           Version: 'v0.9.2'
           Node ID: 'a51633be-0115-ccd2-dd25-4d70cf5d6afa'
         Node name: 'Shammis-MacBook-Pro.local'
        Datacenter: 'dc1'
            Server: true (bootstrap: true)
       Client Addr: 127.0.0.1 (HTTP: 8500, HTTPS: -1, DNS: 8600)
      Cluster Addr: 10.0.0.3 (LAN: 8301, WAN: 8302)
    Gossip encrypt: false, RPC-TLS: false, TLS-Incoming: false

==> Log data will now stream in as it occurs:

    2017/09/01 08:32:46 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:10.0.0.3:8300 Address:10.0.0.3:8300}]
    2017/09/01 08:32:46 [INFO] raft: Node at 10.0.0.3:8300 [Follower] entering Follower state (Leader: "")
    2017/09/01 08:32:46 [INFO] serf: EventMemberJoin: Shammis-MacBook-Pro.local.dc1 10.0.0.3
    2017/09/01 08:32:46 [WARN] serf: Failed to re-join any previously known node
    2017/09/01 08:32:46 [INFO] serf: EventMemberJoin: Shammis-MacBook-Pro.local 10.0.0.3
    2017/09/01 08:32:46 [WARN] serf: Failed to re-join any previously known node
    2017/09/01 08:32:46 [INFO] consul: Handled member-join event for server "Shammis-MacBook-Pro.local.dc1" in area "wan"
    2017/09/01 08:32:46 [INFO] consul: Adding LAN server Shammis-MacBook-Pro.local (Addr: tcp/10.0.0.3:8300) (DC: dc1)
    2017/09/01 08:32:46 [INFO] agent: Started DNS server 127.0.0.1:8600 (udp)
    2017/09/01 08:32:46 [INFO] agent: Started DNS server 127.0.0.1:8600 (tcp)
    2017/09/01 08:32:46 [INFO] agent: Started HTTP server on 127.0.0.1:8500
    2017/09/01 08:32:52 [WARN] raft: Heartbeat timeout from "" reached, starting election
    2017/09/01 08:32:52 [INFO] raft: Node at 10.0.0.3:8300 [Candidate] entering Candidate state in term 8
    2017/09/01 08:32:52 [INFO] raft: Election won. Tally: 1
    2017/09/01 08:32:52 [INFO] raft: Node at 10.0.0.3:8300 [Leader] entering Leader state
    2017/09/01 08:32:52 [INFO] consul: cluster leadership acquired
    2017/09/01 08:32:52 [INFO] consul: New leader elected: Shammis-MacBook-Pro.local
    2017/09/01 08:32:52 [INFO] agent: Synced node info

Now the consul server is up and running on my local machine. Now what we need to do is we need to add a key-value pair to the consul registry.

Insert a Key-Value pair using API call

curl     --request PUT     --data "http://www.mocky.io/v2/59a96c49100000300d3e0afa"     http://127.0.0.1:8500/v1/kv/MyMockEndpoint

check and verify whether the key-value pair is available in consul

curl     http://127.0.0.1:8500/v1/kv/MyMockEndpoint


[{"LockIndex":0,"Key":"MyMockEndpoint","Flags":0,"Value":"aHR0cDovL3d3dy5tb2NreS5pby92Mi81OWE5NmM0OTEwMDAwMDMwMGQzZTBhZmE=","CreateIndex":8,"ModifyIndex":183}]

Now it is verified that with the API call, We can retrieve the value we stored.

Next step is to use this within wso2 API Manager. In order to do that, I have created a mediation extension with the following synapse configuration.

 <?xml version="1.0" encoding="UTF-8"?>  
 <sequence name="customRoutingSequence" trace="disable" xmlns="http://ws.apache.org/ns/synapse">  
   <log>  
     <property name="State" value="Inside Custom Routing Sequence"/>  
   </log>  
   <call blocking="true">  
     <endpoint>  
       <address uri="http://127.0.0.1:8500/v1/kv/MyMockEndpoint"/>  
     </endpoint>  
   </call>  
   <log level="full">  
     <property name="Response From Consul" value="======"/>  
   </log>  
   <property description="consulEndpoint" expression="json-eval($.[0].Value)" name="consulEndpoint" scope="default" type="STRING"/>  
   <property description="decodedConsulEndpoint" expression="base64Decode($ctx:consulEndpoint)" name="decodedConsulEndpoint" scope="default" type="STRING"/>  
   <log>  
     <property expression="$ctx:decodedConsulEndpoint" name="decodedConsulEndpoint"/>  
   </log>  
   <header name="To" expression="$ctx:decodedConsulEndpoint"/>  
 </sequence>  

As you can notice, In above mediation extension, I am using a blocking call to the consul endpoint and retrieve the endpoint. But as it is encoded with base 64, In order to use it, I have to decode it.

I am decoding it with the following property mediator.

 <property description="decodedConsulEndpoint" expression="base64Decode($ctx:consulEndpoint)" name="decodedConsulEndpoint" scope="default" type="STRING"/>  

Then I am setting it to the TO header as we discussed in the beginning.

 
  <header name="To" expression="$ctx:decodedConsulEndpoint"/>  

Then we have to create an API with using
- Above created mediation extension
- Using the endpoint type as Dynamic


Once deployed, Invoke the API.

Then in order to check the dynamic endpoint change, Change the value of the above key by following curl command.

 curl   --request PUT   --data "[newEndpointURL]"   http://127.0.0.1:8500/v1/kv/MyMockEndpoint  

Once you provide the new endpoint URL above, You should be able to see that API is invoking this given backend URL without republishing the API.

With this way, We can make the API Manager really flexible in a distributed environment, As consul can be configured as a cluster and supported with Multi Data Centers. So, You can get your requiement of changing the backend endpoint of a cluster of API Manager which contains any number of nodes with a single curl command.


[1] https://docs.wso2.com/display/AM210/Working+with+Endpoints

Deploy WSO2 products with valid CA (Certificate Authority) signed certificate

This blog post will contain multiple posts as it is too long to have all the information in one post.

Part 1 - Creating a keystore and generating Certificate Signing Request (CSR)

When you are searching for the topic of this post or for the following exception on the internet with related to WSO2, you will come across following article from Amila Jayasekara [1]

 curl: (60) Peer certificate cannot be authenticated with known CA certificates  
 More details here: http://curl.haxx.se/docs/sslcerts.html  
 curl performs SSL certificate verification by default, using a "bundle"  
 of Certificate Authority (CA) public keys (CA certs). If the default  
 bundle file isn't adequate, you can specify an alternate file  
 using the --cacert option.  
 If this HTTPS server uses a certificate signed by a CA represented in  
 the bundle, the certificate verification probably failed due to a  
 problem with the certificate (it might be expired, or the name might  
 not match the domain name in the URL).  
 If you'd like to turn off curl's verification of the certificate, use  
 the -k (or --insecure) option.  

It is a great article from Amila and i followed the same some time back. However, I thought to share my experience on using easy UI tool for the same task.

When it comes to using CA (Certificate Authority) signed certificate in your production server, There are few steps to carry out.

First, you need to decide whether you are going to use your already existing and valid CA signed certificate or whether you are going to create new keystore and generate key pair and get them signed from a CA.

So here we are discussing both of those approaches.

1. Create a keystore and generate keypair and use them for configuring
2. Use existing keypair in default wso2 keystore

The tool which i am going to use here is Keystore Explorer. You can get it from [2]

Creating a keystore and keypair

Launch Keystore Explorer

Select Create a new key store

Choose KeyStore type as JKS and then save (CTRL + S). It will ask for password for keystore.

Note: When using WSO2 products, Key password and Keystore password should be same.

After setting the password, It will ask for the name for the keystore. You can provide any name and save it.

Once you saved it, Generate a key pair from the tools menu as bellow.

It will ask for the algorithm

It will ask for the other fields

You need to provide the host name as your CN when you configuring the name field by clicking on the icon in front of the name field as bellow.

then you have to confirm the information.

Now it will ask for the key alias, By default it will select the given CN name.

Provide a password for the key pair. As in the above given note When using WSO2 products, Key password and Keystore password should be same. 

Now we done with the process of generating the keypair.

Our next step is to create a Certificate Signing Request (CSR) from the above keypair

Creating a CSR (Certificate Signing Request)

By right clicking on the keypair , you can select the Generate CSR option. It will generate the CSR and ask for saving.

In my case it will generate it as myhostname.net.csr. When you open it with text editor, It will look like follows. This is the one you need to provide to the Certificate Authority (CA) to get it signed.

[1] http://wso2.com/library/knowledge-base/2011/08/adding-ca-certificate-authority-signed-certificate-wso2-products/
[2] http://keystore-explorer.org/

WSO2 Server Startup Taking a lot of time on Mac ??? Solved...

With MacOS Sierra, I was experiencing a huge delay in server start ups for WSO2 latest versions. They were like follows.

Server		Version		Java Version		Startup Time
WSO2 ESB		4.8.1		1.7.0_80		15 Seconds
WSO2 ESB		5.0.0		1.7.0_80		90 Seconds
WSO2 ESB		5.0.0		1.8.0_101		89 Seconds
API Manager		1.7.0		1.7.0_80		17 seconds
API Manager		2.0.0		1.7.0_80		166 seconds
API Manager		2.0.0		1.8.0_101		167 seconds

My Processing power was as bellow.

I was in really doubt, Why it took so much of time to start the server.  When researching on that i could locate following discussion [1]. It was really interesting , you can go through it and understand it.

The solution i did as int he above blog post, i added a mapping to the canonical 127.0.0.1 address of my macbook hostname to my /etc/hosts file as bellow.



Once i done that, My ESB 5.0.0 server startup was 13 seconds..  So it reduced from 90 --> 13 seconds... Amazing haa... :D

[1] https://thoeni.io/post/macos-sierra-java/

How to get all the default claims when using JWT - WSO2 API Manager

There are situations like we need to pass the enduser's attributes to the backend services when using WSO2 API Manager.  We can use Java Web Tokens (JWT) for that.

You can find the documentation for this in WSO2 site [1]

Here I am going to discuss on how we can get all default claims for JWT token since by just enabling the configuration EnableJWTGeneration it will not give you all claims. 

If you just enable above , the configuration will look like follows. 

   <JWTConfiguration>  
     <!-- Enable/Disable JWT generation. Default is false. -->  
     <EnableJWTGeneration>true</EnableJWTGeneration>  
     <!-- Name of the security context header to be added to the validated requests. -->  
     <JWTHeader>X-JWT-Assertion</JWTHeader>  
     <!-- Fully qualified name of the class that will retrieve additional user claims  
        to be appended to the JWT. If not specified no claims will be appended.If user wants to add all user claims in the  
        jwt token, he needs to enable this parameter.  
        The DefaultClaimsRetriever class adds user claims from the default carbon user store. -->  
     <!--ClaimsRetrieverImplClass>org.wso2.carbon.apimgt.impl.token.DefaultClaimsRetriever</ClaimsRetrieverImplClass-->  
     <!-- The dialectURI under which the claimURIs that need to be appended to the  
        JWT are defined. Not used with custom ClaimsRetriever implementations. The  
        same value is used in the keys for appending the default properties to the  
        JWT. -->  
     <!--ConsumerDialectURI>http://wso2.org/claims</ConsumerDialectURI-->  
     <!-- Signature algorithm. Accepts "SHA256withRSA" or "NONE". To disable signing explicitly specify "NONE". -->  
     <!--SignatureAlgorithm>SHA256withRSA</SignatureAlgorithm-->  
     <!-- This parameter specifies which implementation should be used for generating the Token. JWTGenerator is the  
         default implementation provided. -->  
     <JWTGeneratorImpl>org.wso2.carbon.apimgt.keymgt.token.JWTGenerator</JWTGeneratorImpl>  
     <!-- This parameter specifies which implementation should be used for generating the Token. For URL safe JWT  
        Token generation the implementation is provided in URLSafeJWTGenerator -->  
     <!--<JWTGeneratorImpl>org.wso2.carbon.apimgt.keymgt.token.URLSafeJWTGenerator</JWTGeneratorImpl>-->  
     <!-- Remove UserName from JWT Token -->  
     <!-- <RemoveUserNameFromJWTForApplicationToken>true</RemoveUserNameFromJWTForApplicationToken>-->  
   </JWTConfiguration>  

Then, By enabling wire logs[2], We can get the encrypted JWT Token as bellow when you invoke an API.

When we decode it, It will look like follows.

You can notice that, It is not showing the role claim. Basically, If you need to have all the default claims passed in this JWT token, You need to enable following two configurations in api-manager.xml

  <ClaimsRetrieverImplClass>org.wso2.carbon.apimgt.impl.token.DefaultClaimsRetriever</ClaimsRetrieverImplClass>  

 <ConsumerDialectURI>http://wso2.org/claims</ConsumerDialectURI>  

Once you enable them and restart the server, You will get the all the default claims in the token as bellow.

[1] https://docs.wso2.com/display/AM210/Passing+Enduser+Attributes+to+the+Backend+Using+JWT

[2] http://mytecheye.blogspot.com/2013/09/wso2-esb-all-about-wire-logs.html