Category: ESB

How to read dynamic query parameter in ESB/APIM synapse.

Following way you will be able to get the dynamic parameters
For e.g
1. My api has the following url  http://xxxx8243/uridynamic/1
2. and i have added a GET resource(uri-template) to api  /add?*
that means there can be zero or many query parameters.
3. Now my get request takes following format
4. in the sequence you can read the value of ‘a‘ and ‘b‘ as using $url
if a and b is not present in the url value of SYMBOL and value of SYMBOL2 will have
null value.
<log level=”custom”>
            <property name=”SYMBOL” expression=”$url:a“></property>
            <property name=”SYMBOL2″ expression=”$url:c“></property>
 </log>

Wso2 ESB/APIM add query parameter to endpoint

Lets say there is a http endpoint http://localhost:8080/restapi defined in a api and want to add query parameter in sequence level before invoking the endpoint.

i.e   http://localhost:8080/restapi?id=2&name=admin&role=admin

By using Property “REST_URL_POSTFIX” this can be achieved as follow.


<?xml version="1.0" encoding="UTF-8"?>
<sequence
 xmlns="http://ws.apache.org/ns/synapse" name="admin--test:v1--In">
 <log level="full">
 <property name="IN_MESSAGE" value="IN_MESSAGE"/>
 </log>
 <property name="id" value="2"/>
 <property name="name" value="admin"/>
 <property name="role" value="admin"/>
 <property name="REST_URL_POSTFIX" expression="fn:concat('?id=',$ctx:id,'&amp;name=',$ctx:name,'&amp;role=',$ctx:role)" scope="axis2"/>
 <endpoint
 xmlns="http://ws.apache.org/ns/synapse" name="HTTPEndpoint">
 <http uri-template="http://localhost:8080/restapi" method="GET"></http>
 </endpoint>
</sequence>

How to do a POST/PUT request with no body in WSO2 ESB?

ESB message flow will go through builders and formatters when ESB is configured to use NHTTP transport or pass-thru transport with content aware mediators used[1]. In both cases ESB will build the message body. So when come to POST/PUT request by default ESB expects a message body to be present in the request to send to the backend. So when there is POST/PUT request without body request will hang in the server.

This default behavior can be changed by setting the property

<property name="FORCE_POST_PUT_NOBODY" value="true" scope="axis2" type="BOOLEAN" />

Once this property is set ESB can send the request to back end even though there is no body with request. If pass-thru transport is used and there is no content aware mediators used in proxy/api message flow will not hit the builders and formatters. That means ESB will not deal with message body , it will pass what ever come to the server as it is to back-end. So in that case no need to set this special property.

  1. POST request no (body passthru transport and content aware mediator(log full) is used )- request hangs
[2015-09-05 17:45:40,075] DEBUG - wire >> "POST /test/1/preq HTTP/1.1[\r][\n]"
[2015-09-05 17:45:40,076] DEBUG - wire >> "Host: localhost:8280[\r][\n]"
[2015-09-05 17:45:40,076] DEBUG - wire >> "Connection: keep-alive[\r][\n]"
[2015-09-05 17:45:40,076] DEBUG - wire >> "Content-Length: 0[\r][\n]"
[2015-09-05 17:45:40,076] DEBUG - wire >> "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/41.0.2272.76 Chrome/41.0.2272.76 Safari/537.36[\r][\n]"
[2015-09-05 17:45:40,076] DEBUG - wire >> "Cache-Control: no-cache[\r][\n]"
[2015-09-05 17:45:40,076] DEBUG - wire >> "Origin: chrome-extension://fdmmgilgnpjigdojojpjoooidkmcomcm[\r][\n]"
[2015-09-05 17:45:40,076] DEBUG - wire >> "Authorization: Bearer 84d2c7f333f93c27952ec33be2c0e5b5[\r][\n]"
[2015-09-05 17:45:40,076] DEBUG - wire >> "Accept: */*[\r][\n]"
[2015-09-05 17:45:40,076] DEBUG - wire >> "Accept-Encoding: gzip, deflate[\r][\n]"
[2015-09-05 17:45:40,076] DEBUG - wire >> "Accept-Language: en-US,en;q=0.8[\r][\n]"
[2015-09-05 17:45:40,076] DEBUG - wire >> "Cookie: region4_monitor_menu=visible; region1_identity_entitlement_menu=visible; region3_metadata_menu=visible; region1_manage_menu=visible; region3_registry_menu=visible; menuPanel=visible; menuPanelType=main; i18next=en-US[\r][\n]"
[2015-09-05 17:45:40,076] DEBUG - wire >> "[\r][\n]"
[2015-09-05 17:45:40,083]  INFO - LogMediator To: /test/1/preq, MessageID: urn:uuid:e00a9af5-d0ec-4c22-9019-8cdae17ac3cb, Direction: request, IN_MESSAGE = IN_MESSAGE, Envelope: 
[2015-09-05 17:46:40,144]  WARN - SourceHandler Connection time out after request is read: http-incoming-1
[2015-09-05 17:46:40,145] DEBUG - wire >> "POST /test/1/preq HTTP/1.1[\r][\n]"
[2015-09-05 17:46:40,146] DEBUG - wire >> "Host: localhost:8280[\r][\n]"
[2015-09-05 17:46:40,146] DEBUG - wire >> "Connection: keep-alive[\r][\n]"
[2015-09-05 17:46:40,146] DEBUG - wire >> "Content-Length: 0[\r][\n]"
[2015-09-05 17:46:40,146] DEBUG - wire >> "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/41.0.2272.76 Chrome/41.0.2272.76 Safari/537.36[\r][\n]"
[2015-09-05 17:46:40,146] DEBUG - wire >> "Cache-Control: no-cache[\r][\n]"
[2015-09-05 17:46:40,146] DEBUG - wire >> "Origin: chrome-extension://fdmmgilgnpjigdojojpjoooidkmcomcm[\r][\n]"
[2015-09-05 17:46:40,146] DEBUG - wire >> "Authorization: Bearer 84d2c7f333f93c27952ec33be2c0e5b5[\r][\n]"
[2015-09-05 17:46:40,146] DEBUG - wire >> "Accept: */*[\r][\n]"
[2015-09-05 17:46:40,147] DEBUG - wire >> "Accept-Encoding: gzip, deflate[\r][\n]"
[2015-09-05 17:46:40,147] DEBUG - wire >> "Accept-Language: en-US,en;q=0.8[\r][\n]"
[2015-09-05 17:46:40,147] DEBUG - wire >> "Cookie: region4_monitor_menu=visible; region1_identity_entitlement_menu=visible; region3_metadata_menu=visible; region1_manage_menu=visible; region3_registry_menu=visible; menuPanel=visible; menuPanelType=main; i18next=en-US[\r][\n]"
[2015-09-05 17:46:40,147] DEBUG - wire >> "[\r][\n]"
[2015-09-05 17:46:40,150]  INFO - LogMediator To: /test/1/preq, MessageID: urn:uuid:b203f072-b044-41fb-9a96-f8d56cfc0aee, Direction: request, IN_MESSAGE = IN_MESSAGE, Envelope: 
[2015-09-05 17:47:40,213]  WARN - SourceHandler Connection time out after request is read: http-incoming-2
[2015-09-05 17:47:43,509]  WARN - TimeoutHandler Expiring message ID : urn:uuid:24e8733b-3184-4bae-8831-43e1d8366d68; dropping message after global timeout of : 120 seconds
[2015-09-05 17:48:43,510]  WARN - TimeoutHandler Expiring message ID : urn:uuid:4037382a-c498-4377-b599-182095c662f4; dropping message after global timeout of : 120 seconds

2. POST request no body  “FORCE_POST_PUT_NOBODY” is set(passthru transport and content aware mediator(log full) is used )- request doesn’t hang

[2015-08-28 17:21:51,776] DEBUG - wire >> "POST /test/1/preq HTTP/1.1[\r][\n]"
[2015-08-28 17:21:51,776] DEBUG - wire >> "Host: xxxxx:8280[\r][\n]"
[2015-08-28 17:21:51,776] DEBUG - wire >> "Connection: keep-alive[\r][\n]"
[2015-08-28 17:21:51,776] DEBUG - wire >> "Content-Length: 0[\r][\n]"
[2015-08-28 17:21:51,776] DEBUG - wire >> "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/41.0.2272.76 Chrome/41.0.2272.76 Safari/537.36[\r][\n]"
[2015-08-28 17:21:51,776] DEBUG - wire >> "Cache-Control: no-cache[\r][\n]"
[2015-08-28 17:21:51,776] DEBUG - wire >> "Origin: chrome-extension://fdmmgilgnpjigdojojpjoooidkmcomcm[\r][\n]"
[2015-08-28 17:21:51,776] DEBUG - wire >> "Authorization: Bearer 1e519d8be6ab6bd12bf3a9378bba646[\r][\n]"
[2015-08-28 17:21:51,776] DEBUG - wire >> "Accept: */*[\r][\n]"
[2015-08-28 17:21:51,776] DEBUG - wire >> "Accept-Encoding: gzip, deflate[\r][\n]"
[2015-08-28 17:21:51,776] DEBUG - wire >> "Accept-Language: en-US,en;q=0.8[\r][\n]"
[2015-08-28 17:21:51,776] DEBUG - wire >> "[\r][\n]"
[2015-08-28 17:21:51,778]  INFO - LogMediator IN_MESSAGE = IN_MESSAGE
[2015-08-28 17:21:51,779]  INFO - LogMediator To: /test/1/preq, MessageID: urn:uuid:05b53455-6ed1-47f1-aee3-9808fe694328, Direction: request, Envelope: <?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope"><soapenv:Body><mediate/></soapenv:Body></soapenv:Envelope>
[2015-08-28 17:21:51,781] DEBUG - wire << "POST /post/preq HTTP/1.1[\r][\n]"
[2015-08-28 17:21:51,781] DEBUG - wire << "Accept-Language: en-US,en;q=0.8[\r][\n]"
[2015-08-28 17:21:51,781] DEBUG - wire << "Accept-Encoding: gzip, deflate[\r][\n]"
[2015-08-28 17:21:51,781] DEBUG - wire << "Origin: chrome-extension://fdmmgilgnpjigdojojpjoooidkmcomcm[\r][\n]"
[2015-08-28 17:21:51,781] DEBUG - wire << "Content-Type: application/x-www-form-urlencoded; charset=UTF-8[\r][\n]"
[2015-08-28 17:21:51,781] DEBUG - wire << "Accept: */*[\r][\n]"
[2015-08-28 17:21:51,781] DEBUG - wire << "Cache-Control: no-cache[\r][\n]"
[2015-08-28 17:21:51,781] DEBUG - wire << "Transfer-Encoding: chunked[\r][\n]"
[2015-08-28 17:21:51,781] DEBUG - wire << "Host: localhost:8080[\r][\n]"
[2015-08-28 17:21:51,781] DEBUG - wire << "Connection: Keep-Alive[\r][\n]"
[2015-08-28 17:21:51,781] DEBUG - wire << "User-Agent: Synapse-PT-HttpComponents-NIO[\r][\n]"
[2015-08-28 17:21:51,781] DEBUG - wire << "[\r][\n]"
[2015-08-28 17:21:51,781] DEBUG - wire << "0[\r][\n]"
[2015-08-28 17:21:51,781] DEBUG - wire << "[\r][\n]"
[2015-08-28 17:21:51,782] DEBUG - wire >> "HTTP/1.1 200 OK[\r][\n]"
[2015-08-28 17:21:51,782] DEBUG - wire >> "Content-Type: application/xml[\r][\n]"
[2015-08-28 17:21:51,782] DEBUG - wire >> "Content-Encoding: gzip[\r][\n]"
[2015-08-28 17:21:51,782] DEBUG - wire >> "Transfer-Encoding: chunked[\r][\n]"
[2015-08-28 17:21:51,782] DEBUG - wire >> "Server: Jetty(6.1.26)[\r][\n]"
[2015-08-28 17:21:51,783] DEBUG - wire >> "[\r][\n]"
[2015-08-28 17:21:51,783] DEBUG - wire >> "26[\r][\n]"
[2015-08-28 17:21:51,783] DEBUG - wire >> "[0x1f][0x8b][0x8][0x0][0x0][0x0][0x0][0x0][0x0][0x0][0xb3]).I,)-[0xb6]+.MNN-.[0xb6][0xd1][0x87][0xf2][0x1][0xea][0x1a]q[0x6][0x18][0x0][0x0][0x0][\r][\n]"
[2015-08-28 17:21:51,783] DEBUG - wire >> "0[\r][\n]"
[2015-08-28 17:21:51,783] DEBUG - wire >> "[\r][\n]"
[2015-08-28 17:21:51,786] DEBUG - wire << "HTTP/1.1 200 OK[\r][\n]"
[2015-08-28 17:21:51,786] DEBUG - wire << "Access-Control-Allow-Headers: authorization,Access-Control-Allow-Origin,Content-Type[\r][\n]"
[2015-08-28 17:21:51,786] DEBUG - wire << "Access-Control-Allow-Origin: *[\r][\n]"
[2015-08-28 17:21:51,786] DEBUG - wire << "Content-Encoding: gzip[\r][\n]"
[2015-08-28 17:21:51,787] DEBUG - wire << "Access-Control-Allow-Methods: POST[\r][\n]"
[2015-08-28 17:21:51,787] DEBUG - wire << "Content-Type: application/xml[\r][\n]"
[2015-08-28 17:21:51,787] DEBUG - wire << "Date: Fri, 28 Aug 2015 11:51:51 GMT[\r][\n]"
[2015-08-28 17:21:51,787] DEBUG - wire << "Server: WSO2-PassThrough-HTTP[\r][\n]"
[2015-08-28 17:21:51,787] DEBUG - wire << "Transfer-Encoding: chunked[\r][\n]"
[2015-08-28 17:21:51,787] DEBUG - wire << "Connection: keep-alive[\r][\n]"
[2015-08-28 17:21:51,787] DEBUG - wire << "[\r][\n]"
[2015-08-28 17:21:51,787] DEBUG - wire << "26[\r][\n]"
[2015-08-28 17:21:51,787] DEBUG - wire << "[0x1f][0x8b][0x8][0x0][0x0][0x0][0x0][0x0][0x0][0x0][0xb3]).I,)-[0xb6]+.MNN-.[0xb6][0xd1][0x87][0xf2][0x1][0xea][0x1a]q[0x6][0x18][0x0][0x0][0x0][\r][\n]"
[2015-08-28 17:21:51,787] DEBUG - wire << "0[\r][\n]"
[2015-08-28 17:21:51,788] DEBUG - wire << "[\r][\n]"

But when <property name=”FORCE_POST_PUT_NOBODY” value=”true” scope=”axis2″ type=”BOOLEAN”/>” is set in the synapse config, ESB will set the default content type (application/x-www-form-urlencoded) and do the post/put with no body. We cannot remove the content type completely but we can change the value of it using property.

e.g <property name=”Content-Type” value=”text/plain” scope=”transport”/> will change change the content type to text/plain

  1. http://wso2.com/library/articles/2013/12/demystifying-wso2-esb-pass-thru-transport-part-ii/

Configure WSO2 ESB with reverse proxy (with proxy context path)

Update /etc/hosts with following and update IP address according to your environment.

127.0.0.1       esb.example.com
127.0.0.1       nginx.example.com

update carbon.xml

<HostName>esb.example.com</HostName>
<MgtHostName>esb.example.com</MgtHostName>

Install Nginx
sudo apt-get install nginx

Create ssl certificates and copy then to ssl folder(crealte ssl folder if not exist in /etc/nginx).
sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/nginx/ssl/nginx.key -out /etc/nginx/ssl/nginx.crt

Edit nginx configurations
sudo vi /etc/nginx/sites-enabled/default

Sample Configuration:

server {

listen 443;
server_name nginx.example.com;
ssl on;
ssl_certificate /etc/nginx/ssl/nginx.crt;
ssl_certificate_key /etc/nginx/ssl/nginx.key;

location  /ca-esb-console/  {
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass https://esb.example.com:9443/;
proxy_redirect https://esb.example.com:9443/ https://nginx.example.com/ca-esb-console/;

}
}

server_name : nginx address (https://nginx.example.com/)

proxy_pass :

Sets the protocol and address of a proxied server and an optional URI to which a location should be mapped. As a protocol, “http” or “https” can be specified. The address can be specified as a domain name or IP address, and an optional port[1]

in our case proxies server is ESB -https://esb.example.com:9443/ (host name is changed in carbon.xml as esb.example.com)

proxy_redirect :

Sets the text that should be changed in the “Location” and “Refresh” header fields of a proxied server response. Suppose a proxied server returned the header field “Location: http://localhost:8000/two/some/uri/”. The directive

proxy_redirect http://localhost:8000/two/ http://frontend/one/;

will rewrite this string to “Location: http://frontend/one/some/uri/”.[1]

here /ca-esb-console/ is proxy context : https://nginx.example.com/ /ca-esb-console/carbon will return the login page

One main feature of Nginx, is known as directives, in Nginx configuration, directives specified in a higher block will will filter down to lower blocks within the configurations  as a default value. Nginx comes with 3 blocks, the http block, the server-block and the location block.[2]

A location block enables to match a query and process the request that match the specific location block. For an example, location  /ca-esb-console/ { } matches any query that begins with /ca-esb-console

nginx commands

start -sudo /etc/init.d/nginx start
stop -sudo /etc/init.d/nginx stop
restart -sudo /etc/init.d/nginx restart
reload -sudo /etc/init.d/nginx reload  : any changes to default conf can be applied without shutting down the server

To check whether server is up and running -sudo /etc/init.d/nginx status

1 .http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_redirect
2 .http://shavanthaw.blogspot.com/2014/06/configuring-nginx-for-wso2-carbon.html3.http://sanjeewamalalgoda.blogspot.com/2014/12/configure-wso2-api-manager-with-reverse.html
4.http://udarakr.blogspot.com/2014/05/fronting-wso2-management-console-ui.html

WSO2 ESB – JSON PATH

Logging content from JSON payload using json path

If Payload is a json object as follow

{“symbol”:”WSO2″, “ID”:”StockQuote”}

1 .

<log level=”custom”>
<property name=”jsonlog” expression=”json-eval($.)”/>
</log>

This will log the entire json object – {“symbol”:”WSO2″, “ID”:”StockQuote”}

 

2 .

<log level=”custom”>
<property name=”jsonlog” expression=”json-eval($.symbol)”/>
</log>

 

This will log the symbol -wso2

 

reference

1 .https://docs.wso2.com/display/ESB480/JSON+Support

WSO2 ESB – Wrap XML tags around JSON return

ESB receive request in the format of JSON as follow

{“symbol”:”WSO2″, “ID”:”StockQuote”}

 

But request to the back end service should be in following format

<?xml version=”1.0″ encoding=”utf-8″?>
<soapenv:Envelope xmlns:soapenv=”http://schemas.xmlsoap.org/soap/envelope/”><soapenv:Body&gt;
<ns:xmlwrap xmlns:ns=”http://services.samples”&gt;{“symbol”:”WSO2″, “ID”:”StockQuote”}</ns:xmlwrap>
</soapenv:Body>
</soapenv:Envelope>

 

read the json using json path and use payloadFactory mediator to construct the desired request

<?xml version=”1.0″ encoding=”UTF-8″?>
<proxy xmlns=”http://ws.apache.org/ns/synapse&#8221;
name=”JSONProxy”
transports=”https,http”
statistics=”disable”
trace=”disable”
startOnLoad=”true”>
<target>
<inSequence>
<property name=”JSON_PAYLOAD” expression=”json-eval($.)”/>
<log level=”custom”>
<property name=”jsonlog” expression=”$ctx:JSON_PAYLOAD”/>
</log>
<payloadFactory media-type=”xml”>
<format>
<ns:xmlwrap xmlns:ns=”http://services.samples”>$1</ns:xmlwrap&gt;
</format>
<args>
<arg evaluator=”xml” expression=”get-property(‘JSON_PAYLOAD’)”/>
</args>
</payloadFactory>
<log level=”full”/>
<header name=”Action” value=”urn:getQuote”/>
</inSequence>
<outSequence>
<send/>
</outSequence>
<endpoint>
<address uri=”http://localhost:9001/services/SimpleStockQuoteService&#8221;
format=”soap11″/>
</endpoint>
</target>
<description/>
</proxy>

 

output

<?xml version=”1.0″ encoding=”utf-8″?><soapenv:Envelope xmlns:soapenv=”http://schemas.xmlsoap.org/soap/envelope/”&gt;
<soapenv:Body><ns:xmlwrap xmlns:ns=”http://services.samples”&gt;{“symbol”:”WSO2″, “ID”:”StockQuote”}
</ns:xmlwrap>
</soapenv:Body>
</soapenv:Envelope>

WSO2 ESB – Add Custom level log

By setting the log level to “custom” , we can add custom log in ESB proxy .

1. Log a custom String value (e.g Hello World)

<log level=”custom”>
<property name=”MyCustomLog” value=”Hello World”/>
</log>

 

2. Log a property value using custom log level. use expression attribute to read the value of property.

<property name=”Symbol” value=”wso2″/>

———————————————————————————

<log level=”custom”>
<property name=”customlog” expression=”$ctx:Symbol”/>
</log>

OR

<log level=”custom”>
<property name=”customlog” expression=”get-property(‘Symbol’)/>
</log>