We can use VFS transport in WSO2 ESB as described in the sample [1]. Apart from that Supun which was a project manger of WSO2 ESB has written and article[2] on that. These articles and samples provides a great help on using VFS transport to transfer files.
But when using that configurations for transferring large files (Greater that 500 MB), i got an Out Of Memory exceptions[3] and some other errors[4].
When looking in to the problem more deeply, i could found the solution for the problem. The issue was with the message builder class that i have used and missing a property. We need to use the message builder "org.apache.axis2.format.BinaryBuilder" for this. Apart from that we need to include the property "ClientApiNonBlocking" in the proxy configuration.
As a sample i have defined a custom content type "chs/binary".
Entries in axis2.xml
...
class="org.wso2.carbon.relay.ExpandingMessageFormatter"/>
...
</messageFormatters>
<messageBuilders>
...
<messageBuilder contentType="chs/binary"
class="org.apache.axis2.format.BinaryBuilder"/>
...
</messageBuilders>
Then we need to have a sample VFS proxy as :
<proxy xmlns="http://ws.apache.org/ns/synapse"
name="FileProxy"
transports="vfs"
startOnLoad="true"
trace="disable">
<description/>
<target>
<inSequence>
<log level="custom">
<property name="FileProxy" value="Processing file"/>
</log>
<property name="OUT_ONLY" value="true"/>
<property name="ClientApiNonBlocking"
value="true"
scope="axis2"
action="remove"/>
<send>
<endpoint name="FileEpr">
<address uri="vfs:file:////home/shammi/file-out"/>
</endpoint>
</send>
</inSequence>
</target>
<parameter name="transport.vfs.Streaming">true </parameter>
<parameter name="transport.PollInterval">15 </parameter>
<parameter name="transport.vfs.ActionAfterProcess">MOVE </parameter>
<parameter name="transport.vfs.FileURI">file:///home/shammi/file-in </parameter>
<parameter name="transport.vfs.MoveAfterProcess">file:///home/shammi/file-original </parameter>
<parameter name="transport.vfs.MoveAfterFailure">file:////home/shammi/file-failure </parameter>
<parameter name="transport.vfs.Locking">enable </parameter>
<parameter name="transport.vfs.FileNamePattern">.*.zip|.*.test </parameter>
<parameter name="transport.vfs.ContentType">chs/binary </parameter>
<parameter name="transport.vfs.ActionAfterFailure">MOVE </parameter>
</proxy>
With this proxy , i could use VFS transport to transfer a file with the size 1 GB with out any issue. I have made the points bold which needs to pay more attention..
Cheers.. Thats it......
[1]http://docs.wso2.org/wiki/pages/viewpage.action?pageId=15471427
[2]http://wso2.org/library/articles/2011/01/wso2-esb-example-file-processing
[3]2012-12-19 11:49:04,797 [-] [Framework Event Dispatcher] WARN PollTableEntry transport.vfs.FileURI parameter is missing in the proxy service configuration
2012-12-19 11:50:08,896 [-] [vfs-Worker-3] ERROR NativeWorkerPool Uncaught exception
java.lang.OutOfMemoryError: Java heap space
at org.apache.commons.io.output.ByteArrayOutputStream.needNewBuffer(ByteArrayOutputStream.java:124)
at org.apache.commons.io.output.ByteArrayOutputStream.write(ByteArrayOutputStream.java:155)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1263)
at org.apache.commons.io.IOUtils.copy(IOUtils.java:1236)
at org.apache.commons.io.IOUtils.toByteArray(IOUtils.java:360)
at org.apache.axis2.format.BinaryBuilder.processDocument(BinaryBuilder.java:72)
at org.apache.synapse.transport.vfs.VFSTransportListener.processFile(VFSTransportListener.java:558)
at org.apache.synapse.transport.vfs.VFSTransportListener.scanFileOrDirectory(VFSTransportListener.java:312)
at org.apache.synapse.transport.vfs.VFSTransportListener.poll(VFSTransportListener.java:158)
at org.apache.synapse.transport.vfs.VFSTransportListener.poll(VFSTransportListener.java:107)
at org.apache.axis2.transport.base.AbstractPollingTransportListener$1$1.run(AbstractPollingTransportListener.java:67)
at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
[4]
2012-12-19 13:59:58,033 [-] [Axis2 Task] ERROR VFSTransportSender IO Error while creating response file : file:///home/shammi/wso2/foo/file-out/response.xml
org.apache.axis2.AxisFault: Error serializing binary content of element : {http://ws.apache.org/commons/ns/payload}binary
at org.apache.axis2.format.BinaryFormatter.writeTo(BinaryFormatter.java:66)
at org.apache.synapse.transport.vfs.VFSTransportSender.populateResponseFile(VFSTransportSender.java:235)
at org.apache.synapse.transport.vfs.VFSTransportSender.sendMessage(VFSTransportSender.java:173)
at org.apache.axis2.transport.base.AbstractTransportSender.invoke(AbstractTransportSender.java:112)
at org.apache.axis2.engine.AxisEngine$TransportNonBlockingInvocationWorker.run(AxisEngine.java:627)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.commons.vfs2.FileNotFoundException: Could not read from "file:///home/shammi/wso2/foo/file-in/a.zip" because it is a not a file.
at org.apache.commons.vfs2.provider.AbstractFileObject.getInputStream(AbstractFileObject.java:1316)
at org.apache.commons.vfs2.provider.DefaultFileContent.getInputStream(DefaultFileContent.java:397)
at org.apache.synapse.transport.vfs.FileObjectDataSource.getInputStream(FileObjectDataSource.java:61)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.axis2.format.ManagedDataSourceFactory$DataSourceManager.invoke(ManagedDataSourceFactory.java:91)
at $Proxy15.getInputStream(Unknown Source)
at javax.activation.DataHandler.writeTo(DataHandler.java:290)
at org.apache.axis2.format.BinaryFormatter.writeTo(BinaryFormatter.java:64)
... 7 more
Caused by: java.io.FileNotFoundException: /home/shammi/wso2/foo/file-in/a.zip (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.
at org.apache.commons.vfs2.provider.local.LocalFile.doGetInputStream(LocalFile.java:210)
at org.apache.commons.vfs2.provider.AbstractFileObject.getInputStream(AbstractFileObject.java:1308)
... 17 more
2012-12-19 14:00:26,458 [-] [Timer-7] WARN TimeoutHa
Helpful article. Did you test files above 1Gb? Any downside/drawback to making your configuration file properties the default option? Could the ESB recognized the filesize (from the http header) and fail gracefully or auto-adjust the processing behavior?
ReplyDeleteWalk by any table game or poker room in any casino and you'll hear gambling jargon, which sounds like a foreign language to new and non-gamblers alike. Jargon at the craps table such as, "Put five on the big red! and Gimmie the Yo"! or "Double after split" while watching blackjack. Maybe during your stroll through the poker room, a player shouts to his opponent, "you're drawing dead on the river." This is gambling vocabulary. If you can't speak the language, you can't do the deeds. tctippers.com.au
ReplyDeleteCorporate travel management is crucial for any corporation whether big or small. Keeping track of expenses, obtaining cheap airfares and hotel stays as well as keeping the expense accounting up to par is not a luxury, it is critical for staying competitive and running an efficient business. An established firm has the experience and expertise that will make your corporate travel organization a walk in the park. You would be surprised at how much time and money you can save on your corporate travel operations by allowing a professional to tweak, track, and trim a few things here and there. go to blog article
ReplyDeleteCorporate travel management is crucial for any corporation whether big or small. Keeping track of expenses, obtaining cheap airfares and hotel stays as well as keeping the expense accounting up to par is not a luxury, it is critical for staying competitive and running an efficient business. An established firm has the experience and expertise that will make your corporate travel organization a walk in the park. You would be surprised at how much time and money you can save on your corporate travel operations by allowing a professional to tweak, track, and trim a few things here and there. amsterdam food tours
ReplyDeleteIn today's economic status, travelling is so expensive. It requires not only a long period of preparation, but as well as a big amount of money. 부달
ReplyDeleteIn today's economic status, travelling is so expensive. It requires not only a long period of preparation, but as well as a big amount of money. 아이러브밤
ReplyDeleteThere is an abundance of benefits to staying in a bed and breakfast. Usually built from charming old houses,bed and breakfasts possess an atmosphere intended to make guests feel like they are truly staying in a home. vacation packages to montreal
ReplyDeleteIn today's economic status, travelling is so expensive. It requires not only a long period of preparation, but as well as a big amount of money. 야맵
ReplyDeleteMedical travel can be a lucrative, fun, exciting, and fulfilling career...IF you know how to navigate the system and access the highest salary, most desirable locations, comfortable travel plans, nicest housing accommodations, and the best job scenarios. 야맵
ReplyDeleteIf you know that business travel is not without its risk and the potential for crisis, then you need to read this article. In this article we are going to talk about the management and containment of crisis as it relates to travelers and travel managers. The objective of this article is to share with you the collective knowledge on managing crisis and significantly improve your ability to identify and manage a crisis but also improve your business travel efficiency. 야맵
ReplyDeletehey you should get social media plugin. was looking for the ‘like’ button but couldn’t find it. 토토사이트
ReplyDeleteA powerful share, I just given this onto a colleague who was doing slightly evaluation on this. And he in fact bought me breakfast because I found it for him.. smile. So let me reword that: Thnx for the treat! However yeah Thnkx for spending the time to debate this, I feel strongly about it and love studying more on this topic. If possible, as you change into experience, would you mind updating your weblog with extra details? It’s highly helpful for me. Massive thumb up for this weblog put up! 출장안마
ReplyDeleteI like you blog (désolé, je suis francais, je parle mal anglais) ufayou
ReplyDeleteThe video card (often referred to as the GPU) is one more significant element in any gaming rig, as it handles just about all of the graphics for your online games. A person of the difficulties with movie cards – especially the middle and low conclusion ones – is that they tend to turn out to be obsolete quicker than the other elements of a gaming computer system. Usually, getting a substantial stop video card when you’re upgrading or constructing your gaming rig is vital as it gives you breathing area in advance of it is time to upgrade once more. wedding love hearts
ReplyDeleteAfter study many of the websites on the web site now, i really such as your way of blogging. I bookmarked it to my bookmark internet site list and you will be checking back soon. Pls look at my internet site as well and make me aware what you consider. software sales
ReplyDeleteExcellent blog right here! Additionally your site loads up fast! What web host are you using? Can I get your associate link in your host? I desire my site loaded up as quickly as yours lol is vivo chinese company
ReplyDeleteIntimately, the post is actually the freshest on that laudable topic. I match in with your conclusions and also definitely will thirstily look forward to your approaching updates. Just saying thanks can not simply be adequate, for the outstanding lucidity in your writing. I can immediately grab your rss feed to stay informed of any kind of updates. Solid work and much success in your business enterprize! 먹튀검증
ReplyDeleteHello there, just became alert to your blog through Google, and found that it’s truly informative. I am going to watch out for brussels. I’ll appreciate if you continue this in future. Lots of people will be benefited from your writing. Cheers! crystal soil water beads
ReplyDeleteHello there, just became alert to your blog through Google, and found that it’s truly informative. I am going to watch out for brussels. I’ll appreciate if you continue this in future. Lots of people will be benefited from your writing. Cheers! hospital wheelchair manufacturers
ReplyDeletekinda enjoyed the post you wrote . it just isnt that easy to discover even remotely good posts toactually read (you know.. READ! and not simply browsing through it like some uniterested and flesh eating zombie before going to yet another post to just ignore), so cheers man for not wasting my time! highlighter wholesaler
ReplyDeleteGreat post, I was doing a google search and your site came up for bank owned homes in Oviedo, FL but anyway, I have enjoyed reading it, keep it up! Empire city for rent
ReplyDeleteI image this could be various upon the written content? having said that I nonetheless consider that it can be appropriate for virtually any type of subject material, as a result of it will continuously be enjoyable to decide a warm and delightful face or possibly pay attention a voice while preliminary landing. INDIA MEDICAL VISA
ReplyDeleteNice article thank you for this. Check the Kenya visa requirements before you apply for the Kenya visa on arrival through online visa application. The e visa application offers the fast visa services. Thank you
ReplyDeleteWow.. Very informative article thanks for sharing please keep it up.. You can apply for an India tourist visa urgent in some case of emergency and you can also apply for an India visa super urgent, it takes less as compared to sticker visa.
ReplyDelete