Asynchronous transport for permanent connections

View: New views
17 Messages — Rating Filter:   Alert me  

Asynchronous transport for permanent connections

by Marco D'Alia :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi,
I've made a proof of concept of an asynchronous transport for permanent
socket connections as a final thesis for my bachelor degree.

The main objective was to manage a large number of simultaneous
connections using SEDA, especially with  protocols that need to keep the
connections open between requests.
For this reason I've used non-blocking sockets (through Mina library):
so while a request is processed there is no thread waiting for a
response, greatly lowering the number of threads required.
Looking for a SEDA implementation the choice fell on the excellent Mule ESB.

The code is composed of some general classes for implementing transports
with nio and two sample implementations: a sender/receiver tcp transport
and a receiver-only http transport.

This can be useful not only for long lasting/permanent connections, but
also to send large quantities of data, streaming and push protocols.

Since it is based on Mina it can also be useful to integrate Mina based
protocols with Mule.

I've arranged a trac repository with the code and a better description
of how it works: http://minamule.madarco.net/trac

I'd be glad to hear some feedback from the Mule developer community :)

Marco D'Alia



---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email


Re: Asynchronous transport for permanent connections

by Andrew Perepelytsya :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Marco,

Let me kindly forward you to http://mule.mulesource.org/jira/browse/MULE-64

Some legal work - I see this disclaimer on the page:

This program is distrubuted as-is bla bla bla ecc.. and you can do whatever you want with it (Public Domain)

Are you willing to give up the copyright in order for the code to eventually be incorporated into the Mule codebase? IMNAL, but keep in mind your reply may mean a legal agreement.

Sorry for inconvenience, but some housekeeping is a must. And thanks a lot for being such a great community member :)

Cheers!
Andrew

On 3/6/07, Marco D'Alia <mule-dev@...> wrote:
Hi,
I've made a proof of concept of an asynchronous transport for permanent
socket connections as a final thesis for my bachelor degree.

The main objective was to manage a large number of simultaneous
connections using SEDA, especially with  protocols that need to keep the
connections open between requests.
For this reason I've used non-blocking sockets (through Mina library):
so while a request is processed there is no thread waiting for a
response, greatly lowering the number of threads required.
Looking for a SEDA implementation the choice fell on the excellent Mule ESB.

The code is composed of some general classes for implementing transports
with nio and two sample implementations: a sender/receiver tcp transport
and a receiver-only http transport.

This can be useful not only for long lasting/permanent connections, but
also to send large quantities of data, streaming and push protocols.

Since it is based on Mina it can also be useful to integrate Mina based
protocols with Mule.

I've arranged a trac repository with the code and a better description
of how it works: http://minamule.madarco.net/trac

I'd be glad to hear some feedback from the Mule developer community :)

Marco D'Alia



---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email



Re: Asynchronous transport for permanent connections

by Andrew Perepelytsya :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Marco,

I can't find a direct SVN link to checkout everything, Trac never displays it in the UI. Could you please post one (read-only is ok)?

Andrew

Re: Asynchronous transport for permanent connections

by Andrew Perepelytsya :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

If there's gonna be a follow up, I'll start. Minor things, but this one should be addressed first-hand, IMO:

MinaConnector uses URIs as keys for the ConcurrentHashMap. I try to avoid URIs as plague, it's just unfixable, period. Have a look at http://www.intertwingly.net/blog/2004/07/31/URI-Equivalence , especially the first comment about RFC 2616.

Andrew

On 3/6/07, Andrew Perepelytsya <aperepel@...> wrote:
Marco,

I can't find a direct SVN link to checkout everything, Trac never displays it in the UI. Could you please post one (read-only is ok)?

Andrew


Re: Asynchronous transport for permanent connections

by Marco D'Alia :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Sure, I would be happy if my code will become part of Mule, I wrote that
copyright notice for this reason.

I've found that issue in my preliminary work for the thesis, and I know
that this transport can be a pain since some changes from the Tcp
transport are deep, but I hope that someone could find a good use for it.

I'm sorry, the svn repository is: http://minamule.madarco.net/svn

Andrew Perepelytsya ha scritto:

> Marco,
>
> Let me kindly forward you to
> http://mule.mulesource.org/jira/browse/MULE-64
>
> Some legal work - I see this disclaimer on the page:
>
> This program is distrubuted as-is bla bla bla ecc.. and you can do
> whatever you want with it (Public Domain)
>
> Are you willing to give up the copyright in order for the code to
> eventually be incorporated into the Mule codebase? IMNAL, but keep in
> mind your reply may mean a legal agreement.
>
> Sorry for inconvenience, but some housekeeping is a must. And thanks a
> lot for being such a great community member :)
>
> Cheers!
> Andrew
>
> On 3/6/07, * Marco D'Alia* <mule-dev@...
> <mailto:mule-dev@...>> wrote:
>
>     Hi,
>     I've made a proof of concept of an asynchronous transport for
>     permanent
>     socket connections as a final thesis for my bachelor degree.
>
>     The main objective was to manage a large number of simultaneous
>     connections using SEDA, especially with  protocols that need to
>     keep the
>     connections open between requests.
>     For this reason I've used non-blocking sockets (through Mina library):
>     so while a request is processed there is no thread waiting for a
>     response, greatly lowering the number of threads required.
>     Looking for a SEDA implementation the choice fell on the excellent
>     Mule ESB.
>
>     The code is composed of some general classes for implementing
>     transports
>     with nio and two sample implementations: a sender/receiver tcp
>     transport
>     and a receiver-only http transport.
>
>     This can be useful not only for long lasting/permanent
>     connections, but
>     also to send large quantities of data, streaming and push protocols.
>
>     Since it is based on Mina it can also be useful to integrate Mina
>     based
>     protocols with Mule.
>
>     I've arranged a trac repository with the code and a better description
>     of how it works: http://minamule.madarco.net/trac
>
>     I'd be glad to hear some feedback from the Mule developer community :)
>
>     Marco D'Alia
>
>
>
>     ---------------------------------------------------------------------
>     To unsubscribe from this list please visit:
>
>         http://xircles.codehaus.org/manage_email
>
>



---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email


Re: Asynchronous transport for permanent connections

by Marco D'Alia :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

I totally agree,
In that piece of code there is also the problem that the endpoint uri in
the reply-to shoud match exactly the one in the inbound endpoint.

In addSession, getSession ecc the URI can be removed, and use instead a
global hashmap, but it is required in hasServer(..)

Andrew Perepelytsya ha scritto:

> If there's gonna be a follow up, I'll start. Minor things, but this
> one should be addressed first-hand, IMO:
>
> MinaConnector uses URIs as keys for the ConcurrentHashMap. I try to
> avoid URIs as plague, it's just unfixable, period. Have a look at
> http://www.intertwingly.net/blog/2004/07/31/URI-Equivalence ,
> especially the first comment about RFC 2616.
>
> Andrew
>
> On 3/6/07, *Andrew Perepelytsya* <aperepel@...
> <mailto:aperepel@...>> wrote:
>
>     Marco,
>
>     I can't find a direct SVN link to checkout everything, Trac never
>     displays it in the UI. Could you please post one (read-only is ok)?
>
>     Andrew
>
>


---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email


Parent Message unknown RE: Asynchronous transport for permanent connections

by 0x6e6562 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi Marco,

Interesting work!

>This can be useful not only for long lasting/permanent
>connections, but also to send large quantities of data,
>streaming and push protocols.

How have you found streaming to work with the MINA API?

I was looking at the class HttpServerConnection from your transport to
find out how you are passing off a stream to a Mule component and if I'm
not wrong, this is using the HttpResponse class from the mule http
transport.

Doesn't this materialize the stream before passing it on to the actual
processing component?

Cheers,

Ben



*****************************************************

    You can find us at www.voca.com

*****************************************************
This communication is confidential and intended for
the exclusive use of the addressee only. You should
not disclose its contents to any other person.
If you are not the intended recipient please notify
the sender named above immediately.

Registered in England, No 1023742,
Registered Office: Voca Limited
Drake House, Three Rivers Court,
Homestead Road, Rickmansworth,
Hertfordshire, WD3 1FX


This message has been checked for all email viruses by MessageLabs.

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email


Re: Asynchronous transport for permanent connections

by Ross Mason-3 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Good work Marco!

We'd be interested in incorporating this module into Mule after a  
code review. The steps to becoming a committer can be found here:  
http://muledocs.org/Becoming+a+Committer

Cheers,

Ross

On 6 Mar 2007, at 19:42, Marco D'Alia wrote:

> Hi,
> I've made a proof of concept of an asynchronous transport for  
> permanent socket connections as a final thesis for my bachelor degree.
>
> The main objective was to manage a large number of simultaneous
> connections using SEDA, especially with  protocols that need to  
> keep the
> connections open between requests.
> For this reason I've used non-blocking sockets (through Mina library):
> so while a request is processed there is no thread waiting for a
> response, greatly lowering the number of threads required.
> Looking for a SEDA implementation the choice fell on the excellent  
> Mule ESB.
>
> The code is composed of some general classes for implementing  
> transports
> with nio and two sample implementations: a sender/receiver tcp  
> transport
> and a receiver-only http transport.
>
> This can be useful not only for long lasting/permanent connections,  
> but
> also to send large quantities of data, streaming and push protocols.
>
> Since it is based on Mina it can also be useful to integrate Mina  
> based
> protocols with Mule.
>
> I've arranged a trac repository with the code and a better description
> of how it works: http://minamule.madarco.net/trac
>
> I'd be glad to hear some feedback from the Mule developer community :)
>
> Marco D'Alia
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list please visit:
>
>    http://xircles.codehaus.org/manage_email
>


---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email


Re: Asynchronous transport for permanent connections

by Marco D'Alia :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi Ben,
I'm sorry but that feature isn't working, in the first revision in the
svn repository you can find an utility class that fix some problem in
the Mina's StreamIoHandler and some tests transport for streaming.
I've not continued that feature for my lack of understanding of the Mule
stream support ( getStreamMessageAdapter and the meaning of the
OutputStream parameter of routeMessage).

Beware that the StreamIoHandler allocate a bytebuffer with ALL the data
in the stream, I think that the only way to implement streaming is to
use messageSent method in the ServerHandler to keep the bytebuffer full.

Ben Hood ha scritto:

> Hi Marco,
>
> Interesting work!
>
>  
>> This can be useful not only for long lasting/permanent
>> connections, but also to send large quantities of data,
>> streaming and push protocols.
>>    
>
> How have you found streaming to work with the MINA API?
>
> I was looking at the class HttpServerConnection from your transport to
> find out how you are passing off a stream to a Mule component and if I'm
> not wrong, this is using the HttpResponse class from the mule http
> transport.
>
> Doesn't this materialize the stream before passing it on to the actual
> processing component?
>
> Cheers,
>
> Ben
>
>
>
> *****************************************************
>
>     You can find us at www.voca.com
>
> *****************************************************
> This communication is confidential and intended for
> the exclusive use of the addressee only. You should
> not disclose its contents to any other person.
> If you are not the intended recipient please notify
> the sender named above immediately.
>
> Registered in England, No 1023742,
> Registered Office: Voca Limited
> Drake House, Three Rivers Court,
> Homestead Road, Rickmansworth,
> Hertfordshire, WD3 1FX
>
>
> This message has been checked for all email viruses by MessageLabs.
>
> ---------------------------------------------------------------------
> To unsubscribe from this list please visit:
>
>     http://xircles.codehaus.org/manage_email
>
>
>  


---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email


Re: Asynchronous transport for permanent connections

by Marco D'Alia :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Thanks Ross :)

In the current version, the transport lack all configuration support,
good exception handling, a complete "outbound" support and support for
stream message payload.

Ross Mason ha scritto:

> Good work Marco!
>
> We'd be interested in incorporating this module into Mule after a code
> review. The steps to becoming a committer can be found here:
> http://muledocs.org/Becoming+a+Committer
>
> Cheers,
>
> Ross
>
> On 6 Mar 2007, at 19:42, Marco D'Alia wrote:
>
>> Hi,
>> I've made a proof of concept of an asynchronous transport for
>> permanent socket connections as a final thesis for my bachelor degree.
>>
>> The main objective was to manage a large number of simultaneous
>> connections using SEDA, especially with  protocols that need to keep the
>> connections open between requests.
>> For this reason I've used non-blocking sockets (through Mina library):
>> so while a request is processed there is no thread waiting for a
>> response, greatly lowering the number of threads required.
>> Looking for a SEDA implementation the choice fell on the excellent
>> Mule ESB.
>>
>> The code is composed of some general classes for implementing transports
>> with nio and two sample implementations: a sender/receiver tcp transport
>> and a receiver-only http transport.
>>
>> This can be useful not only for long lasting/permanent connections, but
>> also to send large quantities of data, streaming and push protocols.
>>
>> Since it is based on Mina it can also be useful to integrate Mina based
>> protocols with Mule.
>>
>> I've arranged a trac repository with the code and a better description
>> of how it works: http://minamule.madarco.net/trac
>>
>> I'd be glad to hear some feedback from the Mule developer community :)
>>
>> Marco D'Alia
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list please visit:
>>
>>    http://xircles.codehaus.org/manage_email
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>


---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email


Parent Message unknown RE: Asynchronous transport for permanent connections

by 0x6e6562 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Marco,

>Beware that the StreamIoHandler allocate a bytebuffer with ALL
>the data in the stream, I think that the only way to implement
>streaming is to use messageSent method in the ServerHandler to
>keep the bytebuffer full.

I've been thinking about that too.

Basically I was wondering whether the InputStream as passed into the
StreamIoHandler is actually backed by a ByteBuffer, thus consuming the
whole stream. I got this suspicion when I saw the asInputStream() and
asOutputStream() methods in the ByteBuffer class. Checking this out is
on my list of TODOs.

What did you mean about keeping the buffer full?

Ben


*****************************************************

    You can find us at www.voca.com

*****************************************************
This communication is confidential and intended for
the exclusive use of the addressee only. You should
not disclose its contents to any other person.
If you are not the intended recipient please notify
the sender named above immediately.

Registered in England, No 1023742,
Registered Office: Voca Limited
Drake House, Three Rivers Court,
Homestead Road, Rickmansworth,
Hertfordshire, WD3 1FX


This message has been checked for all email viruses by MessageLabs.

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email


Parent Message unknown RE: Asynchronous transport for permanent connections

by 0x6e6562 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Marco,

>I'm sorry but that feature isn't working, in the first
>revision in the svn repository you can find an utility class
>that fix some problem in the Mina's StreamIoHandler and some
>tests transport for streaming.
>I've not continued that feature for my lack of understanding
>of the Mule stream support ( getStreamMessageAdapter and the
>meaning of the OutputStream parameter of routeMessage).

The issue of streaming is being tackled in the 1.4 version of Mule, but
is still at an early stage, because I don't if it has been fully
conceptualized (please correct me if I'm wrong).

In 1.4 you can declare a model type to be streaming, but then you can
only use streaming based components in that model because it passes in a
reference to the raw stream to the component.

There are also some issues about passing around references to streams
between different components so this is why the approach has been
conservative.

This is why in my transport I decided to handle streaming natively (i.e.
within the connector) until such a stage that the mule based methodology
stabilizes. I've tried to keep the code as abstract and decoupled as
possible in order to be able to migrate in the future.

HTH,

Ben


*****************************************************

    You can find us at www.voca.com

*****************************************************
This communication is confidential and intended for
the exclusive use of the addressee only. You should
not disclose its contents to any other person.
If you are not the intended recipient please notify
the sender named above immediately.

Registered in England, No 1023742,
Registered Office: Voca Limited
Drake House, Three Rivers Court,
Homestead Road, Rickmansworth,
Hertfordshire, WD3 1FX


This message has been checked for all email viruses by MessageLabs.

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email


Re: Asynchronous transport for permanent connections

by Marco D'Alia :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Ben,
I was thinking it was experimental because of many "todo" comments in
the code.
In some cases, such when there are large chunks of data to send, it can
be convenient to send a stream reference payload, even if there are some
limitations.

In the dispatcher then, instead of send the entire stream, it is
possible to allocate only a part of it and try to check when this buffer
is empty and refill it. I think this can be done in the messageSent()
method of the IoHandler.

Marco

Ben Hood ha scritto:

> Marco,
>
>  
>> I'm sorry but that feature isn't working, in the first
>> revision in the svn repository you can find an utility class
>> that fix some problem in the Mina's StreamIoHandler and some
>> tests transport for streaming.
>> I've not continued that feature for my lack of understanding
>> of the Mule stream support ( getStreamMessageAdapter and the
>> meaning of the OutputStream parameter of routeMessage).
>>    
>
> The issue of streaming is being tackled in the 1.4 version of Mule, but
> is still at an early stage, because I don't if it has been fully
> conceptualized (please correct me if I'm wrong).
>
> In 1.4 you can declare a model type to be streaming, but then you can
> only use streaming based components in that model because it passes in a
> reference to the raw stream to the component.
>
> There are also some issues about passing around references to streams
> between different components so this is why the approach has been
> conservative.
>
> This is why in my transport I decided to handle streaming natively (i.e.
> within the connector) until such a stage that the mule based methodology
> stabilizes. I've tried to keep the code as abstract and decoupled as
> possible in order to be able to migrate in the future.
>
> HTH,
>
> Ben
>
>
> *****************************************************
>
>     You can find us at www.voca.com
>
> *****************************************************
> This communication is confidential and intended for
> the exclusive use of the addressee only. You should
> not disclose its contents to any other person.
> If you are not the intended recipient please notify
> the sender named above immediately.
>
> Registered in England, No 1023742,
> Registered Office: Voca Limited
> Drake House, Three Rivers Court,
> Homestead Road, Rickmansworth,
> Hertfordshire, WD3 1FX
>
>
> This message has been checked for all email viruses by MessageLabs.
>
> ---------------------------------------------------------------------
> To unsubscribe from this list please visit:
>
>     http://xircles.codehaus.org/manage_email
>
>
>  


---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email


Re: Asynchronous transport for permanent connections

by Marco D'Alia :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi,
I've committed to my repository an example on streaming and some small
bugfixes.
I've used org.apache.mina.filter.StreamWriteFilter, after adding it as a
IoFilter, writing an InputStream to a session will gradually send the
data while it is readed from the remote host (will consume the stream in
small chunks).
hope it may help.


Ben Hood ha scritto:

> Marco,
>
>  
>> Beware that the StreamIoHandler allocate a bytebuffer with ALL
>> the data in the stream, I think that the only way to implement
>> streaming is to use messageSent method in the ServerHandler to
>> keep the bytebuffer full.
>>    
>
> I've been thinking about that too.
>
> Basically I was wondering whether the InputStream as passed into the
> StreamIoHandler is actually backed by a ByteBuffer, thus consuming the
> whole stream. I got this suspicion when I saw the asInputStream() and
> asOutputStream() methods in the ByteBuffer class. Checking this out is
> on my list of TODOs.
>
> What did you mean about keeping the buffer full?
>
> Ben
>
>
> *****************************************************
>
>     You can find us at www.voca.com
>
> *****************************************************
> This communication is confidential and intended for
> the exclusive use of the addressee only. You should
> not disclose its contents to any other person.
> If you are not the intended recipient please notify
> the sender named above immediately.
>
> Registered in England, No 1023742,
> Registered Office: Voca Limited
> Drake House, Three Rivers Court,
> Homestead Road, Rickmansworth,
> Hertfordshire, WD3 1FX
>
>
> This message has been checked for all email viruses by MessageLabs.
>
> ---------------------------------------------------------------------
> To unsubscribe from this list please visit:
>
>     http://xircles.codehaus.org/manage_email
>
>
>  


---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email


Re: Asynchronous transport for permanent connections

by Marco D'Alia :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi Ross,
Thanks for the interest in my transport, I'll be glad if it will be
included into Mule and I'd like to help you and the others to review the
code.
Since I don't understand some Mule internals and I know there are some
"weak points" in the transport, I'd like if someone can take a look at
the code, and help me refining it.

I'll continue using this mailing list, or I have to open an issue in Jira?

Thanks,

Marco D'Alia

Ross Mason ha scritto:

> Good work Marco!
>
> We'd be interested in incorporating this module into Mule after a code
> review. The steps to becoming a committer can be found here:
> http://muledocs.org/Becoming+a+Committer
>
> Cheers,
>
> Ross
>
> On 6 Mar 2007, at 19:42, Marco D'Alia wrote:
>
>> Hi,
>> I've made a proof of concept of an asynchronous transport for
>> permanent socket connections as a final thesis for my bachelor degree.
>>
>> The main objective was to manage a large number of simultaneous
>> connections using SEDA, especially with  protocols that need to keep the
>> connections open between requests.
>> For this reason I've used non-blocking sockets (through Mina library):
>> so while a request is processed there is no thread waiting for a
>> response, greatly lowering the number of threads required.
>> Looking for a SEDA implementation the choice fell on the excellent
>> Mule ESB.
>>
>> The code is composed of some general classes for implementing transports
>> with nio and two sample implementations: a sender/receiver tcp transport
>> and a receiver-only http transport.
>>
>> This can be useful not only for long lasting/permanent connections, but
>> also to send large quantities of data, streaming and push protocols.
>>
>> Since it is based on Mina it can also be useful to integrate Mina based
>> protocols with Mule.
>>
>> I've arranged a trac repository with the code and a better description
>> of how it works: http://minamule.madarco.net/trac
>>
>> I'd be glad to hear some feedback from the Mule developer community :)
>>
>> Marco D'Alia
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list please visit:
>>
>>    http://xircles.codehaus.org/manage_email
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>


---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email


Re: Asynchronous transport for permanent connections

by Ben Hood :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi Marco,

I would start a Jira issue if were you, things get lost really easily
in the mailing list.

I haven't had a chance yet to look at your new code, will do so shortly.

I had a quick look at the ByteBuffer issue (without diving into the
Mina code just yet) and it seems to have a smooth memory consumption
for a very large throughput size (>500000 messages whilst limiting the
heap to a max of 20M). I've still got some more testing to go before
I'm convinced that this is really working in a streaming fashion, but
my initial assumption is that it is chunking off the stream somehow
and avoiding excessive memory consumption. You can see my test code at
http://0x6e6562.zone.mod3.co.uk/hg/mule-transport-hessian/file/813e4ea3d879/src/test/java/org/mule/providers/hessian/HessianStreamingTest.java

HTH,

Ben

On 3/7/07, Marco D'Alia <mule-dev@...> wrote:

> Hi Ross,
> Thanks for the interest in my transport, I'll be glad if it will be
> included into Mule and I'd like to help you and the others to review the
> code.
> Since I don't understand some Mule internals and I know there are some
> "weak points" in the transport, I'd like if someone can take a look at
> the code, and help me refining it.
>
> I'll continue using this mailing list, or I have to open an issue in Jira?
>
> Thanks,
>
> Marco D'Alia
>
> Ross Mason ha scritto:
> > Good work Marco!
> >
> > We'd be interested in incorporating this module into Mule after a code
> > review. The steps to becoming a committer can be found here:
> > http://muledocs.org/Becoming+a+Committer
> >
> > Cheers,
> >
> > Ross
> >
> > On 6 Mar 2007, at 19:42, Marco D'Alia wrote:
> >
> >> Hi,
> >> I've made a proof of concept of an asynchronous transport for
> >> permanent socket connections as a final thesis for my bachelor degree.
> >>
> >> The main objective was to manage a large number of simultaneous
> >> connections using SEDA, especially with  protocols that need to keep the
> >> connections open between requests.
> >> For this reason I've used non-blocking sockets (through Mina library):
> >> so while a request is processed there is no thread waiting for a
> >> response, greatly lowering the number of threads required.
> >> Looking for a SEDA implementation the choice fell on the excellent
> >> Mule ESB.
> >>
> >> The code is composed of some general classes for implementing transports
> >> with nio and two sample implementations: a sender/receiver tcp transport
> >> and a receiver-only http transport.
> >>
> >> This can be useful not only for long lasting/permanent connections, but
> >> also to send large quantities of data, streaming and push protocols.
> >>
> >> Since it is based on Mina it can also be useful to integrate Mina based
> >> protocols with Mule.
> >>
> >> I've arranged a trac repository with the code and a better description
> >> of how it works: http://minamule.madarco.net/trac
> >>
> >> I'd be glad to hear some feedback from the Mule developer community :)
> >>
> >> Marco D'Alia
> >>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe from this list please visit:
> >>
> >>    http://xircles.codehaus.org/manage_email
> >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe from this list please visit:
> >
> >    http://xircles.codehaus.org/manage_email
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list please visit:
>
>     http://xircles.codehaus.org/manage_email
>
>


--
int main(){long u=0x6e6562;printf("%.4s",(char*)&u);}

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email


Re: Asynchronous transport for permanent connections

by Ross Mason-3 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message


On 7 Mar 2007, at 22:06, Marco D'Alia wrote:

> Hi Ross,
> Thanks for the interest in my transport, I'll be glad if it will be  
> included into Mule and I'd like to help you and the others to  
> review the code.
> Since I don't understand some Mule internals and I know there are  
> some "weak points" in the transport, I'd like if someone can take a  
> look at the code, and help me refining it.
No problem, we can help out there :)
>
> I'll continue using this mailing list, or I have to open an issue  
> in Jira?
Andrew has already linked you to a Jira: http://mule.mulesource.org/ 
jira/browse/MULE-64. It would be best to do all further communication  
on this Jira

To get Access to the mule-contrib SVN, would you mind going through  
the committer process (its painless :) ) here: http://muledocs.org/ 
Becoming+a+Committer

Cheers,

Ross

>
> Thanks,
>
> Marco D'Alia
>
> Ross Mason ha scritto:
>> Good work Marco!
>>
>> We'd be interested in incorporating this module into Mule after a  
>> code review. The steps to becoming a committer can be found here:  
>> http://muledocs.org/Becoming+a+Committer
>>
>> Cheers,
>>
>> Ross
>>
>> On 6 Mar 2007, at 19:42, Marco D'Alia wrote:
>>
>>> Hi,
>>> I've made a proof of concept of an asynchronous transport for  
>>> permanent socket connections as a final thesis for my bachelor  
>>> degree.
>>>
>>> The main objective was to manage a large number of simultaneous
>>> connections using SEDA, especially with  protocols that need to  
>>> keep the
>>> connections open between requests.
>>> For this reason I've used non-blocking sockets (through Mina  
>>> library):
>>> so while a request is processed there is no thread waiting for a
>>> response, greatly lowering the number of threads required.
>>> Looking for a SEDA implementation the choice fell on the  
>>> excellent Mule ESB.
>>>
>>> The code is composed of some general classes for implementing  
>>> transports
>>> with nio and two sample implementations: a sender/receiver tcp  
>>> transport
>>> and a receiver-only http transport.
>>>
>>> This can be useful not only for long lasting/permanent  
>>> connections, but
>>> also to send large quantities of data, streaming and push protocols.
>>>
>>> Since it is based on Mina it can also be useful to integrate Mina  
>>> based
>>> protocols with Mule.
>>>
>>> I've arranged a trac repository with the code and a better  
>>> description
>>> of how it works: http://minamule.madarco.net/trac
>>>
>>> I'd be glad to hear some feedback from the Mule developer  
>>> community :)
>>>
>>> Marco D'Alia
>>>
>>>
>>>
>>> --------------------------------------------------------------------
>>> -
>>> To unsubscribe from this list please visit:
>>>
>>>    http://xircles.codehaus.org/manage_email
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list please visit:
>>
>>    http://xircles.codehaus.org/manage_email
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list please visit:
>
>    http://xircles.codehaus.org/manage_email
>


---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email