Velocity, ByteBuffers and GatheringByteChannels

[Babak Farhang-2]

Hi all,

I'm tinkering with a java, non-blocking, HTTP server implementation
and am considering tweaking Velocity (because of its
container-agnostic design) so that I can use it's output in such an
environment.

Specifically, instead of having the Velocity Template "merge" its
output to a Writer, I want to create a Template that "merges" output
to something like a java.util.List<java.nio.ByteBuffer>.

The purpose of this modified output would be to ultimately pass (it in
the form of a ByteBuffer array) as the argument to a
java.nio.channels.SocketChannel.write -- a non-blocking "gathering"
operation.

The idea with the java.util.List<java.nio.ByteBuffer> is that the
content of some ByteBuffers will be dynamically generated; the
contents of the rest would be static.

So how to pull this off?  The first place I considered hacking was the
Template class itself (something like intercepting every ASTText
instance's render method for static content and doing something else
with the dynamic bits of content).  Needless to say, this has a
'hacky' feel to it ;)  And since I'm new to Velocity, I'm thinking
maybe there's a more elegant approach.

And ideas?

Kind regards,
-Babak

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@...
For additional commands, e-mail: user-help@...

[Byron Foster-2]

On Apr 21, 2009, at 19:18 , Babak Farhang wrote:

> I'm tinkering with a java, non-blocking, HTTP server implementation
> and am considering tweaking Velocity (because of its
> container-agnostic design) so that I can use it's output in such an
> environment.
>
> Specifically, instead of having the Velocity Template "merge" its
> output to a Writer, I want to create a Template that "merges" output
> to something like a java.util.List<java.nio.ByteBuffer>.
>
> The purpose of this modified output would be to ultimately pass (it in
> the form of a ByteBuffer array) as the argument to a
> java.nio.channels.SocketChannel.write -- a non-blocking "gathering"
> operation.
>
> The idea with the java.util.List<java.nio.ByteBuffer> is that the
> content of some ByteBuffers will be dynamically generated; the
> contents of the rest would be static.
>
> So how to pull this off?  The first place I considered hacking was the
> Template class itself (something like intercepting every ASTText
> instance's render method for static content and doing something else
> with the dynamic bits of content).  Needless to say, this has a
> 'hacky' feel to it ;)  And since I'm new to Velocity, I'm thinking
> maybe there's a more elegant approach.

I'm not sure why you need to hack Velocity for this.  You can create  
your own Writer that is wired up to do whatever you need, for example  
write to a ByteBuffer.  Then pass your custom Writer to the merge.  Is  
there any functionality that this approach wouldn't accommodate?


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@...
For additional commands, e-mail: user-help@...

[Babak Farhang-2]

> I'm not sure why you need to hack Velocity for this.  You can create your
> own Writer that is wired up to do whatever you need, for example write to a
> ByteBuffer.  Then pass your custom Writer to the merge.  Is there any
> functionality that this approach wouldn't accommodate?

That approach is certainly "functional", but it's not as efficient as
the approach I'm describing. The approach I have in mind does away
with the extra copying of the static parts of the output.

In Velocity, the static parts of the output, if I understand right,
are internally represented at runtime as char arrays (in ASTText), no?
 I'm suggesting if we managed to create a ByteBuffer representation
along-side that char array (e.g. UTF-8 encoded), then we could
"collect" these ByteBuffers by just returning a new (read-only) view
of them. I'm thinking since most of the generated content is in fact
made up of static parts, the memory savings might be significant.


On Tue, Apr 21, 2009 at 11:09 PM, Byron Foster <byron@...> wrote:

> On Apr 21, 2009, at 19:18 , Babak Farhang wrote:
>
>> I'm tinkering with a java, non-blocking, HTTP server implementation
>> and am considering tweaking Velocity (because of its
>> container-agnostic design) so that I can use it's output in such an
>> environment.
>>
>> Specifically, instead of having the Velocity Template "merge" its
>> output to a Writer, I want to create a Template that "merges" output
>> to something like a java.util.List<java.nio.ByteBuffer>.
>>
>> The purpose of this modified output would be to ultimately pass (it in
>> the form of a ByteBuffer array) as the argument to a
>> java.nio.channels.SocketChannel.write -- a non-blocking "gathering"
>> operation.
>>
>> The idea with the java.util.List<java.nio.ByteBuffer> is that the
>> content of some ByteBuffers will be dynamically generated; the
>> contents of the rest would be static.
>>
>> So how to pull this off?  The first place I considered hacking was the
>> Template class itself (something like intercepting every ASTText
>> instance's render method for static content and doing something else
>> with the dynamic bits of content).  Needless to say, this has a
>> 'hacky' feel to it ;)  And since I'm new to Velocity, I'm thinking
>> maybe there's a more elegant approach.
>
>
> I'm not sure why you need to hack Velocity for this.  You can create your
> own Writer that is wired up to do whatever you need, for example write to a
> ByteBuffer.  Then pass your custom Writer to the merge.  Is there any
> functionality that this approach wouldn't accommodate?
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@...
> For additional commands, e-mail: user-help@...
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@...
For additional commands, e-mail: user-help@...

[Byron Foster-2]

On Apr 22, 2009, at 0:52 , Babak Farhang wrote:

>> I'm not sure why you need to hack Velocity for this.  You can  
>> create your
>> own Writer that is wired up to do whatever you need, for example  
>> write to a
>> ByteBuffer.  Then pass your custom Writer to the merge.  Is there any
>> functionality that this approach wouldn't accommodate?
>
> That approach is certainly "functional", but it's not as efficient as
> the approach I'm describing. The approach I have in mind does away
> with the extra copying of the static parts of the output.
>
> In Velocity, the static parts of the output, if I understand right,
> are internally represented at runtime as char arrays (in ASTText), no?
> I'm suggesting if we managed to create a ByteBuffer representation
> along-side that char array (e.g. UTF-8 encoded), then we could
> "collect" these ByteBuffers by just returning a new (read-only) view
> of them. I'm thinking since most of the generated content is in fact
> made up of static parts, the memory savings might be significant.

In the interest of performance you may make some gains since the  
conversion of Char to some specified output encoding would not be  
necessary.  My guess is that in practice the gains would be minimal,  
but it would be interesting to test.

In the interest of memory it depends on what you would consider  
significant.  As it stands there are actually two copies of the static  
text stored, one within the Char array as you mention above, and a  
String object that is created as an artifact of the parser (look at  
ASTText.init method).  A byte array that contains the UTF-8 encoding  
would be about half the size of the existing Char array.  The very  
best you could do with a very simple template is reduce the total  
memory size by 25%.  With a more complex template that contains many  
AST nodes your results will not be as good.

In practice you would loose the flexibility of a Template being output  
encoding neutral, but in practice there is usually only one output  
encoding anyway.  I'm dubious about the gains, but it would be  
interesting to explore.



---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@...
For additional commands, e-mail: user-help@...

[Babak Farhang-2]

> In the interest of memory it depends on what you would consider significant.

My bad for not communicating clearly.  I was just referring to the
*dynamic* memory savings of taking one approach over another to create
an array of ByteBuffers representing the generated content.

Like Byron says, the straight-forward way (the one that works!) is to
wrap a Writer around one (or a few) ByteBuffer(s) and call
Template.merge as usual, and then retrieve the Writer's backing
ByteBuffer(s) that were filled during the merge.

The possible dynamic memory savings I have in mind come from that
observation that we might be able to "cache" the parts of the output
(ByteBuffers) that are static. Here's what the output might look like
using this "caching" strategy:

ByteBuffer[] output = {
  b0,   //  "static" cached content produced by ByteBuffer.duplicate()
  b1,   // dynamically generated content
  b2,  //  "static" cached content produced by ByteBuffer.duplicate()
};

Above, only the b1 element is actually a newly written ByteBuffer.
The b0 and b1 elements are cached (perhaps maintained as a field in an
ASTText node). So the memory savings in this crude example is
dependent on the relative sizes of b0, b1, and b2.  If b1 is small
relative to b0 or b2, then we might expect both faster performance
(less stuff to write to ByteBuffers) and less memory use (since
ByteBuffer.duplicate() just returns another *view* of the "cached"
buffer.)

> A byte
> array that contains the UTF-8 encoding would be about half the size of the
> existing Char array.  The very best you could do with a very simple template
> is reduce the total memory size by 25%.  With a more complex template that
> contains many AST nodes your results will not be as good.

Interesting observations regarding implications to memory footprint of
a Template instance.  Hadn't thought about that..  Again, I didn't
have the memory footprint of the Template instance itself in
mind--just the memory footprint of calling something like
Template.merge to create the ByteBuffers. Sorry for mis-communicating.

> In practice you would loose the flexibility of a Template being output
> encoding neutral, but in practice there is usually only one output encoding
> anyway.

Yes, that's a definite drawback to have in mind (or to somehow
mitigate through the use of some kind of abstraction).

> I'm dubious about the gains, but it would be interesting to
> explore.

I'll report back my results. Are there any benchmark
templates/projects that I should use for this purpose?


On Wed, Apr 22, 2009 at 8:29 PM, Byron Foster <byron@...> wrote:

>
> On Apr 22, 2009, at 0:52 , Babak Farhang wrote:
>
>>> I'm not sure why you need to hack Velocity for this.  You can create your
>>> own Writer that is wired up to do whatever you need, for example write to
>>> a
>>> ByteBuffer.  Then pass your custom Writer to the merge.  Is there any
>>> functionality that this approach wouldn't accommodate?
>>
>> That approach is certainly "functional", but it's not as efficient as
>> the approach I'm describing. The approach I have in mind does away
>> with the extra copying of the static parts of the output.
>>
>> In Velocity, the static parts of the output, if I understand right,
>> are internally represented at runtime as char arrays (in ASTText), no?
>> I'm suggesting if we managed to create a ByteBuffer representation
>> along-side that char array (e.g. UTF-8 encoded), then we could
>> "collect" these ByteBuffers by just returning a new (read-only) view
>> of them. I'm thinking since most of the generated content is in fact
>> made up of static parts, the memory savings might be significant.
>
>
> In the interest of performance you may make some gains since the conversion
> of Char to some specified output encoding would not be necessary.  My guess
> is that in practice the gains would be minimal, but it would be interesting
> to test.
>
> In the interest of memory it depends on what you would consider significant.
>  As it stands there are actually two copies of the static text stored, one
> within the Char array as you mention above, and a String object that is
> created as an artifact of the parser (look at ASTText.init method).  A byte
> array that contains the UTF-8 encoding would be about half the size of the
> existing Char array.  The very best you could do with a very simple template
> is reduce the total memory size by 25%.  With a more complex template that
> contains many AST nodes your results will not be as good.
>
> In practice you would loose the flexibility of a Template being output
> encoding neutral, but in practice there is usually only one output encoding
> anyway.  I'm dubious about the gains, but it would be interesting to
> explore.
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@...
> For additional commands, e-mail: user-help@...
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@...
For additional commands, e-mail: user-help@...

[Byron Foster-2]

On Apr 23, 2009, at 12:11 , Babak Farhang wrote:

> I'll report back my results. Are there any benchmark
> templates/projects that I should use for this purpose?


If you have the trunk checked out there's a benchmark in experimental/
benchmark.  It's simple, but it covers allot of core functionality,  
and it's multi-threaded. However, the run script (run.sh) requires  
bash, and I don't know how Cygwin friendly it is...  run.sh has some  
info at the top for running it, but basically you just build Velocity,  
then execute run.sh


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@...
For additional commands, e-mail: user-help@...