precompile mode for embedding

[Yoko Harada]

Hi,

I'm interested in what Charles said in
http://www.nabble.com/-ANN--ruby2java-0.0.1-Released-td23679153.html
about "jruby.compile.mode=FORCE." This option works in
Ruby#runNomally(Node scriptNode) method and causes compilation. So, I
implemented a feature to set this option and compared with no
compilation mode. The result was not that expected. The reason would
be that runNormally() method compile the script everytime before
evaluation. When I executed testString.rb a hundred times, the result
was:

CompileMode.OFF          2983 ms
CompileMode.JIT          21463 ms
CompileMode.FORCE  21032 ms

I used a parse-once-eval-many-times feature to test it, so I could
conclude that the compilation itself was so slow.

Next, I implemented a "compile-once-eval-many-times" feature. To make
this, I changed two private methods to be public, "public Script
tryCompile(Node node)" and "public IRubyObject runScript(Script
script), so that external programs can execute compilation and
evaluation individually. This time the result was:

CompileMode.OFF        2979 ms
CompileMode.JIT          2785 ms
CompileMode.FORCE   2671 ms

This was a great improvement, but the two compilation modes were still
not so fast compared with no compilation.
In general, how fast does a compiled script run compared with no
compilation mode?

If ruby2java makes it really fast, I want to execute ruby2java script
in Java before evaluations. In this case, the problem would be how to
get a compiled script and eval it. I want to avoid reading a compiled
script from a file. Any idea?

-Yoko

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Charles Oliver Nutter-4]

Yoko Harada wrote:

> I'm interested in what Charles said in
> http://www.nabble.com/-ANN--ruby2java-0.0.1-Released-td23679153.html
> about "jruby.compile.mode=FORCE." This option works in
> Ruby#runNomally(Node scriptNode) method and causes compilation. So, I
> implemented a feature to set this option and compared with no
> compilation mode. The result was not that expected. The reason would
> be that runNormally() method compile the script everytime before
> evaluation. When I executed testString.rb a hundred times, the result
> was:
>
> CompileMode.OFF          2983 ms
> CompileMode.JIT          21463 ms
> CompileMode.FORCE  21032 ms

Yes, I would not expect compilation to do much better than JIT mode, and
if it's forced to compile every time then it would certainly be slower
than no compiling at all.

> Next, I implemented a "compile-once-eval-many-times" feature. To make
> this, I changed two private methods to be public, "public Script
> tryCompile(Node node)" and "public IRubyObject runScript(Script
> script), so that external programs can execute compilation and
> evaluation individually. This time the result was:
>
> CompileMode.OFF        2979 ms
> CompileMode.JIT          2785 ms
> CompileMode.FORCE   2671 ms
>
> This was a great improvement, but the two compilation modes were still
> not so fast compared with no compilation.
> In general, how fast does a compiled script run compared with no
> compilation mode?

It really depends what the script is doing. If it's just a simple script
that you're running many times, it may not make a difference. Are you
just precompiling the contents of testString.rb and re-running them?
Given that the bulk of the calls in that script would just be calling
String methods, I don't expect precompilation would improve things very
much.

> If ruby2java makes it really fast, I want to execute ruby2java script
> in Java before evaluations. In this case, the problem would be how to
> get a compiled script and eval it. I want to avoid reading a compiled
> script from a file. Any idea?

ruby2java currently does no compilation *at all*. The contents of the
specified scripts are actually evaluated on class load. It could
certainly precompile and run the precompiled content at startup, but it
does not at the moment.

Are you interested in this to wire up the "compiled" logic in JSR223 or
for some other specific purpose?

- Charlie

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[iceslice]

Hi

I've written the benchmark which runs ruwiki on top of Jruby as Servlet and measures the throughput making large number of requests in parallel. I've tried all three modes with Ruby#runNomally as script entry point. To my surprise I've found out that JIT mode works 20% slower than mode OFF and FORCE causes OOME due to permgen outage. 

Netherless I was able to use the compiler and boost the throughput 2x compared to mode off by replacing the LoadService in the Runtime with custom one which performs on demand compilation and class caching. The implementation is slightly ugly due to lack of API in LoadService to override the resource loading after the resource is resolved, but it does the job nicely.

You can look for the details into the project http://code.google.com/p/wikimark/

The specific code is located in org.apache.cgiservlet.RubyServlet and org.apache.cgiservlet.RubyCachedLoadServiceCreator

PS: By the way according to my ad hok measurments with this optimization the throughput in benchmark is about 10% better compared to native Ruby 1.8 using mod_ruby. 

On Tue, Jun 9, 2009 at 6:02 AM, Charles Oliver Nutter <headius@...> wrote:

Yoko Harada wrote:

I'm interested in what Charles said in
http://www.nabble.com/-ANN--ruby2java-0.0.1-Released-td23679153.html
about "jruby.compile.mode=FORCE." This option works in
Ruby#runNomally(Node scriptNode) method and causes compilation. So, I
implemented a feature to set this option and compared with no
compilation mode. The result was not that expected. The reason would
be that runNormally() method compile the script everytime before
evaluation. When I executed testString.rb a hundred times, the result
was:

CompileMode.OFF          2983 ms
CompileMode.JIT          21463 ms
CompileMode.FORCE  21032 ms

Yes, I would not expect compilation to do much better than JIT mode, and if it's forced to compile every time then it would certainly be slower than no compiling at all.

Next, I implemented a "compile-once-eval-many-times" feature. To make
this, I changed two private methods to be public, "public Script
tryCompile(Node node)" and "public IRubyObject runScript(Script
script), so that external programs can execute compilation and
evaluation individually. This time the result was:

CompileMode.OFF        2979 ms
CompileMode.JIT          2785 ms
CompileMode.FORCE   2671 ms

This was a great improvement, but the two compilation modes were still
not so fast compared with no compilation.
In general, how fast does a compiled script run compared with no
compilation mode?

It really depends what the script is doing. If it's just a simple script that you're running many times, it may not make a difference. Are you just precompiling the contents of testString.rb and re-running them? Given that the bulk of the calls in that script would just be calling String methods, I don't expect precompilation would improve things very much.

If ruby2java makes it really fast, I want to execute ruby2java script
in Java before evaluations. In this case, the problem would be how to
get a compiled script and eval it. I want to avoid reading a compiled
script from a file. Any idea?

ruby2java currently does no compilation *at all*. The contents of the specified scripts are actually evaluated on class load. It could certainly precompile and run the precompiled content at startup, but it does not at the moment.

Are you interested in this to wire up the "compiled" logic in JSR223 or for some other specific purpose?

- Charlie

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

  http://xircles.codehaus.org/manage_email

--
Thanks.
Sergey.

[Yoko Harada]

On Mon, Jun 8, 2009 at 10:02 PM, Charles Oliver
Nutter<headius@...> wrote:

> Yoko Harada wrote:
>>
>> I'm interested in what Charles said in
>> http://www.nabble.com/-ANN--ruby2java-0.0.1-Released-td23679153.html
>> about "jruby.compile.mode=FORCE." This option works in
>> Ruby#runNomally(Node scriptNode) method and causes compilation. So, I
>> implemented a feature to set this option and compared with no
>> compilation mode. The result was not that expected. The reason would
>> be that runNormally() method compile the script everytime before
>> evaluation. When I executed testString.rb a hundred times, the result
>> was:
>>
>> CompileMode.OFF          2983 ms
>> CompileMode.JIT          21463 ms
>> CompileMode.FORCE  21032 ms
>
> Yes, I would not expect compilation to do much better than JIT mode, and if
> it's forced to compile every time then it would certainly be slower than no
> compiling at all.
>
>> Next, I implemented a "compile-once-eval-many-times" feature. To make
>> this, I changed two private methods to be public, "public Script
>> tryCompile(Node node)" and "public IRubyObject runScript(Script
>> script), so that external programs can execute compilation and
>> evaluation individually. This time the result was:
>>
>> CompileMode.OFF        2979 ms
>> CompileMode.JIT          2785 ms
>> CompileMode.FORCE   2671 ms
>>
>> This was a great improvement, but the two compilation modes were still
>> not so fast compared with no compilation.
>> In general, how fast does a compiled script run compared with no
>> compilation mode?
>
> It really depends what the script is doing. If it's just a simple script
> that you're running many times, it may not make a difference. Are you just
> precompiling the contents of testString.rb and re-running them? Given that
> the bulk of the calls in that script would just be calling String methods, I
> don't expect precompilation would improve things very much.

Not really. I tested this on RedBridge, JSR223 implementation, so
pre/post operations for sharing variables ran in every evaluation.
But, I think the comparison was fair since all three modes ran in
exactly the same way during timing.

>> If ruby2java makes it really fast, I want to execute ruby2java script
>> in Java before evaluations. In this case, the problem would be how to
>> get a compiled script and eval it. I want to avoid reading a compiled
>> script from a file. Any idea?
>
> ruby2java currently does no compilation *at all*. The contents of the
> specified scripts are actually evaluated on class load. It could certainly
> precompile and run the precompiled content at startup, but it does not at
> the moment.

OK, I understood what ruby2java does now. I expect ruby2java makes
precompilation.

>
> Are you interested in this to wire up the "compiled" logic in JSR223 or for
> some other specific purpose?

I want this for JSR223. If a compiled script runs faster than before,
it would be a powerful feature for new JSR223 implementation.

I'm wondering whether I should remove the feature to choose a mode
among three from JSR223 implementation. People might expect that
performance will improve, but the fact is not.

-Yoko

>
> - Charlie
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>   http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Charles Oliver Nutter-4]

On Thu, Jul 2, 2009 at 4:58 PM, Yoko Harada<yokolet@...> wrote:
> OK, I understood what ruby2java does now. I expect ruby2java makes
> precompilation.

It may in the future, and there may be even a different approach to
generating Java classes from Ruby code. But at the moment it's a
fairly simple mechanism.

>> Are you interested in this to wire up the "compiled" logic in JSR223 or for
>> some other specific purpose?
>
> I want this for JSR223. If a compiled script runs faster than before,
> it would be a powerful feature for new JSR223 implementation.
>
> I'm wondering whether I should remove the feature to choose a mode
> among three from JSR223 implementation. People might expect that
> performance will improve, but the fact is not.

Well, it's not entirely true that performance won't improve. Compiled
(not necessarily precompiled) code does run faster than interpreted,
but since the JIT kicks in fairly quickly on almost all applications,
most people don't see the difference from precompiling. But
precompiled does have the benefit of avoiding the JIT warmup time,
plus it delivers bytecode to the JVM more quickly, meaning the JVM
itself can warm up faster. Eventual performance may or may not be
better, but it does eliminate some of the startup slowness.

- Charlie

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email