Slowdown on GAE using pre-compiled templates

[Arun Kumar-35]

hi everyone,
i'm new to the list, well Cheetah too, and i really like what i see!

i came across a problem using Cheetah on Google App Engine (GAE). When i
used pre-compiled templates, the time to render some 500 rows of a table
was over 140 seconds. Using dynamically compiled templates the page was
rendering in about 200ms.

After chatting with rtyler on irc and doing some profile tests, we
managed to narrow the problem down to the valueFromFrameOrSearchList
call from Cheetah/NameMapper.py. Which appears to be the main difference
between the two. The dynamically compiled templates uses
valueFromSearchList (and not VFFSL).

One option to circumvent this problem has been to use the
'useStackFrames=False' #compiler-settings. That seems to get the page to
render at about the same speed as the dynamically generated template.

Let me know if anyone wants the profile tests, I can send you a link to
a small GAE app, which you can run (using the GAE SDK) and check results.

cheers,
arun

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Cheetahtemplate-discuss mailing list
Cheetahtemplate-discuss@...
https://lists.sourceforge.net/lists/listinfo/cheetahtemplate-discuss

[tyler-53]

On Wed, 14 Oct 2009, Arun Kumar wrote:

> i came across a problem using Cheetah on Google App Engine (GAE). When i
> used pre-compiled templates, the time to render some 500 rows of a table
> was over 140 seconds. Using dynamically compiled templates the page was
> rendering in about 200ms.
>
> After chatting with rtyler on irc and doing some profile tests, we
> managed to narrow the problem down to the valueFromFrameOrSearchList
> call from Cheetah/NameMapper.py. Which appears to be the main difference
> between the two. The dynamically compiled templates uses
> valueFromSearchList (and not VFFSL).

To help back this up with some of the numbers we were looking at, here's
some of the profiler results from running the pre-compiled template with
useStackFrames=True:

        85442921 function calls (85412686 primitive calls) in 141.942 CPU seconds

           Ordered by: internal time
           List reduced from 153 to 80 due to restriction <80>

           ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        160087/160064   48.591    0.000  103.762    0.001 /usr/lib/python2.5/inspect.py:408(getmodule)
         19768401   26.590    0.000   36.291    0.000 /usr/lib/python2.5/inspect.py:35(ismodule)
         22574035   11.350    0.000   11.350    0.000 {isinstance}
         15176510   10.810    0.000   10.810    0.000 {hasattr}
         14517734    6.690    0.000    6.690    0.000 {method 'get' of 'dict' objects}
           286859    4.140    0.000    6.590    0.000 /usr/lib/python2.5/posixpath.py:373(normpath)
           286831    3.410    0.000   13.300    0.000 /home/arun/projects/libraries/google_appengine/google/appengine/tools/dev_appserver.py:1106(__call__)
           525212    2.690    0.000    2.690    0.000 {built-in method match}
        280312/280112    2.530    0.000   39.211    0.000 /usr/lib/python2.5/inspect.py:381(getsourcefile)
           140056    2.400    0.000  119.022    0.001 /usr/lib/python2.5/inspect.py:454(findsource)
           290316    2.050    0.000    7.140    0.000 /usr/lib/python2.5/inspect.py:342(getfile)
           140056    1.750    0.000  142.182    0.001 /usr/lib/python2.5/inspect.py:820(getframeinfo)
                70029    1.510    0.000    1.510    0.000 {method 'items' of 'dict' objects}
          2592375    1.360    0.000    1.360    0.000 {method 'append' of 'list' objects


While the slowdown doesn't directly come from Cheetah itself, it comes
from Google App Engine's suboptimal sandboxing implementation of the
'inspect' module (and related features).


> One option to circumvent this problem has been to use the
> 'useStackFrames=False' #compiler-settings. That seems to get the page to
> render at about the same speed as the dynamically generated template.
>
> Let me know if anyone wants the profile tests, I can send you a link to
> a small GAE app, which you can run (using the GAE SDK) and check results.


Thanks for the mail Arun; I'm looking forward to further refining your
cStringIO patchset to squeeze a little more performance out of Cheetah
:)


Cheers,
-R. Tyler Ballance


------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Cheetahtemplate-discuss mailing list
Cheetahtemplate-discuss@...
https://lists.sourceforge.net/lists/listinfo/cheetahtemplate-discuss