Catalyst benchmark 5.7 VS 5.8

View: New views
20 Messages — Rating Filter:   Alert me  
< Prev | 1 - 2 | Next >

Catalyst benchmark 5.7 VS 5.8

by Fayland Lam :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

I'm wondering if someone here did a benchmark between Catalyst 5.7 and 5.8

here is the result from our server: http://scsys.co.uk:8001/34323

the background is
Catalyst 5.7011 VS Catalyst 5.80013
CPU: Intel(R) Core(TM)2 Quad  CPU   Q8200  @ 2.33GHz
RAM: 4G
OS: Centos5

from the top, each httpd takes 20M more RAM in 5.8 compared with 5.7

5.7
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
22979 apache    16   0  167m 142m 4248 S 17.0  3.5   0:06.07 httpd

5.8
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
24813 apache    15   0  190m 165m 4000 S 15.6  4.1   0:02.56 httpd


in this case, I really can't let my boss agree me to upgrade the Catalyst.

is it normal? any thoughts?

Thanks.
--
Fayland Lam // http://www.fayland.org/

_______________________________________________
List: Catalyst@...
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/

Re: Catalyst benchmark 5.7 VS 5.8

by Tomas Doran :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Fayland Lam wrote:
> from the top, each httpd takes 20M more RAM in 5.8 compared with 5.7

No, that'll be 20Mb of RAM _in total_, as all of those pages should be
shared between your apache processes (given that you're preloading your
application in the parent process).

top totally doesn't show how much RAM is shared by copy on write at all,
and so is misleading you here.

Cheers
t0m

_______________________________________________
List: Catalyst@...
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/

Re: Catalyst benchmark 5.7 VS 5.8

by Toby Corkindale-3 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

(Apologies for top-posting.. have momentarily lost the option to change quoting styles it seems..)

Fayland, I was looking at the benchmarks that you linked, and was just wondering which version of Perl you're running against?

(CentOS 5 was one of the operating systems that came with the badly-patched Perl with the slow bless performance..
although I'm sure it's been patched by now?
ie. http://blog.vipul.net/2008/08/24/redhat-perl-what-a-tragedy/
)

Cheers,
Toby

----- Original Message -----
From: Fayland Lam <fayland@...>
To: catalyst@...
Sent: Mon, 28 Sep 2009 15:56:36 +1000 (EST)
Subject: [Catalyst] Catalyst benchmark 5.7 VS 5.8

I'm wondering if someone here did a benchmark between Catalyst 5.7 and 5.8

here is the result from our server: http://scsys.co.uk:8001/34323

the background is
Catalyst 5.7011 VS Catalyst 5.80013
CPU: Intel(R) Core(TM)2 Quad  CPU   Q8200  @ 2.33GHz
RAM: 4G
OS: Centos5

from the top, each httpd takes 20M more RAM in 5.8 compared with 5.7

5.7
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
22979 apache    16   0  167m 142m 4248 S 17.0  3.5   0:06.07 httpd

5.8
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
24813 apache    15   0  190m 165m 4000 S 15.6  4.1   0:02.56 httpd


in this case, I really can't let my boss agree me to upgrade the Catalyst.

is it normal? any thoughts?

Thanks.
--
Fayland Lam // http://www.fayland.org/

_______________________________________________
List: Catalyst@...
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/


_______________________________________________
List: Catalyst@...
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/

Re: Catalyst benchmark 5.7 VS 5.8

by Stuart Watt :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Toby Corkindale wrote:
(CentOS 5 was one of the operating systems that came with the badly-patched Perl with the slow bless performance..
although I'm sure it's been patched by now?
ie. http://blog.vipul.net/2008/08/24/redhat-perl-what-a-tragedy/
)
  
This issue is still listed as open for Centos; http://bugs.centos.org/view.php?id=2357

I was surprised at this, but I had checked a couple of months back and no progress. I gave up on Centos in 1997 because of this, and am surprised how long this is taking.

All the best
Stuart
--
Stuart Watt
ARM Product Developer
Information Balance

_______________________________________________
List: Catalyst@...
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/

Re: Catalyst benchmark 5.7 VS 5.8

by David Silva :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Sorry to interrupt, but that is a !!WOW!!

I used CentOS a few weeks ago and i have to say that it could be better if it was updated... But seems that they are with internal problems ...

2009/9/28 Stuart Watt <swatt@...>
Toby Corkindale wrote:
(CentOS 5 was one of the operating systems that came with the badly-patched Perl with the slow bless performance..
although I'm sure it's been patched by now?
ie. http://blog.vipul.net/2008/08/24/redhat-perl-what-a-tragedy/
)
  
This issue is still listed as open for Centos; http://bugs.centos.org/view.php?id=2357

I was surprised at this, but I had checked a couple of months back and no progress. I gave up on Centos in 1997 because of this, and am surprised how long this is taking.

All the best
Stuart
--
Stuart Watt
ARM Product Developer
Information Balance

_______________________________________________
List: Catalyst@...
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/




--
David Silva - http://davidslv.com/

_______________________________________________
List: Catalyst@...
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/

Re: Catalyst benchmark 5.7 VS 5.8

by Fayland Lam :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Toby Corkindale wrote:
> (Apologies for top-posting.. have momentarily lost the option to change quoting styles it seems..)
>
> Fayland, I was looking at the benchmarks that you linked, and was just wondering which version of Perl you're running against?
>
> (CentOS 5 was one of the operating systems that came with the badly-patched Perl with the slow bless performance..
> although I'm sure it's been patched by now?
> ie. http://blog.vipul.net/2008/08/24/redhat-perl-what-a-tragedy/
> )


Thanks for your update. but it doesn't help on the benchmark since they
are run on the same condition. so 5.7 is really better than 5.8 under siege.

Thanks.


>
> Cheers,
> Toby
>
> ----- Original Message -----
> From: Fayland Lam <fayland@...>
> To: catalyst@...
> Sent: Mon, 28 Sep 2009 15:56:36 +1000 (EST)
> Subject: [Catalyst] Catalyst benchmark 5.7 VS 5.8
>
> I'm wondering if someone here did a benchmark between Catalyst 5.7 and 5.8
>
> here is the result from our server: http://scsys.co.uk:8001/34323
>
> the background is
> Catalyst 5.7011 VS Catalyst 5.80013
> CPU: Intel(R) Core(TM)2 Quad  CPU   Q8200  @ 2.33GHz
> RAM: 4G
> OS: Centos5
>
> from the top, each httpd takes 20M more RAM in 5.8 compared with 5.7
>
> 5.7
>   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
> 22979 apache    16   0  167m 142m 4248 S 17.0  3.5   0:06.07 httpd
>
> 5.8
>   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
> 24813 apache    15   0  190m 165m 4000 S 15.6  4.1   0:02.56 httpd
>
>
> in this case, I really can't let my boss agree me to upgrade the Catalyst.
>
> is it normal? any thoughts?
>
> Thanks.


_______________________________________________
List: Catalyst@...
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/

Re: Catalyst benchmark 5.7 VS 5.8

by Fayland Lam :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Tomas Doran wrote:

> Fayland Lam wrote:
>> from the top, each httpd takes 20M more RAM in 5.8 compared with 5.7
>
> No, that'll be 20Mb of RAM _in total_, as all of those pages should be
> shared between your apache processes (given that you're preloading your
> application in the parent process).
>
> top totally doesn't show how much RAM is shared by copy on write at all,
> and so is misleading you here.
>

do you know how to do a real benchmark? the siege result shows 5.7 is
better under pressure.

Thanks.


> Cheers
> t0m
>
> _______________________________________________
> List: Catalyst@...
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive: http://www.mail-archive.com/catalyst@.../
> Dev site: http://dev.catalyst.perl.org/
>


_______________________________________________
List: Catalyst@...
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/

Re: Catalyst benchmark 5.7 vs 5.8 - new test

by Toby Corkindale-3 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Fayland Lam wrote:
> I'm wondering if someone here did a benchmark between Catalyst 5.7 and 5.8

I have a vested interest in knowing the difference between the two
versions as well, so knocked up a "proper" test.
I have two identical virtual machines, only on one I installed
Catalyst::Runtime 5.71001 and the other with 5.80013.

Running the exact same app, I hit them up with Siege for a while,
results follow at the end of this email.

If you want to replicate the test or examine my extremely-simple test
app, see: http://github.com/TJC/Catalyst-Performance-Test
(Patches gleefully accepted ;)

It's interesting to note the headline figures have 5.71 performing 316
tps, vs 5.80 making only 283 tps.
Memory usage (for this small app) has increased by 4MB, but is
presumably shared. I guess I should look into that more.

The same system can serve small static pages from the webserver at about
1900 tps. A real-world application there on Cat 5.8 gets 90 tps.

I don't see that performance difference (5.71 vs 5.80) as significant,
since most of your time ends up being spent in application code, rather
than the Catalyst framework itself.
ie. If you want to make your code go faster, look at optimising your
templating and database queries before you worry about downgrading Catalyst.

-Toby

------------------= results =----------------------
Running 10 second warmup on 5.7..
Running main test on 5.7..

Transactions:       94796 hits
Availability:      100.00 %
Elapsed time:      300.00 secs
Data transferred:       77.35 MB
Response time:        0.03 secs
Transaction rate:      315.99 trans/sec
Throughput:        0.26 MB/sec
Concurrency:       10.00
Successful transactions:       94796
Failed transactions:           0
Longest transaction:        0.98
Shortest transaction:        0.00

Process size:
101m VIRT, 34m RES



Running 10 second warmup on 5.8..
Running main test on 5.8..

Transactions:       84805 hits
Availability:      100.00 %
Elapsed time:      300.00 secs
Data transferred:       69.20 MB
Response time:        0.04 secs
Transaction rate:      282.68 trans/sec
Throughput:        0.23 MB/sec
Concurrency:        9.99
Successful transactions:       84805
Failed transactions:           0
Longest transaction:        1.07
Shortest transaction:        0.00

Process size:
103m VIRT, 38m RES

_______________________________________________
List: Catalyst@...
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/

Parent Message unknown Re: Re: Catalyst benchmark 5.7 VS 5.8

by Andrew Rodland :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Monday 28 September 2009 09:31:13 pm Fayland Lam wrote:

> Toby Corkindale wrote:
> > Fayland, I was looking at the benchmarks that you linked, and was just
> > wondering which version of Perl you're running against?
> >
> > (CentOS 5 was one of the operating systems that came with the
> > badly-patched Perl with the slow bless performance.. although I'm sure
> > it's been patched by now?
> > ie. http://blog.vipul.net/2008/08/24/redhat-perl-what-a-tragedy/
> > )
>
> Thanks for your update. but it doesn't help on the benchmark since they
> are run on the same condition. so 5.7 is really better than 5.8 under
> siege.
>
That doesn't follow. If your perl is broken and very slow at doing something
that 5.8 does much more often, then 5.8 can be slower for you without being
slower for everyone else.

I have a real app and a suitable test system at hand -- I'll run some benches
of my own and at least add a few more data points to the mix.

Andrew

_______________________________________________
List: Catalyst@...
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/

Re: Catalyst benchmark 5.7 VS 5.8

by Andrew Rodland :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Monday 28 September 2009 12:56:36 am Fayland Lam wrote:
> I'm wondering if someone here did a benchmark between Catalyst 5.7 and 5.8

Benchmark, as requested. View this message at http://p3m.org/pfn/3499 if your
mailer is too high-tech for fixed-width text.


The Setup:

Linux 2.6 OpenVZ, on Quad 2.2GHz Opteron.
Perl 5.10.0 + debian patches.
FastCGI via Apache 2.2, mpm_event, mod_fastcgi.
The two test instances were running on the same machine, with the same perl
and the same checkout of the app, but different local::lib directories.

FastCGI was set to 10 processes. The page I was hitting was from a real
checkout of a real production app, however it was the front page of the
site, which is fairly light on dynamic content. I figured this was
appropriate since it would better show any differences in Catalyst rather
than spending a lot of time in the backend. The code still hits several
models, 3 actions, and a view, but perhaps it was a little too fast since,
as you'll see below, my throughput was ultimately limited by the number of
running processes. Each instance was given a "warmup" run (the results of
which were discarded) before the following tests were run. My tool collects
statistics on the return status, but for all tests the returns were all 200
(success) so I've left out that row.

|| 20 requests/second (20 threads) for 60s
| Metric            || Catalyst 5.7010          || Catalyst 5.8011
|===================||==========================||==========================
| Hits              || 1200                     || 1200
| Throughput        || 20.00 req/s              || 20.00 req/s
| Latency (mean)    || 0.072s                   || 0.074s
| Latency (SD)      || 0.013s                   || 0.017s
| Latency (Q1-Q3)   || 0.064 - 0.078s           || 0.066 - 0.080s

|| 40 requests/second (40 threads) for 60s
| Metric            || Catalyst 5.7010          || Catalyst 5.8011
|===================||==========================||==========================
| Hits              || 2400                     || 2400
| Throughput        || 40.00 req/s              || 40.00 req/s
| Latency (mean)    || 0.083s                   || 0.088s
| Latency (SD)      || 0.020s                   || 0.024s
| Latency (Q1-Q3)   || 0.069 - 0.095s           || 0.072 - 0.100s

|| 80 requests/second (80 threads) for 60s
| Metric            || Catalyst 5.7010          || Catalyst 5.8011
|===================||==========================||==========================
| Hits              || 4675                     || 4637
| Throughput        || 77.92 req/s              || 77.28 req/s
| Latency (mean)    || 0.688s                   || 0.708s
| Latency (SD)      || 0.178s                   || 0.187s
| Latency (Q1-Q3)   || 0.617 - 0.800s           || 0.726 - 0.811s

The difference between 5.7 and 5.8 in these results is consistently in favor
of 5.7, but by a margin of between 0% and 5% which is not a whole lot in my
book. By my unscientific measure (i.e. looking at "top") of memory usage,
5.7 used 138MB of RAM (for fcgi-pm + 10x fcgi children) whereas 5.8 used
184MB, so that's a 33% expansion, which is a more significant issue. I have
a feeling that most of that is shared, and so the difference wouldn't
increase much with an increase in the number of processes, but I haven't
investigated that yet.

Questions?

Andrew "hobbs" Rodland

_______________________________________________
List: Catalyst@...
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/

Re: Catalyst benchmark 5.7 vs 5.8 - new test

by Toby Corkindale-3 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Toby Corkindale wrote:
> It's interesting to note the headline figures have 5.71 performing 316
> tps, vs 5.80 making only 283 tps.
> Memory usage (for this small app) has increased by 4MB, but is
> presumably shared. I guess I should look into that more.

Here are some new analysis of memory usage on my test app.

           5.7     5.8
Rss:     35260   38768
Private:  8244    9620
Shared:  27016   29148

So 5.8 is 3500kB larger, but 2000kB is shared. So we're only looking at
1500Kb more per process.

-Toby

_______________________________________________
List: Catalyst@...
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/

Re: Catalyst benchmark 5.7 VS 5.8

by Fayland Lam :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Thanks for that. (Toby Corkindale too)

I'll do more research and update you later.

Thanks.

Andrew Rodland wrote:

> On Monday 28 September 2009 12:56:36 am Fayland Lam wrote:
>> I'm wondering if someone here did a benchmark between Catalyst 5.7 and 5.8
>
> Benchmark, as requested. View this message at http://p3m.org/pfn/3499 if your
> mailer is too high-tech for fixed-width text.
>
>
> The Setup:
>
> Linux 2.6 OpenVZ, on Quad 2.2GHz Opteron.
> Perl 5.10.0 + debian patches.
> FastCGI via Apache 2.2, mpm_event, mod_fastcgi.
> The two test instances were running on the same machine, with the same perl
> and the same checkout of the app, but different local::lib directories.
>
> FastCGI was set to 10 processes. The page I was hitting was from a real
> checkout of a real production app, however it was the front page of the
> site, which is fairly light on dynamic content. I figured this was
> appropriate since it would better show any differences in Catalyst rather
> than spending a lot of time in the backend. The code still hits several
> models, 3 actions, and a view, but perhaps it was a little too fast since,
> as you'll see below, my throughput was ultimately limited by the number of
> running processes. Each instance was given a "warmup" run (the results of
> which were discarded) before the following tests were run. My tool collects
> statistics on the return status, but for all tests the returns were all 200
> (success) so I've left out that row.
>
> || 20 requests/second (20 threads) for 60s
> | Metric            || Catalyst 5.7010          || Catalyst 5.8011
> |===================||==========================||==========================
> | Hits              || 1200                     || 1200
> | Throughput        || 20.00 req/s              || 20.00 req/s
> | Latency (mean)    || 0.072s                   || 0.074s
> | Latency (SD)      || 0.013s                   || 0.017s
> | Latency (Q1-Q3)   || 0.064 - 0.078s           || 0.066 - 0.080s
>
> || 40 requests/second (40 threads) for 60s
> | Metric            || Catalyst 5.7010          || Catalyst 5.8011
> |===================||==========================||==========================
> | Hits              || 2400                     || 2400
> | Throughput        || 40.00 req/s              || 40.00 req/s
> | Latency (mean)    || 0.083s                   || 0.088s
> | Latency (SD)      || 0.020s                   || 0.024s
> | Latency (Q1-Q3)   || 0.069 - 0.095s           || 0.072 - 0.100s
>
> || 80 requests/second (80 threads) for 60s
> | Metric            || Catalyst 5.7010          || Catalyst 5.8011
> |===================||==========================||==========================
> | Hits              || 4675                     || 4637
> | Throughput        || 77.92 req/s              || 77.28 req/s
> | Latency (mean)    || 0.688s                   || 0.708s
> | Latency (SD)      || 0.178s                   || 0.187s
> | Latency (Q1-Q3)   || 0.617 - 0.800s           || 0.726 - 0.811s
>
> The difference between 5.7 and 5.8 in these results is consistently in favor
> of 5.7, but by a margin of between 0% and 5% which is not a whole lot in my
> book. By my unscientific measure (i.e. looking at "top") of memory usage,
> 5.7 used 138MB of RAM (for fcgi-pm + 10x fcgi children) whereas 5.8 used
> 184MB, so that's a 33% expansion, which is a more significant issue. I have
> a feeling that most of that is shared, and so the difference wouldn't
> increase much with an increase in the number of processes, but I haven't
> investigated that yet.
>
> Questions?
>
> Andrew "hobbs" Rodland
>
> _______________________________________________
> List: Catalyst@...
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive: http://www.mail-archive.com/catalyst@.../
> Dev site: http://dev.catalyst.perl.org/
>


_______________________________________________
List: Catalyst@...
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/

Re: Catalyst benchmark 5.7 VS 5.8

by Tobias Kremer-5 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Mon, Sep 28, 2009 at 12:23 PM, Tomas Doran <bobtfish@...> wrote:
> top totally doesn't show how much RAM is shared by copy on write at all, and
> so is misleading you here.

So, what's a better way to find out how much memory is shared? On our
production servers "top" shows

VIRT: 70116,  RES: 64m, SHR: 3480

and I hope that 3480 is really not the amount of memory that is shared
because that'd be quite low.

Thanks!

--Tobias

_______________________________________________
List: Catalyst@...
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/

Re: Catalyst benchmark 5.7 VS 5.8

by Carl Johnstone-4 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Toby Corkindale wrote:
> (CentOS 5 was one of the operating systems that came with the
> badly-patched Perl with the slow bless performance..
> although I'm sure it's been patched by now?
> ie. http://blog.vipul.net/2008/08/24/redhat-perl-what-a-tragedy/
> )

Was patched last year - stop spreading FUD.

The RHEL/CentOS perl build isn't the best one is the world but it's adequate
enough for most uses. Too much FUD will just scare decision makers who don't
understand the technical details and just see perl + RHEL = fail.

Carl


_______________________________________________
List: Catalyst@...
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/

Re: Catalyst benchmark 5.7 VS 5.8

by Carl Johnstone-4 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Tobias Kremer wrote:
> So, what's a better way to find out how much memory is shared? On our
> production servers "top" shows
>
> VIRT: 70116,  RES: 64m, SHR: 3480
>
> and I hope that 3480 is really not the amount of memory that is shared
> because that'd be quite low.

It's a different type of shared. That's amount of memory used through shared
OS libraries.

What everybody else in this thread is referring to as "shared" memory is
actually the amount of memory that hasn't needed to be duplicated because of
the copy-on-write semantics within the Linux kernel. Unfortunately there's
currently no easy way I know of to get these figures on Linux.

Carl


_______________________________________________
List: Catalyst@...
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/

Re: Catalyst benchmark 5.7 VS 5.8

by Andrew Rodland :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Tuesday 29 September 2009 03:44:33 am Carl Johnstone wrote:
> What everybody else in this thread is referring to as "shared" memory is
> actually the amount of memory that hasn't needed to be duplicated because
> of the copy-on-write semantics within the Linux kernel. Unfortunately
> there's currently no easy way I know of to get these figures on Linux.

In my experience with linux+perl+fastcgi, "VmHWM - (VmExe + VmLib)" (stats
from /proc/*/status) gives a reasonable upper bound on the incremental cost of
a process -- that is, how much your "free memory" total will go up or down if
you add or remove one process. It's a loose upper bound because the stuff that
it knows to subtract is quite unlikely to become unshared, but there's a bunch
of data that's probably being shared that it doesn't know about. For an
example: the formula above gives 14,900kB when applied to an instance of a
beta app that I happen to have at hand; actually killing an instance gives an
increase in free memory (as reported by "free") of 11,400kB, so the error is
3,500kB. The error depends on the app, but it's otherwise pretty consistent,
which makes this a good enough tool to use for a "kill processes that start
chewing memory" script.

Andrew

_______________________________________________
List: Catalyst@...
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/

Re: Catalyst benchmark 5.7 VS 5.8

by Tomas Doran :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Tobias Kremer wrote:
> and I hope that 3480 is really not the amount of memory that is shared
> because that'd be quite low.

Shared memory indicates things which are shared at a library linking
level (e.g. libc is a shared object which both processes will share).

This has nothing to do with the copy-on-write memory which is shared.
(I.e. your parent process loads the application, then calls fork - each
process sees 'its own copy' of all the RAM, but only pages written to
are copied - the rest remain "shared").

Cheers
t0m


_______________________________________________
List: Catalyst@...
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/

Re: Re: Catalyst benchmark 5.7 VS 5.8

by Tomas Doran :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Fayland Lam wrote:
> Tomas Doran wrote:

>> top totally doesn't show how much RAM is shared by copy on write at
>> all, and so is misleading you here.
>
> do you know how to do a real benchmark? the siege result shows 5.7 is
> better under pressure.

I didn't actually do any 'real' benchmarking for this, I just pointed
out that the conclusions you were drawing from top were rubbish.

Andrew and Tobys memory benchmarks make sense and are much nearer what
I'd expect.

1.5Mb more per process is much more reasonable than the 20Mb you were
speculating initially.

Cheers
t0m


_______________________________________________
List: Catalyst@...
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/

Re: Catalyst benchmark 5.7 vs 5.8 - new test

by Tomas Doran :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Toby Corkindale wrote:
> It's interesting to note the headline figures have 5.71 performing 316
> tps, vs 5.80 making only 283 tps.

The very important thing you haven't noted (unless I missed it) is what
perl version this benchmark was conducted under.

Some benchmarking was done before 5.8 was released, and it showed that
Catalyst 5.7 was (marginally) quicker on perl 5.8 and that Catalyst 5.8
was (marginally) quicker on perl 5.10 :)

So I'd be very interested to see the benchmark repeated with two
different perls, giving 4 results to compare and contrast.

Cheers
t0m

_______________________________________________
List: Catalyst@...
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/

Re: Catalyst benchmark 5.7 VS 5.8

by Dave Rolsky :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Tue, 29 Sep 2009, Carl Johnstone wrote:

> What everybody else in this thread is referring to as "shared" memory is
> actually the amount of memory that hasn't needed to be duplicated because of
> the copy-on-write semantics within the Linux kernel. Unfortunately there's
> currently no easy way I know of to get these figures on Linux.

http://www.selenic.com/smem/

This can give you some good numbers on shared memory.


-dave

/*============================================================
http://VegGuide.org               http://blog.urth.org
Your guide to all that's veg      House Absolute(ly Pointless)
============================================================*/

_______________________________________________
List: Catalyst@...
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@.../
Dev site: http://dev.catalyst.perl.org/
< Prev | 1 - 2 | Next >