Nabble - RRDTool - User

Re: Showing daily average ; from midnight to midnight local time

2009-12-09T02:36:56Z

Jean-Yves Avenard wrote:

>For the power usage ; I want to show the data as follow:
>daily: midnight to midnight local time
>weekly: midnight of day -7 to midnight previous day local time
>monthly: midnight of day-30 to midnight previous day local time
>yearly: doesn't really matter.
>
>daily view is averaged per hour
>weekly view is averaged per day
>monthly view is averaged per day
>yearly view is averaged per month (30 days)
>
>I created the RRD with data being fed every minute and keeping
>consolidated data as follow:
>100 days of 1 minute average
>5 years of 5 minutes average
>5 years of 30 minutes average
>5 years of 1 hour average
>5 years of 2 hours average
>5 years of daily average
>
>Everything is nice and rosy but for the weekly and monthly view...
>The average isn't calculated from midnight to midnight local time ;
>even though I provide end and start data at those values.
>
>For the weekly view, the x-grid is 'HOUR:3:DAY:1:HOUR:24:86400:%a'
>As you can see on the weekly graph:
>http://htpc.avenard.org/power/?graph=2&view=week
>A plot doesn't start at midnight ; but around 11AM.
>
>I understand that this is UTC time ; I read
>http://www.vandenbogaerdt.nl/rrdtool/timezone.php and in particular
>this thread:
>http://groups.google.com/group/mailing.unix.rrd-users/browse_thread/thread/326e32b152c9f6a9
>
>Is there any way to make the plotted bar goes from midnight to
>midnight local time, and calculate the average of the data of a day
>(local time)?
>If I set TZ to UTC ; then obviously the x axis matches the graph
>perfectly ; but that's not what I want to do...

I believe the answer to that was given in the second of those links
you posted. Essentially, you'll have to store data with a resolution
of no worse than 1 hour, then use rrdtools to get data out by your
wallclock days, stuff those values into another rrd database, and
then graph those.

--
Simon Hobson

Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed
author Gladys Hobson. Novels - poetry - short stories - ideal as
Christmas stocking fillers. Some available as e-books.

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Showing daily average ; from midnight to midnight local time

2009-12-08T16:27:41Z

Hi there...

Ok, I have finally the hang of most of what I intended to do.

Logging my power usage ; then calculating power consumption as charged
by the electricity company (in kWh).

http://htpc.avenard.org/power/

For each graph; you can click on a part of the graph ; and it goes
down to one resolution down (e.g weekly view gives you daily view
etc...)

For the power usage ; I want to show the data as follow:
daily: midnight to midnight local time
weekly: midnight of day -7 to midnight previous day local time
monthly: midnight of day-30 to midnight previous day local time
yearly: doesn't really matter.

daily view is averaged per hour
weekly view is averaged per day
monthly view is averaged per day
yearly view is averaged per month (30 days)

I created the RRD with data being fed every minute and keeping
consolidated data as follow:
100 days of 1 minute average
5 years of 5 minutes average
5 years of 30 minutes average
5 years of 1 hour average
5 years of 2 hours average
5 years of daily average

Everything is nice and rosy but for the weekly and monthly view...
The average isn't calculated from midnight to midnight local time ;
even though I provide end and start data at those values.

For the weekly view, the x-grid is 'HOUR:3:DAY:1:HOUR:24:86400:%a'
As you can see on the weekly graph:
http://htpc.avenard.org/power/?graph=2&view=week
A plot doesn't start at midnight ; but around 11AM.

I understand that this is UTC time ; I read
http://www.vandenbogaerdt.nl/rrdtool/timezone.php and in particular
this thread:
http://groups.google.com/group/mailing.unix.rrd-users/browse_thread/thread/326e32b152c9f6a9

Is there any way to make the plotted bar goes from midnight to
midnight local time, and calculate the average of the data of a day
(local time)?
If I set TZ to UTC ; then obviously the x axis matches the graph
perfectly ; but that's not what I want to do...

Thank you...
Jean-Yves

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: Performance over NFS

2009-12-08T04:35:18Z

Tobias Oetiker wrote:

> > >also try exporting nfs with the async option ... less of a data
>> >integrity guarantee, but it should be faster ...
>>
>> As I read the man page, that'll only affect writes. I've exported the
>> directory as read-only - and it doesn't appear to have any effect.
>
>ah it is the graphing which is slow not the updating ?

Yes, sorry if I didn't make that clear. Updates are fine (local
filesystem), graphing is slow.

Just as a check, I did a simple copy of the file from one machine to
the other :
Via NFS it took around 90 to 120 seconds, via SCP around 30-35
seconds. A bit slow for a 22MByte file ... hmm, 648kbyte/s, that
sounds rather close to our internet connection speed ...

At which point, that clinking noise is the sound of the penny
dropping ! A while ago, as well as doing some upgrades I installed
keepalived and setup two routers in an active/spare setup. We do
traffic shaping on the routers, and the traffic shaping is configured
to allow (in practical terms) unrestricted rate from the routers IP
addresses*. And guess what I forgot to do when I changed the IP
address :-/

Strangely, now I've fixed the traffic shaping, things are back to
normal (copying the file takes about 4s) - so it wasn't NFS at all.

I'll just wander off and clean this egg off my face then ...

* To shape inbound traffic, we shape egress on the internal
interface, but of course we need to allow for traffic that originates
on the firewall itself. By forgetting to change the traffic shaping
setup, we were including the rrd data in the inbound traffic shaping.

--
Simon Hobson

Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed
author Gladys Hobson. Novels - poetry - short stories - ideal as
Christmas stocking fillers. Some available as e-books.

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: Performance over NFS

2009-12-08T02:09:31Z

Hi Simon,

Today Simon Hobson wrote:

> Tobias Oetiker wrote:
>
> >try disabeling mmap io
>
> Am I right in thinking that's a compile time option ? Also, is mmap
> one of the feature that appeared in 1.3 - so version 1.2.15 won't be
> using it. Does mmap on the hosting side (rrdtool updating the file)
> have an effect ?

there was mmap in use in 1.2 as well only less ...

> >also try exporting nfs with the async option ... less of a data
> >integrity guarantee, but it should be faster ...
>
> As I read the man page, that'll only affect writes. I've exported the
> directory as read-only - and it doesn't appear to have any effect.

ah it is the graphing which is slow not the updating ?

cheers
tobi

>
>

--
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
http://it.oetiker.ch tobi@... ++41 62 775 9902 / sb: -9900

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: Performance over NFS

2009-12-08T02:05:47Z

Tobias Oetiker wrote:

>try disabeling mmap io

Am I right in thinking that's a compile time option ? Also, is mmap
one of the feature that appeared in 1.3 - so version 1.2.15 won't be
using it. Does mmap on the hosting side (rrdtool updating the file)
have an effect ?

>also try exporting nfs with the async option ... less of a data
>integrity guarantee, but it should be faster ...

As I read the man page, that'll only affect writes. I've exported the
directory as read-only - and it doesn't appear to have any effect.

--
Simon Hobson

Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed
author Gladys Hobson. Novels - poetry - short stories - ideal as
Christmas stocking fillers. Some available as e-books.

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: Simple questions about RRD

2009-12-07T12:31:41Z

Also I don't know how relevant this is but the file you created is
currentcost3.rrd
and you are reading values from
currentcost.rrd

-David

-----Original Message-----
From: rrd-users-bounces@... [mailto:rrd-users-bounces@...] On Behalf Of Jean-Yves Avenard
Sent: Monday, December 07, 2009 3:34 AM
To: A Darren Dunham
Cc: rrd-users@...
Subject: Re: [rrd-users] Simple questions about RRD

Hi..

Sorry, I'm bumping that one ; as even after investigating for quite a
few days ; I can't figure out how the maximum is calculated in the
graph...

2009/12/6 Jean-Yves Avenard <jyavenard@...>:

> Looking at:
> http://htpc.avenard.org/power/screen.png
>
> It shows the maximum as being 2972W ; which is indeed the maximum with
> the actual data at the highest resolution.
> But it's definitely not the maximum of 5 minutes average as the graph
> shows: nothing is over 2200W
>

I've re-created the RRD with:
100 day of 1 minute average,min,max
5 years of 5 minutes average,min,max
5 years of 30 minutes average,min,max
5 years of 2 hours average,min,max
5 years of daily average,min,max

rrdtool create currentcost3.rrd -s 60 \
DS:total:GAUGE:300:0:U \
DS:ch1:GAUGE:300:0:U \
DS:ch2:GAUGE:300:0:U \
DS:ch3:GAUGE:300:0:U \
RRA:AVERAGE:0.5:1:144000 \
RRA:AVERAGE:0.5:5:525600 \
RRA:AVERAGE:0.5:30:87840 \
RRA:AVERAGE:0.5:120:21960 \
RRA:AVERAGE:0.5:1440:1830 \
RRA:MIN:0.5:1:144000 \
RRA:MIN:0.5:5:525600 \
RRA:MIN:0.5:30:87840 \
RRA:MIN:0.5:120:21960 \
RRA:MIN:0.5:1440:1830 \
RRA:MAX:0.5:1:144000 \
RRA:MAX:0.5:5:525600 \
RRA:MAX:0.5:30:87840 \
RRA:MAX:0.5:120:21960 \
RRA:MAX:0.5:1440:1830

> That graph is created with :
>
> $ret = exec("$RRDTOOL graph $name -l 0 \
> -t '$title' \
> -x $legend \
> --step $res --start e-$start --end $timestamp \
> -w $width -h $height \
> DEF:total=currentcost.rrd:total:AVERAGE DEF:ch2=currentcost.rrd:ch2:AVERAGE \
> DEF:solar=solarprod.rrd:total:AVERAGE DEF:ch1=currentcost.rrd:ch1:AVERAGE \
> DEF:totalmin=currentcost.rrd:total:MIN:reduce=AVERAGE
> DEF:ch2min=currentcost.rrd:ch2:MIN:reduce=AVERAGE \
> DEF:solarmin=solarprod.rrd:total:MIN:reduce=AVERAGE
> DEF:ch1min=currentcost.rrd:ch1:MIN:reduce=AVERAGE \
> DEF:totalmax=currentcost.rrd:total:MAX:reduce=AVERAGE
> DEF:ch2max=currentcost.rrd:ch2:MAX:reduce=AVERAGE \
> DEF:solarmax=solarprod.rrd:total:MAX:reduce=AVERAGE
> DEF:ch1max=currentcost.rrd:ch1:MAX:reduce=AVERAGE \
> VDEF:ds0max=ch2max,MAXIMUM \
> VDEF:ds0avg=ch2,AVERAGE \
> VDEF:ds0min=ch2min,MINIMUM \
> VDEF:ds0pct=ch2,95,PERCENT \
> VDEF:ds1max=totalmax,MAXIMUM \
> VDEF:ds1avg=total,AVERAGE \
> VDEF:ds1min=totalmin,MINIMUM \
> VDEF:ds1pct=total,95,PERCENT \
> VDEF:ds2max=ch1max,MAXIMUM \
> VDEF:ds2avg=ch1,AVERAGE \
> VDEF:ds2min=ch1min,MINIMUM \
> VDEF:ds2pct=ch1,95,PERCENT \
> VDEF:ds3max=solarmax,MAXIMUM \
> VDEF:ds3avg=solar,AVERAGE \
> VDEF:ds3min=solarmin,MINIMUM \
> VDEF:ds3pct=solar,95,PERCENT \
> COMMENT:' ' \
> COMMENT:'Maximum ' \
> COMMENT:'Average ' \
> COMMENT:'Minimum ' \
> COMMENT:'95th percentile\l' \
> LINE1:total#000090:'Total ' \
> GPRINT:ds1max:'%5.0lf W ' \
> GPRINT:ds1avg:'%5.0lf W ' \
> GPRINT:ds1min:'%5.0lf W ' \
> GPRINT:ds1pct:' %5.0lf W\l' \
> LINE1:ch1#00FF00:'Pool ' \
> GPRINT:ds2max:'%5.0lf W ' \
> GPRINT:ds2avg:'%5.0lf W ' \
> GPRINT:ds2min:'%5.0lf W ' \
> GPRINT:ds2pct:' %5.0lf W\l' \
> LINE1:solar#FF0000:'Solar ' \
> GPRINT:ds3max:'%5.0lf W ' \
> GPRINT:ds3avg:'%5.0lf W ' \
> GPRINT:ds3min:'%5.0lf W ' \
> GPRINT:ds3pct:' %5.0lf W\l'
>
> where $res , the argument for --step is 300 (5 minutes) and I do use
> the reduce=AVERAGE option.
>

timestamp is on a multiple of 300 (created in php with: $timestamp =
(floor(mktime() / $res) * $res)

MAX on the graph still shows up at 2972W ; the graph itself shows the
max to be around 2200W.
When I use rrdtool fetch ; and check the maximum ; the value is indeed 2170W...

So why does graph reports the maximum not of the resolution wanted ;
but of the highest level resolution ? (the one that is 1 min average)

I've tried with:
> DEF:totalmax=currentcost.rrd:total:MAX:reduce=AVERAGE
and
> DEF:totalmax=currentcost.rrd:total:MAX:step=300:reduce=AVERAGE
defining --step or not ; always getting the incorred 2972W.

Any help would be appreciated..

Thanks
Jean-Yves

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: Simple questions about RRD

2009-12-07T11:46:57Z

Jean-Yves Avenard wrote:

>It shows the maximum as being 2972W ; which is indeed the maximum with
>the actual data at the highest resolution.
>But it's definitely not the maximum of 5 minutes average as the graph
>shows: nothing is over 2200W
>
>That graph is created with :
>
>$ret = exec("$RRDTOOL graph $name -l 0 \
>-t '$title' \
>-x $legend \
>--step $res --start e-$start --end $timestamp \
>-w $width -h $height \
>DEF:total=currentcost.rrd:total:AVERAGE DEF:ch2=currentcost.rrd:ch2:AVERAGE \
>DEF:solar=solarprod.rrd:total:AVERAGE DEF:ch1=currentcost.rrd:ch1:AVERAGE \
>DEF:totalmin=currentcost.rrd:total:MIN:reduce=AVERAGE
>DEF:ch2min=currentcost.rrd:ch2:MIN:reduce=AVERAGE \
>DEF:solarmin=solarprod.rrd:total:MIN:reduce=AVERAGE
>DEF:ch1min=currentcost.rrd:ch1:MIN:reduce=AVERAGE \
>DEF:totalmax=currentcost.rrd:total:MAX:reduce=AVERAGE
>DEF:ch2max=currentcost.rrd:ch2:MAX:reduce=AVERAGE \
>DEF:solarmax=solarprod.rrd:total:MAX:reduce=AVERAGE
>DEF:ch1max=currentcost.rrd:ch1:MAX:reduce=AVERAGE \

Why are you using reduce=AVERAGE on a maximum ?

Looking back at your first post, it looks like your DS is at 60s
intervals, therefore a graphing interval of 300s will require further
consolidation. The data in the graphs is quite peaky, so this is
likely to have a significant effect - eg at around time 21.5 there is
a quite a narrow peak and it's quite likely that the average of 5 max
values is considerably less than the max of 5 values. For example, it
could be 4 samples of 0.6k and one of 5.6k, or it could be 5 samples
of 1.6k - quite a difference.

Having said that, I can't see where the difference is coming from,
unless the VDEF is using the raw data before it's been consolidated -
someone would have to look at the source to see if that's possible.

I will say that it can make things a lot easier to see if you create
a file of test data - carefully chosen to exercise the functionality
in doubt. So perhaps something like this :

1260214800:1
1260214860:1
1260214920:6
1260214980:1
1260215040:1
1260215100:1

Now that should produce the following values when consolidated over
the 5 minutes from 1260214800 to 1260215100 :
min = 1
ave = 2
max = 6

>Now I have an extra question on how the RRA/consolidation works...
>
>Say I store 5 years of 5 minutes average ; is there any points of also
>storing 5 years of 30 minutes average, 5 years of 2 hours average etc?
>
>I would have assumed that it wouldn't matter ; except when retrieving
>the 30 minutes/2 hours average extra processing is required as it
>needs to retrieve more values.
>
>Now; if I use reduce=AVERAGE for all my graph like
>DEF:total=currentcost.rrd:total:AVERAGE DEF:ch2=currentcost.rrd:ch2:AVERAGE \
>DEF:solar=solarprod.rrd:total:AVERAGE DEF:ch1=currentcost.rrd:ch1:AVERAGE \
>DEF:totalmin=currentcost.rrd:total:MIN:reduce=AVERAGE
>DEF:ch2min=currentcost.rrd:ch2:MIN:reduce=AVERAGE \
>DEF:solarmin=solarprod.rrd:total:MIN:reduce=AVERAGE
>DEF:ch1min=currentcost.rrd:ch1:MIN:reduce=AVERAGE \
>DEF:totalmax=currentcost.rrd:total:MAX:reduce=AVERAGE
>DEF:ch2max=currentcost.rrd:ch2:MAX:reduce=AVERAGE \
>DEF:solarmax=solarprod.rrd:total:MAX:reduce=AVERAGE
>DEF:ch1max=currentcost.rrd:ch1:MAX:reduce=AVERAGE \
>
>Do I even need to create the RRD with MIN and MAX ? or it can be all
>done from the AVERAGE RRD database ?
>In my case, I create the RRD with:
>100 day of 1 minute average
>5 years of 5 minutes average
>
>rrdtool create currentcost.rrd -s 60 \
>DS:total:GAUGE:300:0:U \
>DS:ch1:GAUGE:300:0:U \
>DS:ch2:GAUGE:300:0:U \
>DS:ch3:GAUGE:300:0:U \
>RRA:AVERAGE:0.5:1:144000 \
>RRA:AVERAGE:0.5:5:525600 \
>RRA:MIN:0.5:1:144000 \
>RRA:MIN:0.5:5:525600 \
>RRA:MAX:0.5:1:144000 \
>RRA:MAX:0.5:5:525600
>
>Not creating a MIN and MAX one would reduce considerably the size rrd file.

You seem to have some very woolly thinking there. If you only store
average, then you cannot possibly derive min and max from
consolidated data. Eg, suppose someone boils the kettle and it takes
3kW for one minute (and it happens to coincide nicely with one sample
period). You could have a 5 minute period with values of 3,0,0,0,0
which average to 0.6. There is no way to know the minimum other than
it's <= 0.6, and no way to know the max other than it's >= 0.6. In
this case they are 0 and 3, but they could just as well be 0.6 if it
was a steady load.

Extend that to 30 minute and 2 hour consolidations and the difference
widens - eg you boil the kettle but otherwise use no power and the
max is still 3, the min is still 0, but the average is now only 0.1
and 0.025 respectively.

Now, if you are only interested in 5 minute smoothed data and NOT 1
minute min and max, then you are correct that you don't need to keep
separate min and max for a ds - provided you do not keep or try to
graph data from any consolidation of that.

There are two main reasons people use consolidated storage :

1) To reduce storage requirements. Most people aren't bothered by the
fine detail once it's moderately aged. So for example, at work I keep
5 minute samples for only 2 days, I keep 1/2 hour consolidations for
longer, 2 hour consolidations for longer still, and 1 day
consolidations for 2 years. So I can graph in detail, or over a long
time, but not both - I can't plot a details graph for data a year old
and that's fine by us.

2) Reduce processing to generate graphs. Naturally, if you plot a
year long graph with 5 minute samples then the graphing program has
to read in and consolidate a lot of data - and that takes time and
memory. A while ago I realised I'd made a mistake and was storing 12
hour data rather than 24 hour data - and graphing at 24 hour
resolution. I found that re-working things significantly decreased
runtime and memory requirements - particularly on the complex graphs
with hundreds of data sources in them.

--
Simon Hobson

Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed
author Gladys Hobson. Novels - poetry - short stories - ideal as
Christmas stocking fillers. Some available as e-books.

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: Simple questions about RRD

2009-12-07T10:49:09Z

On Sun, Dec 06, 2009 at 12:19:36AM +1100, Jean-Yves Avenard wrote:

> 2009/12/5 A Darren Dunham <ddunham@...>:
> >
> > Hopefully that demonstrates the differences pretty well.
> >
>
> Actually ; looking at the graph ; I can't understand how the MAX is
> actually calculated.
>
> Looking at:
> http://htpc.avenard.org/power/screen.png
>
> It shows the maximum as being 2972W ; which is indeed the maximum with
> the actual data at the highest resolution.
> But it's definitely not the maximum of 5 minutes average as the graph
> shows: nothing is over 2200W
>
> That graph is created with :
>
> $ret = exec("$RRDTOOL graph $name -l 0 \
> -t '$title' \
> -x $legend \
> --step $res --start e-$start --end $timestamp \
> -w $width -h $height \
> DEF:total=currentcost.rrd:total:AVERAGE DEF:ch2=currentcost.rrd:ch2:AVERAGE \
> DEF:solar=solarprod.rrd:total:AVERAGE DEF:ch1=currentcost.rrd:ch1:AVERAGE \
> DEF:totalmin=currentcost.rrd:total:MIN:reduce=AVERAGE
> DEF:ch2min=currentcost.rrd:ch2:MIN:reduce=AVERAGE \
> DEF:solarmin=solarprod.rrd:total:MIN:reduce=AVERAGE
> DEF:ch1min=currentcost.rrd:ch1:MIN:reduce=AVERAGE \
> DEF:totalmax=currentcost.rrd:total:MAX:reduce=AVERAGE
> DEF:ch2max=currentcost.rrd:ch2:MAX:reduce=AVERAGE \
> DEF:solarmax=solarprod.rrd:total:MAX:reduce=AVERAGE
> DEF:ch1max=currentcost.rrd:ch1:MAX:reduce=AVERAGE \
> VDEF:ds0max=ch2max,MAXIMUM \
> VDEF:ds0avg=ch2,AVERAGE \

[snip lots more stuff]

Easiest way for others to help is if it's really easy to reproduce it.
Can you create a minimal RRD create setup (probably don't need all your
production DEFs) and some canned data that shows the issue? The exact
data that enters the RRD is often the key to what's going on.

--
Darren

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: Simple questions about RRD

2009-12-07T00:34:06Z

Hi..

Sorry, I'm bumping that one ; as even after investigating for quite a
few days ; I can't figure out how the maximum is calculated in the
graph...

2009/12/6 Jean-Yves Avenard <jyavenard@...>:

> Looking at:
> http://htpc.avenard.org/power/screen.png
>
> It shows the maximum as being 2972W ; which is indeed the maximum with
> the actual data at the highest resolution.
> But it's definitely not the maximum of 5 minutes average as the graph
> shows: nothing is over 2200W
>

I've re-created the RRD with:
100 day of 1 minute average,min,max
5 years of 5 minutes average,min,max
5 years of 30 minutes average,min,max
5 years of 2 hours average,min,max
5 years of daily average,min,max

rrdtool create currentcost3.rrd -s 60 \
DS:total:GAUGE:300:0:U \
DS:ch1:GAUGE:300:0:U \
DS:ch2:GAUGE:300:0:U \
DS:ch3:GAUGE:300:0:U \
RRA:AVERAGE:0.5:1:144000 \
RRA:AVERAGE:0.5:5:525600 \
RRA:AVERAGE:0.5:30:87840 \
RRA:AVERAGE:0.5:120:21960 \
RRA:AVERAGE:0.5:1440:1830 \
RRA:MIN:0.5:1:144000 \
RRA:MIN:0.5:5:525600 \
RRA:MIN:0.5:30:87840 \
RRA:MIN:0.5:120:21960 \
RRA:MIN:0.5:1440:1830 \
RRA:MAX:0.5:1:144000 \
RRA:MAX:0.5:5:525600 \
RRA:MAX:0.5:30:87840 \
RRA:MAX:0.5:120:21960 \
RRA:MAX:0.5:1440:1830

Re: Questions of new user

2009-12-07T00:00:46Z

Pavel Rubtsov wrote:

>Hm... Ok, but:
>
>01. 1260111600 Start period
>02. 1260111900
>03. 1260112200
>04. 1260112500
>05. 1260112800
>06. 1260113100
>07. 1260113400
>08. 1260113700
>09. 1260114000
>10. 1260114300
>11. 1260114600
>12. 1260114900
>?? 13. 1260115200 End period?
>
>Why 13 steps?

It isn't 13, it's 12 !

12. 1260111600 End of previous consolidated period.
01. 1260111900
02. 1260112200
03. 1260112500
04. 1260112800
05. 1260113100
06. 1260113400
07. 1260113700
08. 1260114000
09. 1260114300
10. 1260114600
11. 1260114900
12. 1260115200 End period

At each timestamp, you have the value for the previous interval -
rrdtool only talks about history, it never tries to predict the
future. So when you do the update for 1260115200, that supplies data
for the period starting at 1260114900 and ending at 1260115200, and
that in turn gives the data required to consolidate for the period
starting at 1260111600 and ending at 1260115200.

I agree, it can take a little while to get your head round these
details. The key to remember is that rrdtool only ever deals with
rates over periods, and each timestamp in the output relates to the
END of a period.

--
Simon Hobson

Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed
author Gladys Hobson. Novels - poetry - short stories - ideal as
Christmas stocking fillers. Some available as e-books.

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Smoother graph with rrdtool?

2009-12-06T20:51:12Z

Hi.

The graph produced by RRDtool ; while accurate aren't really great
looking graph, lots of broken lines :P

Is there a way to smooth the graph out , like a bezier line between
points ? Like what facebook or google does for their statistic
reports?

Thanks
Jean-Yves

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: RRDs perl module question for Perl 5.10.1

2009-12-06T16:30:37Z

>
> $ perldoc perllocal:
> Insecure dependency in require while running with -T switch at
> /opt/sdr/report/perl/lib/5.10.1/Pod/Perldoc.pm line 1548.
>

RRDp,s builds fine on Solaris 10 sparc, x86. After starting
a fresh xterm I was able to perldoc, without any problems.

Cheers,
Stefan

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: remOcular - what tobi has been up to ...

2009-12-06T15:11:31Z

Hi

2009/12/7 Tobias Oetiker <tobi@...>:

> Over the last few weeks I have spent a lot of time hacking on a new
> project
>
> remOcular - your eyes in the cloud
>
> The first test version is now online on
>
> http://tobi.oetiker.ch/test/remocular/build/
>
> remOcular is an indirect decendent of SmokeTrace. It can run comand
> line tools remotely and provides a live presentation of their
> output in the webbrowser.

Very impressive stuff ; looks ace ...

The main logo says "your eyes in the could"
I think it meant to say "your eyes in the cloud"

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

remOcular - what tobi has been up to ...

2009-12-06T14:55:43Z

Over the last few weeks I have spent a lot of time hacking on a new
project

remOcular - your eyes in the cloud

The first test version is now online on

http://tobi.oetiker.ch/test/remocular/build/

remOcular is an indirect decendent of SmokeTrace. It can run comand
line tools remotely and provides a live presentation of their
output in the webbrowser.

remOcular ...

. has a very powerful api for writing new backend plugins to integrate
additional command lined tools. Developement time for a new
plugin can be as little as 10 minutes.

. consists of a simple backend written in perl and a interactive
frontend written in pure qooxdoo javascript.

. can show nifty inline graphs using canvas
graphics (only with ff, chrome, safari, opera)

. will be released as GPL opensource software in the next few
days.

cheers
tobi

--
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
http://it.oetiker.ch tobi@... ++41 62 775 9902 / sb: -9900

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: Questions of new user

2009-12-06T09:46:33Z

кто: Sun, 6 Dec 2009 16:25:10 +0000
когда: Simon Hobson <linux@...>

> >Result:
> >1260111600: nan
> >1260115200: 4.1111111111e+01
> >1260118800: nan
> >
> >But (10+20+...+50+40)/12 = 48,3! Why?
>
> You have your periods wrong - your first completed consolidation
> period is at 1260115200 (Sun 6 Dec 2009 16:00:00 GMT) and started at
> 1260111600 (Sun 6 Dec 2009 15:00:00 GMT).

Hm... Ok, but:

01. 1260111600 Start period
02. 1260111900
03. 1260112200
04. 1260112500
05. 1260112800
06. 1260113100
07. 1260113400
08. 1260113700
09. 1260114000
10. 1260114300
11. 1260114600
12. 1260114900
?? 13. 1260115200 End period?

Why 13 steps?
I was
>RRA:AVERAGE:0.5:12:1

Thanks for explanation!

--
Pavel Rubtsov <qweion@...>
Paused: Pavliga - Stop (2006 Edit)

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

RRDs perl module question for Perl 5.10.1

2009-12-06T09:03:42Z

Hi,

Im trying to understand whats going on here. I have built perl 5.10.1
on Solaris 10U8 x86, like this:

sh Configure -des -Dprefix=/opt/sdr/report/perl

and did install as well some CPAN modules: HTML::CalendarMonth etc.
Everything is fine.

When Im trying to add RRDs, RRDp modules
from RRDtool 1.4.2 I get the following error when running

$ perldoc perllocal:
Insecure dependency in require while running with -T switch at
/opt/sdr/report/perl/lib/5.10.1/Pod/Perldoc.pm line 1548.

The RRDs modules installs ok and I can use it too.

If I remove the RRDs module [1] perldoc perllocal
seems to work again. Any ideas whats going on here ?
Is perl complaining about some security config of certain files ?
Permissions, user id credentials seem to be fine.

Thanks a lot,
Stefan

[1] RRDs module
# make install
Files found in blib/arch: installing files in blib/lib into architecture
dependent library tree
Installing
/opt/sdr/report/perl/lib/site_perl/5.10.1/i86pc-solaris/auto/RRDs/RRDs.so
Installing
/opt/sdr/report/perl/lib/site_perl/5.10.1/i86pc-solaris/auto/RRDs/RRDs.bs
Installing /opt/sdr/report/perl/lib/site_perl/5.10.1/i86pc-solaris/RRDs.pm
Installing /opt/sdr/report/perl/man/man3/RRDs.3

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: Questions of new user

2009-12-06T08:25:10Z

Pavel Rubtsov wrote:

>But there is some misunderstanding:
>
>C:\rrd\rrdtool.exe create test.rrd --start N --step 300 ^
>DS:test:GAUGE:600:0:100 ^
>RRA:AVERAGE:0.5:12:1
>
>C:\rrd\rrdupdate.exe test.rrd ^
>1260113100:10 ^
>1260113400:20 ^
>1260113700:30 ^
>1260114000:40 ^
>1260114300:50 ^
>1260114600:60 ^
>1260114900:70 ^
>1260115200:80 ^
>1260115500:70 ^
>1260115800:60 ^
>1260116100:50 ^
>1260116400:40
>
>Result:
>1260111600: nan
>1260115200: 4.1111111111e+01
>1260118800: nan
>
>But (10+20+...+50+40)/12 = 48,3! Why?

You have your periods wrong - your first completed consolidation
period is at 1260115200 (Sun 6 Dec 2009 16:00:00 GMT) and started at
1260111600 (Sun 6 Dec 2009 15:00:00 GMT). So you only have 9
complete step periods for which you have data. Remember that all
intervals are "n * s" from unix epoch - ie a whole multiple of the
step size (or in this case, step size times steps per consolidated
data point). Put simply, if the step (or ocnsolidation) is one hour,
then all intervals start on the hour - nn:00:00.

You have the following values in the consolidated interval :

Unknown * 3
10
10
20
30
40
50
60
70
80
So for the average rate for this period is (10+10+20+..+70+80)/9 = 41.1111

Or really, the calculation is :
(10* 300 + 10*300 + 20*300 + 30*300 + ... + 70*300 + 80*300) / 2700
Remember that rrdtool only ever works in rates, so a gauge value of
30 for 300 seconds is really a rate of 0.1. 2700 is simply 9x300 - ie
you've got data for 9 step times of 300 seconds.

Why are there two entries of 10 in that calculation ? Your first
update comes after all the data is unknown. You've configured your DS
to allow up to 600 seconds between samples before the data is unknown
- therefore your first update of 10 actually works out to "the data
for the previous 2 sample periods (600/300) is "10". If you didn't
want that then you'd need to set an initial value (eg update with
1260112800:0) to explicitly set a start boundary to your data.

That last bit isn't a statement based on profound knowledge or
examination of the source code - it's what the numbers work out to
and is logically consistent with the documentation !

--
Simon Hobson

Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed
author Gladys Hobson. Novels - poetry - short stories - ideal as
Christmas stocking fillers. Some available as e-books.

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: Questions of new user

2009-12-06T07:39:04Z

кто: Sun, 06 Dec 2009 12:49:59 +0100
когда: rrd-users@...

Thanks a lot for ansvers, i have many expirements today with data storing and fetching. Some of this was success.

But there is some misunderstanding:

C:\rrd\rrdtool.exe create test.rrd --start N --step 300 ^
DS:test:GAUGE:600:0:100 ^
RRA:AVERAGE:0.5:12:1

C:\rrd\rrdupdate.exe test.rrd ^
1260113100:10 ^
1260113400:20 ^
1260113700:30 ^
1260114000:40 ^
1260114300:50 ^
1260114600:60 ^
1260114900:70 ^
1260115200:80 ^
1260115500:70 ^
1260115800:60 ^
1260116100:50 ^
1260116400:40

Result:
1260111600: nan
1260115200: 4.1111111111e+01
1260118800: nan

But (10+20+...+50+40)/12 = 48,3! Why?

--
Pavel Rubtsov <qweion@...>
Playing: Fun Factory - Take Your Chance

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: Performance over NFS

2009-12-06T05:14:29Z

Hi Simon,

try disabeling mmap io

also try exporting nfs with the async option ... less of a data
integrity guarantee, but it should be faster ...

cheers
tobi

> Unfortunately, it's such a common term that I can't find anything
> particularly useful via searches.
>
> I'm collecting data on one box (a gateway router), and graphing on
> another box - with the directory holding the rrd files exported by
> NFS from the gateway. Unfortunately, while it seemed to work when I
> first set it up, it's now giving very poor performance - system
> upgrades have changed the setup from NFSv3 over UDP to V4 over TCP
> (if I remember the details correctly).
>
> The box doing the graphing is running Debian Etch and rrd version
> 1.2.15-0.3. /etc/fstab contains :
> w.x.y.z:/var/rrd /gate nfs ro,soft 0 0
>
> The gateway doing the data collection is running Debian Lenny and rrd
> version 1.3.1-4. /etc/exports contains :
> /var/rrd a.b.c.d(ro,subtree_check,all_squash,anonuid=1000,anongid=1000)
>
> The main rrd file that's giving problems is 22M byte in size and
> contains 510 data sources, with eight aggregations (Average and Max
> for four time periods/resolutions). Some graphs only use a couple of
> data sources, one frequently used one uses nearly 200 (in and out
> traffic for just under 100 addresses in a stacked graph), one
> (infrequently used for good reason) graphs 508 sources (in and out
> traffic for 254 addresses).
>
>
> The rrdtool processes end up spending significant amounts of time
> with ps showing them in state "rpc_wa" or "sync_p" (and I've just
> seen on in state "get_re"). It's not too bad if only one person is
> generating graphs (and waiting for them), but once or twice I've seen
> people being impatient and the box gets bogged down (not to mention
> swapping heavily) with a dozen of more rrdtool processes running.
> Even just one patient user can get the load average up into the teens.
>
>
> Can anyone give me some hints or pointers on this ?
>
>

--
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
http://it.oetiker.ch tobi@... ++41 62 775 9902 / sb: -9900

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Performance over NFS

2009-12-06T04:10:22Z

Unfortunately, it's such a common term that I can't find anything
particularly useful via searches.

I'm collecting data on one box (a gateway router), and graphing on
another box - with the directory holding the rrd files exported by
NFS from the gateway. Unfortunately, while it seemed to work when I
first set it up, it's now giving very poor performance - system
upgrades have changed the setup from NFSv3 over UDP to V4 over TCP
(if I remember the details correctly).

The box doing the graphing is running Debian Etch and rrd version
1.2.15-0.3. /etc/fstab contains :
w.x.y.z:/var/rrd /gate nfs ro,soft 0 0

The gateway doing the data collection is running Debian Lenny and rrd
version 1.3.1-4. /etc/exports contains :
/var/rrd a.b.c.d(ro,subtree_check,all_squash,anonuid=1000,anongid=1000)

The main rrd file that's giving problems is 22M byte in size and
contains 510 data sources, with eight aggregations (Average and Max
for four time periods/resolutions). Some graphs only use a couple of
data sources, one frequently used one uses nearly 200 (in and out
traffic for just under 100 addresses in a stacked graph), one
(infrequently used for good reason) graphs 508 sources (in and out
traffic for 254 addresses).

The rrdtool processes end up spending significant amounts of time
with ps showing them in state "rpc_wa" or "sync_p" (and I've just
seen on in state "get_re"). It's not too bad if only one person is
generating graphs (and waiting for them), but once or twice I've seen
people being impatient and the box gets bogged down (not to mention
swapping heavily) with a dozen of more rrdtool processes running.
Even just one patient user can get the load average up into the teens.

Can anyone give me some hints or pointers on this ?

--
Simon Hobson

Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed
author Gladys Hobson. Novels - poetry - short stories - ideal as
Christmas stocking fillers. Some available as e-books.

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: Questions of new user

2009-12-06T03:49:59Z

>>>> if you update more often than required for you step size, the data will
>>>> be aggregated using the aggregation function you've chosen (MIN, MAX,
>>>> AVERAGE etc) so either the smallest or the highest or the average of the
>>>> values will be stored.
>>>
>>> No, the aggregation functions are NOT used for sub-step updates.
>>>
>>> See Alex's tutorial on rates and normalising - it's all explained well
>>> there.
>>> http://www.vandenbogaerdt.nl/rrdtool/
>>
>> Actually I did not really describe sub-interval updates. And I don't know
>> from the top of my head. I'd have to look in the sources to see what really
>> happens when you do that.
>
> Well it's "sort of" in there, but not really highlighted. If you
> update every second with a step size of 60, then you can feed in one
> value of 60 and 59 values of 0 - and get the result of 1/second for
> the step. Min. Max, etc only come in when consolidating PDPs.
>
> That's what I've observed anyway.

Agreed. That's what I've observed, too. Sorry for the somewhat misleading info
in my first reply. I'm using mostly AVERAGE as aggregation function, so I got
confused myself, but aggregation happens at the RRA-entry-level. For PDPs it
seems to be always average.

- Karl

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: Questions of new user

2009-12-06T03:40:08Z

Alex van den Bogaerdt wrote:

> >>if you update more often than required for you step size, the data will be
>>>aggregated using the aggregation function you've chosen (MIN, MAX,
>>>AVERAGE etc)
>>>so either the smallest or the highest or the average of the values
>>>will be stored.
>>
>> No, the aggregation functions are NOT used for sub-step updates.
>>
>> See Alex's tutorial on rates and normalising - it's all explained well
>> there.
> > http://www.vandenbogaerdt.nl/rrdtool/
>
>
>
>Actually I did not really describe sub-interval updates. And I don't know
>from the top of my head. I'd have to look in the sources to see what really
>happens when you do that.

Well it's "sort of" in there, but not really highlighted. If you
update every second with a step size of 60, then you can feed in one
value of 60 and 59 values of 0 - and get the result of 1/second for
the step. Min. Max, etc only come in when consolidating PDPs.

That's what I've observed anyway.

--
Simon Hobson

Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed
author Gladys Hobson. Novels - poetry - short stories - ideal as
Christmas stocking fillers. Some available as e-books.

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: Questions of new user

2009-12-06T02:18:52Z

----- Original Message -----
From: "Simon Hobson" <linux@...>
To: <rrd-users@...>
Sent: Sunday, December 06, 2009 10:37 AM
Subject: Re: [rrd-users] Questions of new user

> rrd-users@... wrote:
>
>> > 2. Example:
>>>
>>> i have --step 300
>>> If i will update often, then 300 sec (for example every 10 seconds)
>>> what will be? Which data is stored?
>>
>>if you update more often than required for you step size, the data will be
>>aggregated using the aggregation function you've chosen (MIN, MAX,
>>AVERAGE etc)
>>so either the smallest or the highest or the average of the values
>>will be stored.
>
> No, the aggregation functions are NOT used for sub-step updates.
>
> See Alex's tutorial on rates and normalising - it's all explained well
> there.
> http://www.vandenbogaerdt.nl/rrdtool/

Actually I did not really describe sub-interval updates. And I don't know
from the top of my head. I'd have to look in the sources to see what really
happens when you do that.

I'd probably have step==1, and 300 steps per step in each of
{MIN,MAX,AVERAGE} RRAs. That results in one value per five minutes, with
aggregation as described above.

For the OP:

Those tutorials are probably going to help you. Start with "the" tutorial.

Experiment a lot. Create a database, update it, fetch the data and see if
you can predict what rrdtool is going to show you.

HTH
Alex

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: Questions of new user

2009-12-06T01:37:24Z

rrd-users@... wrote:

> > 2. Example:
>>
>> i have --step 300
>> If i will update often, then 300 sec (for example every 10 seconds)
>> what will be? Which data is stored?
>
>if you update more often than required for you step size, the data will be
>aggregated using the aggregation function you've chosen (MIN, MAX,
>AVERAGE etc)
>so either the smallest or the highest or the average of the values
>will be stored.

No, the aggregation functions are NOT used for sub-step updates.

See Alex's tutorial on rates and normalising - it's all explained well there.
http://www.vandenbogaerdt.nl/rrdtool/

--
Simon Hobson

Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed
author Gladys Hobson. Novels - poetry - short stories - ideal as
Christmas stocking fillers. Some available as e-books.

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: Questions of new user

2009-12-05T13:18:09Z

Pavel Rubtsov schrieb:

> Hi All!
>
> Im new user of rrdtool, i have many questions (and i search for web,
> but some thin questions is still no ansver...; and sorry for my english... :)
>
> 1. How rrdtool does updating in time interval?
>
> After creating of database and strict by step?
> Or it countdown timer after last adding data?
>
> Example:
>
> i have --step 300
> i update *.rrd base at 16:00:00 (h:m:s)
> i update *.rrd base at 16:02:00
> NOW next step begin in 16:00 or in 16:07?

Hi Pavel,

rrdtool strictly updates based on (unix) time of epoch (1970-01-01 00:00:00)
modulo your step, so a step 300 means xx:05:00, xx:10:00, xx:15:00 ...

> 2. Example:
>
> i have --step 300
> If i will update often, then 300 sec (for example every 10 seconds)
> what will be? Which data is stored?

if you update more often than required for you step size, the data will be
aggregated using the aggregation function you've chosen (MIN, MAX, AVERAGE etc)
so either the smallest or the highest or the average of the values will be stored.

> 3. I create rrd with:
> --step 60
>
> DS:test:GAUGE:120:0:100
> RRA:MIN:0.5:1:30
>
> i update every 60 sec with values:
> 10, then 20, then 30, etc.
> But when i create graph i see NOT Minimum value, its looks like averaged... why?

see above ...
first, if you don't update on the 'top' of the interval, eg. xx:xx:00 for a 60s,
the data will be interpolated, so for example you updated at

08:22:10 value 10
08:23:10 value 20

then the database will need the value for 08:23:00 which you didn't provide, so
it will be interpolated. At 08:23:00 it was 5/6 on the way from 10 to 20, so it
will write 10 + (20 - 10) * 5 / 6 = 18.333

hth

- Karl

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Questions of new user

2009-12-05T12:31:00Z

Hi All!

Im new user of rrdtool, i have many questions (and i search for web, but some thin questions is still no ansver...; and sorry for my english... :)

1. How rrdtool does updating in time interval?

After creating of database and strict by step?
Or it countdown timer after last adding data?

Example:

i have --step 300
i update *.rrd base at 16:00:00 (h:m:s)
i update *.rrd base at 16:02:00
NOW next step begin in 16:00 or in 16:07?

2. Example:

i have --step 300
If i will update often, then 300 sec (for example every 10 seconds) what will be? Which data is stored?

3. I create rrd with:
--step 60

DS:test:GAUGE:120:0:100
RRA:MIN:0.5:1:30

i update every 60 sec with values:
10, then 20, then 30, etc.
But when i create graph i see NOT Minimum value, its looks like averaged... why?

i have rrdtool-1.2.15-cygwin-1.5.20.zip

--
nothing playing

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: Simple questions about RRD

2009-12-05T05:19:36Z

2009/12/5 A Darren Dunham <ddunham@...>:
>
> Hopefully that demonstrates the differences pretty well.
>

Actually ; looking at the graph ; I can't understand how the MAX is
actually calculated.

Looking at:
http://htpc.avenard.org/power/screen.png

It shows the maximum as being 2972W ; which is indeed the maximum with
the actual data at the highest resolution.
But it's definitely not the maximum of 5 minutes average as the graph
shows: nothing is over 2200W

That graph is created with :

$ret = exec("$RRDTOOL graph $name -l 0 \
-t '$title' \
-x $legend \
--step $res --start e-$start --end $timestamp \
-w $width -h $height \
DEF:total=currentcost.rrd:total:AVERAGE DEF:ch2=currentcost.rrd:ch2:AVERAGE \
DEF:solar=solarprod.rrd:total:AVERAGE DEF:ch1=currentcost.rrd:ch1:AVERAGE \
DEF:totalmin=currentcost.rrd:total:MIN:reduce=AVERAGE
DEF:ch2min=currentcost.rrd:ch2:MIN:reduce=AVERAGE \
DEF:solarmin=solarprod.rrd:total:MIN:reduce=AVERAGE
DEF:ch1min=currentcost.rrd:ch1:MIN:reduce=AVERAGE \
DEF:totalmax=currentcost.rrd:total:MAX:reduce=AVERAGE
DEF:ch2max=currentcost.rrd:ch2:MAX:reduce=AVERAGE \
DEF:solarmax=solarprod.rrd:total:MAX:reduce=AVERAGE
DEF:ch1max=currentcost.rrd:ch1:MAX:reduce=AVERAGE \
VDEF:ds0max=ch2max,MAXIMUM \
VDEF:ds0avg=ch2,AVERAGE \
VDEF:ds0min=ch2min,MINIMUM \
VDEF:ds0pct=ch2,95,PERCENT \
VDEF:ds1max=totalmax,MAXIMUM \
VDEF:ds1avg=total,AVERAGE \
VDEF:ds1min=totalmin,MINIMUM \
VDEF:ds1pct=total,95,PERCENT \
VDEF:ds2max=ch1max,MAXIMUM \
VDEF:ds2avg=ch1,AVERAGE \
VDEF:ds2min=ch1min,MINIMUM \
VDEF:ds2pct=ch1,95,PERCENT \
VDEF:ds3max=solarmax,MAXIMUM \
VDEF:ds3avg=solar,AVERAGE \
VDEF:ds3min=solarmin,MINIMUM \
VDEF:ds3pct=solar,95,PERCENT \
COMMENT:' ' \
COMMENT:'Maximum ' \
COMMENT:'Average ' \
COMMENT:'Minimum ' \
COMMENT:'95th percentile\l' \
LINE1:total#000090:'Total ' \
GPRINT:ds1max:'%5.0lf W ' \
GPRINT:ds1avg:'%5.0lf W ' \
GPRINT:ds1min:'%5.0lf W ' \
GPRINT:ds1pct:' %5.0lf W\l' \
LINE1:ch1#00FF00:'Pool ' \
GPRINT:ds2max:'%5.0lf W ' \
GPRINT:ds2avg:'%5.0lf W ' \
GPRINT:ds2min:'%5.0lf W ' \
GPRINT:ds2pct:' %5.0lf W\l' \
LINE1:solar#FF0000:'Solar ' \
GPRINT:ds3max:'%5.0lf W ' \
GPRINT:ds3avg:'%5.0lf W ' \
GPRINT:ds3min:'%5.0lf W ' \
GPRINT:ds3pct:' %5.0lf W\l'

where $res , the argument for --step is 300 (5 minutes) and I do use
the reduce=AVERAGE option.

Thanks for your help once again.
Jean-Yves

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: Simple questions about RRD

2009-12-05T05:01:53Z

Hi

2009/12/5 A Darren Dunham <ddunham@...>:
[snip]
> You can change the RRA's consolidation function at the top easily to see
> how if it were AVERAGE already, there's no differences between the
> second two graphs.
>
> Hopefully that demonstrates the differences pretty well.

It does ... thank you for enlighting me

As I was thinking more into yesterday ; I too had felt that for
AVERAGE is result would be quite obvious ; but for the MIN and MAX
some issues would arise...

Now I have an extra question on how the RRA/consolidation works...

Say I store 5 years of 5 minutes average ; is there any points of also
storing 5 years of 30 minutes average, 5 years of 2 hours average etc?

I would have assumed that it wouldn't matter ; except when retrieving
the 30 minutes/2 hours average extra processing is required as it
needs to retrieve more values.

Now; if I use reduce=AVERAGE for all my graph like
DEF:total=currentcost.rrd:total:AVERAGE DEF:ch2=currentcost.rrd:ch2:AVERAGE \
DEF:solar=solarprod.rrd:total:AVERAGE DEF:ch1=currentcost.rrd:ch1:AVERAGE \
DEF:totalmin=currentcost.rrd:total:MIN:reduce=AVERAGE
DEF:ch2min=currentcost.rrd:ch2:MIN:reduce=AVERAGE \
DEF:solarmin=solarprod.rrd:total:MIN:reduce=AVERAGE
DEF:ch1min=currentcost.rrd:ch1:MIN:reduce=AVERAGE \
DEF:totalmax=currentcost.rrd:total:MAX:reduce=AVERAGE
DEF:ch2max=currentcost.rrd:ch2:MAX:reduce=AVERAGE \
DEF:solarmax=solarprod.rrd:total:MAX:reduce=AVERAGE
DEF:ch1max=currentcost.rrd:ch1:MAX:reduce=AVERAGE \

Do I even need to create the RRD with MIN and MAX ? or it can be all
done from the AVERAGE RRD database ?
In my case, I create the RRD with:
100 day of 1 minute average
5 years of 5 minutes average

rrdtool create currentcost.rrd -s 60 \
DS:total:GAUGE:300:0:U \
DS:ch1:GAUGE:300:0:U \
DS:ch2:GAUGE:300:0:U \
DS:ch3:GAUGE:300:0:U \
RRA:AVERAGE:0.5:1:144000 \
RRA:AVERAGE:0.5:5:525600 \
RRA:MIN:0.5:1:144000 \
RRA:MIN:0.5:5:525600 \
RRA:MAX:0.5:1:144000 \
RRA:MAX:0.5:5:525600

Not creating a MIN and MAX one would reduce considerably the size rrd file.

Here is my result so far ... You can click on parts of the graph to
automatically zoom-in for extra details.
http://htpc.avenard.org/power

Thanks
Jean-Yves

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: Simple questions about RRD

2009-12-04T10:22:40Z

On Fri, Dec 04, 2009 at 06:16:45PM +0000, A Darren Dunham wrote:

[perl program ...]
> # create two graphs, one normal, one at 5 min step.
> my @options = (
> "--end", $last,
> "--start", "end-2h",
> "AREA:total#FF0000FF");
> my $graph_dir
> RRDs::graph ("$rrd_dir/norm.png",
[snip...]

Argh.. scrolled back too far in the window and cut&pasted one section
from an older edit.

The "my $graph_dir" line is extraneous and will probably cause an error. It
shouldn't be present. Please delete it before trying to run.

--
Darren

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: Simple questions about RRD

2009-12-04T10:16:45Z

On Fri, Dec 04, 2009 at 11:52:33AM +1100, Jean-Yves Avenard wrote:

> I store two years of one minute data (just because I have plenty of disk space)
>
> rrdtool create currentcost.rrd -s 60 \
> DS:total:GAUGE:600:0:U \
> DS:ch1:GAUGE:600:0:U \
> DS:ch2:GAUGE:600:0:U \
> DS:ch3:GAUGE:600:0:U \
> RRA:AVERAGE:0.5:1:1054080 \
> RRA:AVERAGE:0.5:1440:1830 \
> RRA:MIN:0.5:1:527040 \
> RRA:MIN:0.5:1440:1830 \
> RRA:MAX:0.5:1:527040 \
> RRA:MAX:0.5:1440:1830
>
>
> I draw my daily graph with
> $ret = exec("$RRDTOOL graph images/powerday.png -l 0 \
> -t 'POWER - DAILY' \
> -x HOUR:1:HOUR:4:HOUR:2:0:%k \
> --step 60 --end $timestamp -w $DAYWIDTH -h $DAYHEIGHT \
> DEF:total=currentcost.rrd:total:AVERAGE DEF:ch2=currentcost.rrd:ch2:AVERAGE \
> DEF:solar=solarprod.rrd:total:AVERAGE DEF:ch1=currentcost.rrd:ch1:AVERAGE \
> DEF:totalmin=currentcost.rrd:total:MIN DEF:ch2min=currentcost.rrd:ch2:MIN \
> DEF:solarmin=solarprod.rrd:total:MIN DEF:ch1min=currentcost.rrd:ch1:MIN \
> DEF:totalmax=currentcost.rrd:total:MAX DEF:ch2max=currentcost.rrd:ch2:MAX \
> DEF:solarmax=solarprod.rrd:total:MAX DEF:ch1max=currentcost.rrd:ch1:MAX \
>
> scale is calculated so no rounding is done ....
>
> But with the way power usage go ; it's a very broken graph... So I
> would like to show 5 minutes average instead for the daily graph...
> does calling rrdgraph with --step 300 is all that is required to get 5
> minutes average ? It doesn't seem to be a 5 minutes average to me
> looking at the generated graph ; still having big spike on the minute
> I turned on my boiler kettle. A 5 minutes average should have removed
> the skipe.

>From rrd_data:
If consolidation needs to be done, the CF of the RRA specified in the
DEF itself will be used to reduce the data density. This behavior can
be changed using :reduce=<CF>. This optional parameter specifies the
CF to use during the data reduction phase.

So your AVERAGE RRAs will be doing 5 minute averages, but your MIN and
MAX RRAs will not.

> If that's not the way, how do I graph 5 minutes average when my data
> is made of 1 minute recording ?

Specify an explicit reduction CF and step value in your graph DEF.
Here's a perl example. Apologies for using RRDs. It shouldn't be too
hard to translate to using 'rrdtool' directly if you prefer.

use RRDs;
use Time::Local;
my $ts = timelocal(0,0,0,1,0,2008); #1 Jan, 2008, midnight local time
my $cf = "MAX";

my $rrd_dir = "/tmp";

my $rrd = "$rrd_dir/avg_test.rrd";
RRDs::create ($rrd, "--step", "60",
"--start", "$ts",
"DS:total:GAUGE:600:0:U",
"RRA:${cf}:0.5:1:5000");

# put in 1 "high" value and 4 "low" values every 5 min

foreach (0 .. 24) # 24 5 min periods => 2 hr
{
RRDs::update ($rrd, "${ts}:16");
$ts += 60;
foreach (0 .. 3)
{
RRDs::update ($rrd, "${ts}:1");
$ts += 60;
}
}

my $last = RRDs::last($rrd);
# create two graphs, one normal, one at 5 min step.
my @options = (
"--end", $last,
"--start", "end-2h",
"AREA:total#FF0000FF");
my $graph_dir
RRDs::graph ("$rrd_dir/norm.png",
"DEF:total=$rrd:total:${cf}",
@options);
RRDs::graph ("$rrd_dir/avg.png",
"DEF:total=$rrd:total:${cf}",
"--step", "300",
@options);
RRDs::graph ("$rrd_dir/avg2.png",
"DEF:total=$rrd:total:${cf}:step=300:reduce=AVERAGE",
@options);

This uses a single "MAX" RRA and does a normal graph (no consolidation,
with spikes), a --step=300 graph (consolidation uses the RRA, so MAX
consolidation), and a graph where the consolidation is specified to be
AVERAGE.

You can change the RRA's consolidation function at the top easily to see
how if it were AVERAGE already, there's no differences between the
second two graphs.

Hopefully that demonstrates the differences pretty well.

--
Darren

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: Non Linear X-Axis

2009-12-04T08:27:51Z

> Without considering the graphing issues, is it possible to store the
> data in an RRD thought ?

Anyone able to help out on this.

I'd like to take a reading every 10 seconds from a temp sensor (got
this bit working)
and store the following

6 x 10 seconds readings
60 x 1 minute averages
24x 1 hour averages
7 x 1 day averages

And then be able to retrieve these values.

How would I go about this in an RRD ?

Thanks

Stuart

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Simple questions about RRD

2009-12-03T16:52:33Z

Hi.

Long time user of RRDtool ; but I haven't got into creating new graph
in a while and I'm a bit confused about one detail...

I'm monitoring the power usage at my house with a new gizmo I installed...

Data is collected by a python script every 6s ; I average the data
over a minute and feed it to a rrd database.

I store two years of one minute data (just because I have plenty of disk space)

rrdtool create currentcost.rrd -s 60 \
DS:total:GAUGE:600:0:U \
DS:ch1:GAUGE:600:0:U \
DS:ch2:GAUGE:600:0:U \
DS:ch3:GAUGE:600:0:U \
RRA:AVERAGE:0.5:1:1054080 \
RRA:AVERAGE:0.5:1440:1830 \
RRA:MIN:0.5:1:527040 \
RRA:MIN:0.5:1440:1830 \
RRA:MAX:0.5:1:527040 \
RRA:MAX:0.5:1440:1830

I draw my daily graph with
$ret = exec("$RRDTOOL graph images/powerday.png -l 0 \
-t 'POWER - DAILY' \
-x HOUR:1:HOUR:4:HOUR:2:0:%k \
--step 60 --end $timestamp -w $DAYWIDTH -h $DAYHEIGHT \
DEF:total=currentcost.rrd:total:AVERAGE DEF:ch2=currentcost.rrd:ch2:AVERAGE \
DEF:solar=solarprod.rrd:total:AVERAGE DEF:ch1=currentcost.rrd:ch1:AVERAGE \
DEF:totalmin=currentcost.rrd:total:MIN DEF:ch2min=currentcost.rrd:ch2:MIN \
DEF:solarmin=solarprod.rrd:total:MIN DEF:ch1min=currentcost.rrd:ch1:MIN \
DEF:totalmax=currentcost.rrd:total:MAX DEF:ch2max=currentcost.rrd:ch2:MAX \
DEF:solarmax=solarprod.rrd:total:MAX DEF:ch1max=currentcost.rrd:ch1:MAX \

scale is calculated so no rounding is done ....

But with the way power usage go ; it's a very broken graph... So I
would like to show 5 minutes average instead for the daily graph...
does calling rrdgraph with --step 300 is all that is required to get 5
minutes average ? It doesn't seem to be a 5 minutes average to me
looking at the generated graph ; still having big spike on the minute
I turned on my boiler kettle. A 5 minutes average should have removed
the skipe.

If that's not the way, how do I graph 5 minutes average when my data
is made of 1 minute recording ?
Thanks
Jean-Yves

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: rrdcached and commands

2009-12-03T14:48:02Z

Hi Steve,

As documentation says, QUIT command just "disconnect from rrdcached",
but it doesn't make the proccess to finish its execution.

Best regards,
Fernando

On Thu, Dec 3, 2009 at 3:31 AM, Stephen S. Disbrow <steve@...> wrote:

> Hi,
> I didn't receive a response on a previous inquiry, but I wrote my own
> socket program to send and receive messages from the rrdcached process. When
> I send the "QUIT\n" command I expect the process to exit, but it's not going
> away There is nothing in the queue so I'd expect it to exit.
>
> Thanks in advanced,
>
> Steve Disbrow
> .
> _______________________________________________
> rrd-users mailing list
> rrd-users@...
> https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
>
>

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Re: rrdtool update error

2009-12-03T14:27:43Z

Hi,
Here's all the info, please let me know if any further information is
required. Also I don't think I'm setting any rrdtool specific variables in
the environment.

12/02/2009 10:19:13 AM
The rrdcached process is run as the metis user.
all the rrdtooll commands are run as the metis user.
The rrdcached startup command line
METIS_HOME="/usr/local/metis"
BASEPATH=${METIS_HOME}/db/rrd
/usr/pkg/bin/rrdcached -l unix:/tmp/metis.rrd.sock -w 60 -p
/tmp/metis.rrd.pid -t 5 -j ${BASEPATH}/journal
-b ${BASEPATH} -B
These files are in /tmp
-rw-r----- 1 metis metis 6 Dec 1 12:26 metis.rrd.pid
srwxrwx--- 1 metis metis 0 Dec 1 12:26 metis.rrd.sock
The $BASEPATH path is on an NFS file system.
The update command is as follows;
/usr/pkg/bin/         rrdtool update --daemon unix:/tmp/metis.rrd.sock
${BASEPATH}/volumes.rrd N:$accesses:$reads:$writes    &
The error is;
ERROR: rrdcached: Permission denied
The rrd file in question perms are as follows;
-rw-rw---- 1 metis metis 438480 Dec 1 13:25 volumes.rrd

Thanks in advance,
Stephen Disbrow

----- Original Message -----
From: "Sebastian Harl" <sh@...>
To: "Stephen S. Disbrow" <steve@...>
Cc: <rrd-users@...>
Sent: Wednesday, December 02, 2009 2:59 AM
Subject: Re: [rrd-users] rrdtool update error
Hi Steve,

On Tue, Dec 01, 2009 at 08:01:59PM -0600, Stephen S. Disbrow wrote:
>     I'm getting the following error from rrdtool update when using the
> rrdcached process;
> ERROR: rrdcached: Permission denied
>
> I don't know what permissions it's talking about. The rrdfiles have
> the proper permissions. I'm not using the -P option for rrdcached
> because I thought without it that all permissions will be opened. Any
> ideas as to what to look at?

I suppose, you are connecting to rrdcached through a UNIX socket (see
the -l command line option). In that case, the user doing rrdupdate
needs (filesystem) write permissions for the appropriate socket file.

If that is not the case, please provide more information about your
setup. What rrdcached command line options do you use? Does rrdcached
run as root? Which user is doing the rrdupdate? What is the command line
used for that (please note that some environment variables might
overwrite some of those options -- see the appropriate manpages for
details)? ...

HTH,
Sebastian

--
Sebastian "tokkee" Harl +++ GnuPG-ID: 0x8501C7FC +++ http://tokkee.org/

Those who would give up Essential Liberty to purchase a little Temporary
Safety, deserve neither Liberty nor Safety.         -- Benjamin Franklin

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

rrdtool 1.4.2 binaries for hp-ux 11.23 hppa available

2009-12-03T11:04:01Z

hi all,

there are now rrdtool 1.4.2 binaries for hp-ux available for download.
they are packaged as "depot" for use on hp-ux (pa-risc) machines.

you can find more information at:
http://www.filibeto.org/~aduritz/truetrue/rrdtool/1.4.2/

cheers,

Stoyan

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users