newbie rrd question - probably a very old issue

I'm not really a newbie to rrdtool, but this seems like a newbie question,

and it probably has been asked since the beginning of time, but I've never
seen a good explanation
of what's going on with the graph/consolidation functions.

we have this problem and it seems obvious, but we can't seem to figure out
how to fix it.
We have a data set, a script collects data every two minutes and puts data
in the database.

we defined the data set to be 3 months of daata as follows:
93 days (3 months x 31 - worst case) x 24 hours x 60 minutes x 60 seconds.

so that's 3 months of data points for every second, even though we only use
once every 2 minutes.

now technically we have enough data points that creating a graph for 1 day
or 1 week or 1 month, should have the ability to plot all the values.

but it doesn't.

for example, we do a 1 day graph and the "maximum" is 90
we do the same graph for 1 week and the "maximum" has magically changed to
30
do it again for 1 month, and it gets even worse.

when we query the rrd database directly, we see that rrd gives us these
values, so it's not the graphing function doing it, it's rrd.

but when we check the raw data, we can see all the data points are there.
if we have so many data points, it shouldn't be consolidating the data, but
it is.

and it's driving me crazy trying to figure out why.
we thought of using separate data files for min, max and average, but triple
the data storatge and triple the overhead is killing our server.

can someone tell us what we're doing wrong or give us a suggestion so that
the data will be reported properly?

It seems I recall this issue, and there some some setting we could use, like
ABS, but that doesn't seem to fit properly.

thanks

_________________________________________________________________
Show Your Messenger Buddies How You Really Feel
http://www.freemessengeremoticons.ca/?icid=EMENCA122

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

[Alex van den Bogaerdt]

On Fri, Aug 31, 2007 at 01:07:44PM -0400, Dan Gahlinger wrote:

> for example, we do a 1 day graph and the "maximum" is 90
> we do the same graph for 1 week and the "maximum" has magically changed to
> 30
> do it again for 1 month, and it gets even worse.

An RRA stores average, minimum or maximum rates.
GPRINT writes the average, minimum or maximum on the graph.

If you are graphing data from an AVERAGE RRA, you can print the
maximum of those averages, not the maximum out of another RRA.


1 step per row:  1,2,3,4,5,6,7,8,9,10,11,12

MAX(1,2,3,4,5,6,7,8,9,10,11,12) -> 12

consolidation function average, 3 steps per entry:

avg(1,2,3),avg(4,5,6),avg(7,8,9),avg(10,11,12) -> 2,5,8,11
MAX(2,5,8,11) -> 11, not 12.

consolidation function average, 6 steps per entry:
avg(1,2,3,4,5,6),avg(7,8,9,10,11,12) -> 3.5 , 9.5
MAX(3.5,9.5) -> 9.5, not 12

and so on.

> but when we check the raw data, we can see all the data points are there.
> if we have so many data points, it shouldn't be consolidating the data, but
> it is.

It shouldn't? Or perhaps you ask it to do so but don't want to?

> and it's driving me crazy trying to figure out why.
> we thought of using separate data files for min, max and average, but triple
> the data storatge and triple the overhead is killing our server.
>
> can someone tell us what we're doing wrong or give us a suggestion so that
> the data will be reported properly?

Post the command you use to create your RRD file.  Also post the
relevant parts of your graph command.

Keep it simple. We don't need to see all fancy formatting and such,
a simplified example would be best, as long as it shows what you are
talking about.

HTH
--
Alex van den Bogaerdt
http://www.vandenbogaerdt.nl/rrdtool/

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

I'll have to do this tomorrow or tuesday (monday is a holiday here),
and I don't have access to it at the moment.

you may be correct that we are asking it to consolidate, and we don't want
to. we simply don't know.
it just seems, totally wrong.

we used to have the RRD file setup for a years worth of data points, and it
was still doing this.

what's the point of defining so many data points if its going to consolidate
it anyhow?

we have learned it is NOT the graphing function doing it, though.

if we look at the raw data it is correct, but if we query the data, we get
the consolidated results.
it shouldn't be consolidating.

there is no reason that an RRD file created with 93 days of data points
should start consolidating data after only one week - or less. there has to
be something we're doing wrong.

so i'll post it either later this weekend or at latest tuesday.

>From: Alex van den Bogaerdt <alex@...>
>To: rrd-users@...
>Subject: Re: [rrd-users] newbie rrd question - probably a very old issue
>Date: Fri, 31 Aug 2007 19:35:57 +0200
>
>On Fri, Aug 31, 2007 at 01:07:44PM -0400, Dan Gahlinger wrote:
>
> > for example, we do a 1 day graph and the "maximum" is 90
> > we do the same graph for 1 week and the "maximum" has magically changed
>to
> > 30
> > do it again for 1 month, and it gets even worse.
>
>An RRA stores average, minimum or maximum rates.
>GPRINT writes the average, minimum or maximum on the graph.
>
>If you are graphing data from an AVERAGE RRA, you can print the
>maximum of those averages, not the maximum out of another RRA.
>
>
>1 step per row:  1,2,3,4,5,6,7,8,9,10,11,12
>
>MAX(1,2,3,4,5,6,7,8,9,10,11,12) -> 12
>
>consolidation function average, 3 steps per entry:
>
>avg(1,2,3),avg(4,5,6),avg(7,8,9),avg(10,11,12) -> 2,5,8,11
>MAX(2,5,8,11) -> 11, not 12.
>
>consolidation function average, 6 steps per entry:
>avg(1,2,3,4,5,6),avg(7,8,9,10,11,12) -> 3.5 , 9.5
>MAX(3.5,9.5) -> 9.5, not 12
>
>and so on.
>
> > but when we check the raw data, we can see all the data points are
>there.
> > if we have so many data points, it shouldn't be consolidating the data,
>but
> > it is.
>
>It shouldn't? Or perhaps you ask it to do so but don't want to?
>
> > and it's driving me crazy trying to figure out why.
> > we thought of using separate data files for min, max and average, but
>triple
> > the data storatge and triple the overhead is killing our server.
> >
> > can someone tell us what we're doing wrong or give us a suggestion so
>that
> > the data will be reported properly?
>
>Post the command you use to create your RRD file.  Also post the
>relevant parts of your graph command.
>
>Keep it simple. We don't need to see all fancy formatting and such,
>a simplified example would be best, as long as it shows what you are
>talking about.
>
>HTH
>--
>Alex van den Bogaerdt
>http://www.vandenbogaerdt.nl/rrdtool/
>
>_______________________________________________
>rrd-users mailing list
>rrd-users@...
>https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

_________________________________________________________________
Windows Live Hotmail. Even hotter than before. Get a better look now.
www.newhotmail.ca?icid=WLHMENCA148

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

[Alex van den Bogaerdt]

On Fri, Aug 31, 2007 at 10:06:24PM -0400, Dan Gahlinger wrote:

> you may be correct that we are asking it to consolidate, and we don't want
> to. we simply don't know.
> it just seems, totally wrong.

If you have 400 pixels on the graph, and you have 4000 averages to display,
it won't fit. Somehow you will have to combine 10 intervals into one.
If you don't, then rrdtool will.

> what's the point of defining so many data points if its going to
> consolidate it anyhow?

With many datarows you can look back in time at the highest possible
resolution. You won't be using all of the data, just a portion of it.

Consolidated data can be used to quickly report on a longer interval,
without having to wait for rrdtool to do "consolidation" on the fly.
No need to reduce 4000 rows into 400, if you already have the data
available in a more appropriate resolution.

They both serve a purpose.


> we have learned it is NOT the graphing function doing it, though.

then it has to be another available RRA.

> if we look at the raw data it is correct, but if we query the data, we get
> the consolidated results.
> it shouldn't be consolidating.

Then you shouldn't have defined that RRA.

More likely you should have created another RRA, or if you did, you should
be using it.  If you want to see maxima, then you should use the MAX RRA,
not the AVERAGE RRA.
(capitals not for shouting, but because that's how it's written!)

--
Alex van den Bogaerdt
http://www.vandenbogaerdt.nl/rrdtool/

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

I think you're missing the point.

true, the graph only has so much space, but I call "BS" on your overall
explanation for 3 reasons.
#1 we don't have that MANY points to plot - so there's more than enough
space on the graph to show everything
#2 - even so it shouldn't be changing the actual points it does show. a peak
of 90 should be a peak of 90 regardless, it shouldn't shift it to a peak of
30, no matter what the numbers are. - if you can see the point on the graph,
it should be the same value.
#3 - using "max" instead of, or in addition to "avg" for RRA is a bogus
answer. if we use "max" instead of "Avg" we lose the "min", and the "avg" is
skewed. if we use it in addition, we have to define, "min", "avg" and "max"
- three RRA's for one data set, which kills the load on the server. imagine
1000 service checks having to write 3x the data instead of one. this isn't
the answer. the answer is to find out why RRD/RRA is doing what its doing
and make it stop.

in our case, we only had 2 days worth of data, actually less than 1000 bits
of data.
so on the weekly graph there are enough points.

on the graph we can clearly see the "90" spike on the daily graph, but the
weekly graph, that days spike has been changed to "30" and is plotted as 30.
this is very bad. it shouldn't happen.

but again, it's not the graph function that's doing it.
with the command line query, it produces the same results.

the graph is graphing it that way because RRD is giving it the data that
way.
RRD command line query for the week retrieves the max value as "30"
even though the raw data shows the "90" is there.

this is definitely broken. and badly. I'm not sure what's wrong with it.
again, it could be our fault in how the RRA is setup, but I don't see how.

I'll post that as soon as I can.

>From: Alex van den Bogaerdt <alex@...>
>To: rrd-users@...
>Subject: Re: [rrd-users] newbie rrd question - probably a very old issue
>Date: Sat, 1 Sep 2007 13:10:30 +0200
>
>On Fri, Aug 31, 2007 at 10:06:24PM -0400, Dan Gahlinger wrote:
>
> > you may be correct that we are asking it to consolidate, and we don't
>want
> > to. we simply don't know.
> > it just seems, totally wrong.
>
>If you have 400 pixels on the graph, and you have 4000 averages to display,
>it won't fit. Somehow you will have to combine 10 intervals into one.
>If you don't, then rrdtool will.
>
> > what's the point of defining so many data points if its going to
> > consolidate it anyhow?
>
>With many datarows you can look back in time at the highest possible
>resolution. You won't be using all of the data, just a portion of it.
>
>Consolidated data can be used to quickly report on a longer interval,
>without having to wait for rrdtool to do "consolidation" on the fly.
>No need to reduce 4000 rows into 400, if you already have the data
>available in a more appropriate resolution.
>
>They both serve a purpose.
>
>
> > we have learned it is NOT the graphing function doing it, though.
>
>then it has to be another available RRA.
>
> > if we look at the raw data it is correct, but if we query the data, we
>get
> > the consolidated results.
> > it shouldn't be consolidating.
>
>Then you shouldn't have defined that RRA.
>
>More likely you should have created another RRA, or if you did, you should
>be using it.  If you want to see maxima, then you should use the MAX RRA,
>not the AVERAGE RRA.
>(capitals not for shouting, but because that's how it's written!)
>
>--
>Alex van den Bogaerdt
>http://www.vandenbogaerdt.nl/rrdtool/
>
>_______________________________________________
>rrd-users mailing list
>rrd-users@...
>https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

_________________________________________________________________
Enter to win a night a VIP night out at TIFF
http://redcarpet.sympatico.msn.ca/

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

[Tobias Oetiker-3]

Dear Dan,

> I think you're missing the point.

just take a look at the number of posts alex makes. he does
understand how rrdtool works, he know the code that makes it tick
... so if you tell him he is missing the point, chances are that he
will not bother anymore help you understand how rrdtool works ...

to your question.

RRDtool tries hard to work correctly independant of the number of
points it has to work with. So your graph, should turn out the same
results, regardles of the number of samples that are available
(withing reason).

This means for example, that if you input data from an RRA that is
using AVERAGE consolidation, that the graph will also use AVERAGE
consolidation to reduce the data until it fits in the graph. If you
use a MAX function over this, the MAX will return what you see in
the graph.

If you are interested in keeping the MAX of the data you have
originally fed into RRDtool, you must store it in an RRA that has a
MAX function attached.

A little example:

You have to 2 points in your RRA

a: 10
b: 90

On the graph they map onto the same pixel. Depending on the
consolidation function of the RRA, rrdtool will draw the pixle at
the following positions

avg: 50
min: 10
max: 90

on top of this you can use VDEF to calculate, but obviously the
results will depend on the first round of consolidation that has
already happened.

>From a performance point of view, given your box has enough memory,
you should be able to get about

 * 200-400 RRA updates a second per spindle

Note that you can cheat this by increassing the amount of dirty
buffers the OS tolerates as well as the ammount of time the dirty
buffers can stay dirty.

Especially if you run frequent updates this can be a neat strategy
to increasse throughput dramatically (in cache updates, rrdtool 1.3
can do about 20k per second on a current box.)

cheers
tobi

--
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten
http://it.oetiker.ch tobi@... ++41 62 213 9902

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Tobias,

yes, we are just getting rather frustrated with why this is happening.
I hope you and Alex can understand.

and I have said in every post it is likely something we're doing, we just
don't know what.
we can't see why with what we've done - how it's doing this.

it's going to be some small stupid thing we missed, that's my gut feeling.

we've eliminated the graph, which is good.
we are trying to understand how the raw data shows one value and query gives
us another - using the command line functions.

I used RRD years ago, and became quite adept at it, I seem to recall we had
this exact same issue and fixed it, but then I didn't use RRD for many
years, and I forgot these details.

so hopefully Alex can understand the frustration, and I'll definitely post
the details when I can

i appreciate the help, in the meantime i'll look into recreating this on my
local system, if I can do that with a smaller data set we have a much better
chance of finding the issue.

>From: Tobias Oetiker <tobi@...>
>To: Dan Gahlinger <dgahling@...>
>CC: alex@..., rrd-users@...
>Subject: Re: [rrd-users] newbie rrd question - probably a very old issue
>Date: Sat, 1 Sep 2007 17:14:33 +0200 (CEST)
>
>Dear Dan,
>
> > I think you're missing the point.
>
>just take a look at the number of posts alex makes. he does
>understand how rrdtool works, he know the code that makes it tick
>... so if you tell him he is missing the point, chances are that he
>will not bother anymore help you understand how rrdtool works ...
>
>to your question.
>
>RRDtool tries hard to work correctly independant of the number of
>points it has to work with. So your graph, should turn out the same
>results, regardles of the number of samples that are available
>(withing reason).
>
>This means for example, that if you input data from an RRA that is
>using AVERAGE consolidation, that the graph will also use AVERAGE
>consolidation to reduce the data until it fits in the graph. If you
>use a MAX function over this, the MAX will return what you see in
>the graph.
>
>If you are interested in keeping the MAX of the data you have
>originally fed into RRDtool, you must store it in an RRA that has a
>MAX function attached.
>
>A little example:
>
>You have to 2 points in your RRA
>
>a: 10
>b: 90
>
>On the graph they map onto the same pixel. Depending on the
>consolidation function of the RRA, rrdtool will draw the pixle at
>the following positions
>
>avg: 50
>min: 10
>max: 90
>
>on top of this you can use VDEF to calculate, but obviously the
>results will depend on the first round of consolidation that has
>already happened.
>
>From a performance point of view, given your box has enough memory,
>you should be able to get about
>
>  * 200-400 RRA updates a second per spindle
>
>Note that you can cheat this by increassing the amount of dirty
>buffers the OS tolerates as well as the ammount of time the dirty
>buffers can stay dirty.
>
>Especially if you run frequent updates this can be a neat strategy
>to increasse throughput dramatically (in cache updates, rrdtool 1.3
>can do about 20k per second on a current box.)
>
>cheers
>tobi
>
>--
>Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten
>http://it.oetiker.ch tobi@... ++41 62 213 9902

_________________________________________________________________
Show Your Messenger Buddies How You Really Feel
http://www.freemessengeremoticons.ca/?icid=EMENCA122

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

[Marc Powell]

> -----Original Message-----
> From: rrd-users-bounces@... [mailto:rrd-users-
> bounces@...] On Behalf Of Dan Gahlinger
> Sent: Saturday, September 01, 2007 9:31 AM
> To: alex@...; rrd-users@...
> Subject: Re: [rrd-users] newbie rrd question - probably a very old
issue
>


>
> in our case, we only had 2 days worth of data, actually less than 1000
> bits
> of data.
> so on the weekly graph there are enough points.
>
> on the graph we can clearly see the "90" spike on the daily graph, but
the
> weekly graph, that days spike has been changed to "30" and is plotted
as
> 30.
> this is very bad. it shouldn't happen.

Look at it this way. Assuming 5 minute samples, there are 288 data
points in 1 day. On a 300 pixel wide graph, you have enough space to
graph each single data point and then some. Now you ask rrdtool to graph
a weeks worth of data points in a 300 pixel wide graph. There are 2016
possible data points from which it must choose only 300 to display. How
should it do this? RRDTool uses the best fitting RRA and the
consolidation function you specify (likely AVERAGE).

The fact that you only have two days of data doesn't mean RRDTool is
going to change the way it creates a weekly graph. You've told it to
graph a weeks worth of data so it'll do the best it can to show you a
weeks worth of data, NAN or not. You only had two days worth of data but
still asked rrdtool to show 7 days worth. It will, properly, consolidate
what you have to fit the two days and show 5 days worth of NAN data.

--
Marc

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

what about the command-line query function?
there is no "space limitation" with that.

why is that consolidating?

and I still disagree with the analysis.
even as you say, it has to shrink it to fit, that doesn't mean it should be
averaging out the maximum (or the minimum) and adjusting the graph.

you say there isn't enough space, I disagree. take a look at a "Nasdaq"
graph for comparison.
whether you do a day, week, month, 6 months, or a year, the graph is the
same size with the same amount of available space, yet it shows every
minimum, and every maximum correctly. it never averages those out. and all
on one graph.

I'll post a link to some examples if you would like to compare - not sure it
can relate directly though.

the point is, it is possible to do, so rrd should also be doing this. but
even more to the point,
the command-line tools are consolidating - and those have infinite space, so
they definitely should not be consolidating.

can you explain why the command line tool is doing the same thing? I'd
really like an explanation for that.

Dan.

>From: "Marc Powell" <marc@...>
>To: <rrd-users@...>
>Subject: Re: [rrd-users] newbie rrd question - probably a very old issue
>Date: Sat, 1 Sep 2007 10:36:20 -0500
>
>
>
> > -----Original Message-----
> > From: rrd-users-bounces@... [mailto:rrd-users-
> > bounces@...] On Behalf Of Dan Gahlinger
> > Sent: Saturday, September 01, 2007 9:31 AM
> > To: alex@...; rrd-users@...
> > Subject: Re: [rrd-users] newbie rrd question - probably a very old
>issue
> >
>
>
> >
> > in our case, we only had 2 days worth of data, actually less than 1000
> > bits
> > of data.
> > so on the weekly graph there are enough points.
> >
> > on the graph we can clearly see the "90" spike on the daily graph, but
>the
> > weekly graph, that days spike has been changed to "30" and is plotted
>as
> > 30.
> > this is very bad. it shouldn't happen.
>
>Look at it this way. Assuming 5 minute samples, there are 288 data
>points in 1 day. On a 300 pixel wide graph, you have enough space to
>graph each single data point and then some. Now you ask rrdtool to graph
>a weeks worth of data points in a 300 pixel wide graph. There are 2016
>possible data points from which it must choose only 300 to display. How
>should it do this? RRDTool uses the best fitting RRA and the
>consolidation function you specify (likely AVERAGE).
>
>The fact that you only have two days of data doesn't mean RRDTool is
>going to change the way it creates a weekly graph. You've told it to
>graph a weeks worth of data so it'll do the best it can to show you a
>weeks worth of data, NAN or not. You only had two days worth of data but
>still asked rrdtool to show 7 days worth. It will, properly, consolidate
>what you have to fit the two days and show 5 days worth of NAN data.
>
>--
>Marc
>
>_______________________________________________
>rrd-users mailing list
>rrd-users@...
>https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

_________________________________________________________________
Get the device you want, with the Hotmail® you love.
http://www.windowsmobile.com/hotmailmobile?ocid=MobileHMTagline_1


_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Well here it is, I managed to get our definition - from the 1 year of data
version

create rrd_file --start 0 --step 120
"DS:a:GAUGE:360:U:U","RRA:AVERAGE:0.5:1:12*31*24*3600/120"

sorry its cut from the perl script that does it, but that's the definition

the 3 months version is just 3*31*24*3600/120 instead of the above.

again in a sense, the graph being "wrong" makes sense, because the command
line query gives the same results.

if we can solve why the command line query gives the result it does, I think
we'll have our answer.

_________________________________________________________________
Share More On Messenger with a Windows Live Space
http://spaces.live.com/?mkt=en-ca

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

[Jason Fesler]

Based on a side conversation with Dan, this is what I'm understanding:

RRD will graph exactly one consolidated point in time, if there are more
points than pixels on the X axis.  Dan wants *all* value points
represented, not consolidated.  RRD has no option to do that.

I'm thinking that *maybe* it can be faked, with some stacked graphs or
something, *if* MIN and MAX were also included in the rrd file; do a fill
between min and max.

But it isn't going to happen with a single graph statement, and I'm having
problems figuring out how to sanely show min/average/max if desired. :-)

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

that's not exactly what I want.

to be honest, I'd be happy if rrd did one simple thing (or two depending on
your point of view).

I'd recommend the "min" and "max" of the average function be done a bit
differently.
for an average graph, the "maximum" (shown at the bottom) should always be
kept,
and the "minimum" should always be kept

also the points for those on the graph should always be kept, even if it
means the verticals in the graph are stretched a bit.

the graph doesnt have to show ever portion of the scale either. by this I
mean
if there are no values between 10 and 40, you dont need to show the portions
from 11-39 essentially
as long as the maximum, minimum and average are represented properly

it should be simple enough to keep track of the minimum and maximum over
time.
my logic:
if you have a spike of 100 one day and that is you maximum, then a week or a
month or a year later, if that is still your maximum, then it is always your
maximum, and the spike should be shown
today it should be the max, tomorrow it should be the max, and until a
higher value occurs, it should always be the max, same with minimum, if we
got a zero, that will always be the minimum. and the overall average should
be the average between those two values.

otherwise the data is incorrect and quickly becomes meaningless.
yesterday it spiked to 90, but today it shows the max is only 30, so did it
spike? not according to the graph it didnt.

and I still dont understand why the command line (rrdfetch) is
consolidating, it has no graphing limitations

I don't know a better way to do it.
Dan.

>From: Jason Fesler <jfesler@...>
>To: Dan Gahlinger <dgahling@...>
>CC: tobi@..., rrd-users@...
>Subject: Re: [rrd-users] newbie rrd question - probably a very old issue
>Date: Sat, 1 Sep 2007 10:09:32 -0700 (PDT)
>
>Based on a side conversation with Dan, this is what I'm understanding:
>
>RRD will graph exactly one consolidated point in time, if there are more
>points than pixels on the X axis.  Dan wants *all* value points
>represented, not consolidated.  RRD has no option to do that.
>
>I'm thinking that *maybe* it can be faked, with some stacked graphs or
>something, *if* MIN and MAX were also included in the rrd file; do a fill
>between min and max.
>
>But it isn't going to happen with a single graph statement, and I'm having
>problems figuring out how to sanely show min/average/max if desired. :-)
>

_________________________________________________________________
Show Your Messenger Buddies How You Really Feel
http://www.freemessengeremoticons.ca/?icid=EMENCA122

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

[Tobias Oetiker-3]

Jason,

if you want to have this, you create TWO rras, one with MIN and one
with MAX consolidation.

in the graph you draw

MIN and stack MAX-MIN on top of it and you get that 'magic' graph
dans seems to be looking for ... it will look odd, since it will
change its 'width' depending on the distance between min and max,
but I can't see how he expects this to work any other way ...

I wonder what his nsdaq graph does if in a single day there is an
extreem high and and extreem low, but there is only a single pixel
in the graph to represent that day ...

cheers
tobi

Today Jason Fesler wrote:

> Based on a side conversation with Dan, this is what I'm understanding:
>
> RRD will graph exactly one consolidated point in time, if there are more
> points than pixels on the X axis.  Dan wants *all* value points represented,
> not consolidated.  RRD has no option to do that.
>
> I'm thinking that *maybe* it can be faked, with some stacked graphs or
> something, *if* MIN and MAX were also included in the rrd file; do a fill
> between min and max.
>
> But it isn't going to happen with a single graph statement, and I'm having
> problems figuring out how to sanely show min/average/max if desired. :-)
>
>

--
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten
http://it.oetiker.ch tobi@... ++41 62 213 9902

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

[Tobias Oetiker-3]

Dan,

note that rrdfetch is NOT consolidating anything, it shows you what
is stored in the rrd ... if you want to see what is actually
stored, use rrdtool dump ...

you may also want to read through the mailinglist archives, since
we get someone like you every few months how needs many days of
patient explaining until he understand why rrdtool is designed the
way it is designed and why it works the way it works, and why this
is the right thing todo ... (it has been like this for the last 8
years actually and tousands have used and understood the logic) ...
you will get there ...

cheers
tobi

>
> that's not exactly what I want.
>
> to be honest, I'd be happy if rrd did one simple thing (or two depending on
> your point of view).
>
> I'd recommend the "min" and "max" of the average function be done a bit
> differently.
> for an average graph, the "maximum" (shown at the bottom) should always be
> kept,
> and the "minimum" should always be kept
>
> also the points for those on the graph should always be kept, even if it means
> the verticals in the graph are stretched a bit.
>
> the graph doesnt have to show ever portion of the scale either. by this I mean
> if there are no values between 10 and 40, you dont need to show the portions
> from 11-39 essentially
> as long as the maximum, minimum and average are represented properly
>
> it should be simple enough to keep track of the minimum and maximum over time.
> my logic:
> if you have a spike of 100 one day and that is you maximum, then a week or a
> month or a year later, if that is still your maximum, then it is always your
> maximum, and the spike should be shown
> today it should be the max, tomorrow it should be the max, and until a higher
> value occurs, it should always be the max, same with minimum, if we got a
> zero, that will always be the minimum. and the overall average should be the
> average between those two values.
>
> otherwise the data is incorrect and quickly becomes meaningless.
> yesterday it spiked to 90, but today it shows the max is only 30, so did it
> spike? not according to the graph it didnt.
>
> and I still dont understand why the command line (rrdfetch) is consolidating,
> it has no graphing limitations
>
> I don't know a better way to do it.
> Dan.
> > From: Jason Fesler <jfesler@...>
> > To: Dan Gahlinger <dgahling@...>
> > CC: tobi@..., rrd-users@...
> > Subject: Re: [rrd-users] newbie rrd question - probably a very old issue
> > Date: Sat, 1 Sep 2007 10:09:32 -0700 (PDT)
> >
> > Based on a side conversation with Dan, this is what I'm understanding:
> >
> > RRD will graph exactly one consolidated point in time, if there are more
> > points than pixels on the X axis.  Dan wants *all* value points represented,
> > not consolidated.  RRD has no option to do that.
> >
> > I'm thinking that *maybe* it can be faked, with some stacked graphs or
> > something, *if* MIN and MAX were also included in the rrd file; do a fill
> > between min and max.
> >
> > But it isn't going to happen with a single graph statement, and I'm having
> > problems figuring out how to sanely show min/average/max if desired. :-)
> >
>
> _________________________________________________________________
> Show Your Messenger Buddies How You Really Feel
> http://www.freemessengeremoticons.ca/?icid=EMENCA122
>
>

--
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten
http://it.oetiker.ch tobi@... ++41 62 213 9902

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

check it out for yourself.
a company everyone hates as the reference

here's a year NASDAQ graph showing highs and lows with points between
and no loss of accuracy

http://charting.nasdaq.com/ext/charts.dll?2-1-14-0-0-512-03NA000000SCOX-&SF:4|5-WD=484-HT=395-
if that doesnt work (for whatever reason) use this:

http://quotes.nasdaq.com/quote.dll?page=charting&mode=basics&symbol=SCOX%60&selected=SCOX%60

>From: Tobias Oetiker <tobi@...>
>To: Jason Fesler <jfesler@...>
>CC: Dan Gahlinger <dgahling@...>, rrd-users@...
>Subject: Re: [rrd-users] newbie rrd question - probably a very old issue
>Date: Sat, 1 Sep 2007 20:45:19 +0200 (CEST)
>
>Jason,
>
>if you want to have this, you create TWO rras, one with MIN and one
>with MAX consolidation.
>
>in the graph you draw
>
>MIN and stack MAX-MIN on top of it and you get that 'magic' graph
>dans seems to be looking for ... it will look odd, since it will
>change its 'width' depending on the distance between min and max,
>but I can't see how he expects this to work any other way ...
>
>I wonder what his nsdaq graph does if in a single day there is an
>extreem high and and extreem low, but there is only a single pixel
>in the graph to represent that day ...
>
>cheers
>tobi
>
>Today Jason Fesler wrote:
>
> > Based on a side conversation with Dan, this is what I'm understanding:
> >
> > RRD will graph exactly one consolidated point in time, if there are more
> > points than pixels on the X axis.  Dan wants *all* value points
>represented,
> > not consolidated.  RRD has no option to do that.
> >
> > I'm thinking that *maybe* it can be faked, with some stacked graphs or
> > something, *if* MIN and MAX were also included in the rrd file; do a
>fill
> > between min and max.
> >
> > But it isn't going to happen with a single graph statement, and I'm
>having
> > problems figuring out how to sanely show min/average/max if desired. :-)
> >
> >
>
>--
>Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten
>http://it.oetiker.ch tobi@... ++41 62 213 9902

_________________________________________________________________
See Fireworks On Live Image Search
http://search.live.com/images/results.aspx?q=Fireworks&mkt=en-ca&FORM=SERNEP

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

Sounds like this is in dire need of a FAQ! :)

so why does rrd dump and rrd fetch show different results?
I'll check the "--resolution" option before continuing that question.

and I could swear I "fixed" this problem with a simple modification to the
RRA definition.
I'm almost positive it had something to do with "ABS", but that was 5 years
ago.

>From: Tobias Oetiker <tobi@...>
>To: Dan Gahlinger <dgahling@...>
>CC: jfesler@..., rrd-users@...
>Subject: Re: [rrd-users] newbie rrd question - probably a very old issue
>Date: Sat, 1 Sep 2007 20:49:26 +0200 (CEST)
>
>Dan,
>
>note that rrdfetch is NOT consolidating anything, it shows you what
>is stored in the rrd ... if you want to see what is actually
>stored, use rrdtool dump ...
>
>you may also want to read through the mailinglist archives, since
>we get someone like you every few months how needs many days of
>patient explaining until he understand why rrdtool is designed the
>way it is designed and why it works the way it works, and why this
>is the right thing todo ... (it has been like this for the last 8
>years actually and tousands have used and understood the logic) ...
>you will get there ...
>
>cheers
>tobi
>
>
> >
> > that's not exactly what I want.
> >
> > to be honest, I'd be happy if rrd did one simple thing (or two depending
>on
> > your point of view).
> >
> > I'd recommend the "min" and "max" of the average function be done a bit
> > differently.
> > for an average graph, the "maximum" (shown at the bottom) should always
>be
> > kept,
> > and the "minimum" should always be kept
> >
> > also the points for those on the graph should always be kept, even if it
>means
> > the verticals in the graph are stretched a bit.
> >
> > the graph doesnt have to show ever portion of the scale either. by this
>I mean
> > if there are no values between 10 and 40, you dont need to show the
>portions
> > from 11-39 essentially
> > as long as the maximum, minimum and average are represented properly
> >
> > it should be simple enough to keep track of the minimum and maximum over
>time.
> > my logic:
> > if you have a spike of 100 one day and that is you maximum, then a week
>or a
> > month or a year later, if that is still your maximum, then it is always
>your
> > maximum, and the spike should be shown
> > today it should be the max, tomorrow it should be the max, and until a
>higher
> > value occurs, it should always be the max, same with minimum, if we got
>a
> > zero, that will always be the minimum. and the overall average should be
>the
> > average between those two values.
> >
> > otherwise the data is incorrect and quickly becomes meaningless.
> > yesterday it spiked to 90, but today it shows the max is only 30, so did
>it
> > spike? not according to the graph it didnt.
> >
> > and I still dont understand why the command line (rrdfetch) is
>consolidating,
> > it has no graphing limitations
> >
> > I don't know a better way to do it.
> > Dan.
> > > From: Jason Fesler <jfesler@...>
> > > To: Dan Gahlinger <dgahling@...>
> > > CC: tobi@..., rrd-users@...
> > > Subject: Re: [rrd-users] newbie rrd question - probably a very old
>issue
> > > Date: Sat, 1 Sep 2007 10:09:32 -0700 (PDT)
> > >
> > > Based on a side conversation with Dan, this is what I'm understanding:
> > >
> > > RRD will graph exactly one consolidated point in time, if there are
>more
> > > points than pixels on the X axis.  Dan wants *all* value points
>represented,
> > > not consolidated.  RRD has no option to do that.
> > >
> > > I'm thinking that *maybe* it can be faked, with some stacked graphs or
> > > something, *if* MIN and MAX were also included in the rrd file; do a
>fill
> > > between min and max.
> > >
> > > But it isn't going to happen with a single graph statement, and I'm
>having
> > > problems figuring out how to sanely show min/average/max if desired.
>:-)
> > >
> >
> > _________________________________________________________________
> > Show Your Messenger Buddies How You Really Feel
> > http://www.freemessengeremoticons.ca/?icid=EMENCA122
> >
> >
>
>--
>Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten
>http://it.oetiker.ch tobi@... ++41 62 213 9902

_________________________________________________________________
Kick back and relax with hot games and cool activities at the Messenger
Café. http://www.cafemessenger.com?ocid=TXT_TAGHM_SeptHMtagline1


_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

[Tobias Oetiker-3]

Dan,

> Sounds like this is in dire need of a FAQ! :)

yes, unfortunately eveyone seems to come from a diffferent angle
 ... but once you got it down, it would be great if you could write
up what you understood ...

> so why does rrd dump and rrd fetch show different results?
> I'll check the "--resolution" option before continuing that question.

rrdtool dump shows the whole rrd file

rrdtool fetch picks one RRA and shows you the data from that RRA

cheers
tobi

> and I could swear I "fixed" this problem with a simple modification to the RRA
> definition.
> I'm almost positive it had something to do with "ABS", but that was 5 years
> ago.

>
>
> > From: Tobias Oetiker <tobi@...>
> > To: Dan Gahlinger <dgahling@...>
> > CC: jfesler@..., rrd-users@...
> > Subject: Re: [rrd-users] newbie rrd question - probably a very old issue
> > Date: Sat, 1 Sep 2007 20:49:26 +0200 (CEST)
> >
> > Dan,
> >
> > note that rrdfetch is NOT consolidating anything, it shows you what
> > is stored in the rrd ... if you want to see what is actually
> > stored, use rrdtool dump ...
> >
> > you may also want to read through the mailinglist archives, since
> > we get someone like you every few months how needs many days of
> > patient explaining until he understand why rrdtool is designed the
> > way it is designed and why it works the way it works, and why this
> > is the right thing todo ... (it has been like this for the last 8
> > years actually and tousands have used and understood the logic) ...
> > you will get there ...
> >
> > cheers
> > tobi
> >
> >
> > >
> > > that's not exactly what I want.
> > >
> > > to be honest, I'd be happy if rrd did one simple thing (or two depending
> > on
> > > your point of view).
> > >
> > > I'd recommend the "min" and "max" of the average function be done a bit
> > > differently.
> > > for an average graph, the "maximum" (shown at the bottom) should always be
> > > kept,
> > > and the "minimum" should always be kept
> > >
> > > also the points for those on the graph should always be kept, even if it
> > means
> > > the verticals in the graph are stretched a bit.
> > >
> > > the graph doesnt have to show ever portion of the scale either. by this I
> > mean
> > > if there are no values between 10 and 40, you dont need to show the
> > portions
> > > from 11-39 essentially
> > > as long as the maximum, minimum and average are represented properly
> > >
> > > it should be simple enough to keep track of the minimum and maximum over
> > time.
> > > my logic:
> > > if you have a spike of 100 one day and that is you maximum, then a week or
> > a
> > > month or a year later, if that is still your maximum, then it is always
> > your
> > > maximum, and the spike should be shown
> > > today it should be the max, tomorrow it should be the max, and until a
> > higher
> > > value occurs, it should always be the max, same with minimum, if we got a
> > > zero, that will always be the minimum. and the overall average should be
> > the
> > > average between those two values.
> > >
> > > otherwise the data is incorrect and quickly becomes meaningless.
> > > yesterday it spiked to 90, but today it shows the max is only 30, so did
> > it
> > > spike? not according to the graph it didnt.
> > >
> > > and I still dont understand why the command line (rrdfetch) is
> > consolidating,
> > > it has no graphing limitations
> > >
> > > I don't know a better way to do it.
> > > Dan.
> > > > From: Jason Fesler <jfesler@...>
> > > > To: Dan Gahlinger <dgahling@...>
> > > > CC: tobi@..., rrd-users@...
> > > > Subject: Re: [rrd-users] newbie rrd question - probably a very old issue
> > > > Date: Sat, 1 Sep 2007 10:09:32 -0700 (PDT)
> > > >
> > > > Based on a side conversation with Dan, this is what I'm understanding:
> > > >
> > > > RRD will graph exactly one consolidated point in time, if there are more
> > > > points than pixels on the X axis.  Dan wants *all* value points
> > represented,
> > > > not consolidated.  RRD has no option to do that.
> > > >
> > > > I'm thinking that *maybe* it can be faked, with some stacked graphs or
> > > > something, *if* MIN and MAX were also included in the rrd file; do a
> > fill
> > > > between min and max.
> > > >
> > > > But it isn't going to happen with a single graph statement, and I'm
> > having
> > > > problems figuring out how to sanely show min/average/max if desired. :-)
> > > >
> > >
> > > _________________________________________________________________
> > > Show Your Messenger Buddies How You Really Feel
> > > http://www.freemessengeremoticons.ca/?icid=EMENCA122
> > >
> > >
> >
> > --
> > Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten
> > http://it.oetiker.ch tobi@... ++41 62 213 9902
>
> _________________________________________________________________
> Kick back and relax with hot games and cool activities at the Messenger Caf??
> http://www.cafemessenger.com?ocid=TXT_TAGHM_SeptHMtagline1
>
>

--
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten
http://it.oetiker.ch tobi@... ++41 62 213 9902

_______________________________________________
rrd-users mailing list
rrd-users@...
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users