Missing data overnight from daylight saving changes
|
View:
New views
4 Messages
—
Rating Filter:
Alert me
|
 |
Missing data overnight from daylight saving changes
by Richard Curtis-7
::
Rate this Message:
Hi,
I have a simple rrd to store filesystem utilisation on an hourly basis. The data is gathered on other hosts, then processed into rrd via a script run at 4am. The rrd was created as: rrdtool create myrouter.rrd DS:used:GAUGE:3600:0:100 RRA:LAST:0.5:1:8760 --start 1256403600 --step 3600 And the following values entered: rrdtool update myrouter.rrd 1256410800:39 rrdtool update myrouter.rrd 1256414400:39 rrdtool update myrouter.rrd 1256418000:39 rrdtool update myrouter.rrd 1256421600:39 rrdtool update myrouter.rrd 1256425200:39 rrdtool update myrouter.rrd 1256432400:39 rrdtool update myrouter.rrd 1256436000:39 rrdtool update myrouter.rrd 1256439600:39 rrdtool update myrouter.rrd 1256443200:39 but I have a gap in data between midnight of the 25-Oct (BST) and 2am (GMT) on 26-Oct. An rrdtool dump shows: <!-- 2009-10-24 20:00:00 BST / 1256410800 --> <row><v> 3.9000000000e+01 </v></row> <!-- 2009-10-24 21:00:00 BST / 1256414400 --> <row><v> 3.9000000000e+01 </v></row> <!-- 2009-10-24 22:00:00 BST / 1256418000 --> <row><v> 3.9000000000e+01 </v></row> <!-- 2009-10-24 23:00:00 BST / 1256421600 --> <row><v> 3.9000000000e+01 </v></row> <!-- 2009-10-25 00:00:00 BST / 1256425200 --> <row><v> 3.9000000000e+01 </v></row> <!-- 2009-10-25 01:00:00 BST / 1256428800 --> <row><v> NaN </v></row> <!-- 2009-10-25 01:00:00 GMT / 1256432400 --> <row><v> NaN </v></row> <!-- 2009-10-25 02:00:00 GMT / 1256436000 --> <row><v> 3.9000000000e+01 </v></row> <!-- 2009-10-25 03:00:00 GMT / 1256439600 --> <row><v> 3.9000000000e+01 </v></row> <!-- 2009-10-25 04:00:00 GMT / 1256443200 --> <row><v> 3.9000000000e+01 </v></row> Am I missing something here? I did insert a value for 256432400, but it looks like rrdtool is expecting a value for both 01:00 BST and 01:00 GMT. Do I need to do some magic when I insert the values and double input when the clocks go back? Richard _______________________________________________ rrd-users mailing list rrd-users@... https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users |
|
|
Re: Missing data overnight from daylight saving changes
by A Darren Dunham
::
Rate this Message:
On Mon, Oct 26, 2009 at 03:53:57PM +0000, Richard Curtis wrote:
> Hi, > I have a simple rrd to store filesystem utilisation on an hourly basis. > The data is gathered on other hosts, then processed into rrd via a > script run at 4am. > > The rrd was created as: > rrdtool create myrouter.rrd DS:used:GAUGE:3600:0:100 > RRA:LAST:0.5:1:8760 --start 1256403600 --step 3600 So you've told RRD that unless a new value arrives within 3600 seconds, it should assume the data between updates is unknown. > And the following values entered: > rrdtool update myrouter.rrd 1256410800:39 > rrdtool update myrouter.rrd 1256414400:39 > rrdtool update myrouter.rrd 1256418000:39 > rrdtool update myrouter.rrd 1256421600:39 > rrdtool update myrouter.rrd 1256425200:39 > rrdtool update myrouter.rrd 1256432400:39 And the gap between these two updates is 7200, which is greater than 3600. So the data between is marked unknown. > rrdtool update myrouter.rrd 1256436000:39 > rrdtool update myrouter.rrd 1256439600:39 > rrdtool update myrouter.rrd 1256443200:39 > > but I have a gap in data between midnight of the 25-Oct (BST) and 2am > (GMT) on 26-Oct. Yup. > <!-- 2009-10-24 22:00:00 BST / 1256418000 --> > <row><v> 3.9000000000e+01 </v></row> > <!-- 2009-10-24 23:00:00 BST / 1256421600 --> > <row><v> 3.9000000000e+01 </v></row> > <!-- 2009-10-25 00:00:00 BST / 1256425200 --> > <row><v> 3.9000000000e+01 </v></row> > <!-- 2009-10-25 01:00:00 BST / 1256428800 --> > <row><v> NaN </v></row> > <!-- 2009-10-25 01:00:00 GMT / 1256432400 --> > <row><v> NaN </v></row> > <!-- 2009-10-25 02:00:00 GMT / 1256436000 --> > <row><v> 3.9000000000e+01 </v></row> > <!-- 2009-10-25 03:00:00 GMT / 1256439600 --> > <row><v> 3.9000000000e+01 </v></row> > <!-- 2009-10-25 04:00:00 GMT / 1256443200 --> > <row><v> 3.9000000000e+01 </v></row> > > Am I missing something here? I did insert a value for 256432400, but > it looks like rrdtool is expecting a value for both 01:00 BST and > 01:00 GMT. It needs a value for 1256428800 to keep the sequence intact. It doesn't care that one is 01:00 this or 01:00 that. It just cares that there's a gap of 7200 seconds, which exceeds the heartbeat limit of 3600. > Do I need to do some magic when I insert the values and double input > when the clocks go back? Did you actaully collect a value for that point in time? (One hour after midnight BST). It's difficult to tell if you're collecting one point every hour and throwing it away in your post-processing script, or if you're only collecting data on local clock hours and are missing the time point. Either way, RRD appears to be behaving correctly. While the "fetch" is translating the timestamps into local timezones, it really doesn't care about the tiemzone. It just needs the update every 3600 seconds and you haven't provided that here. -- Darren _______________________________________________ rrd-users mailing list rrd-users@... https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users |
|
|
Re: Missing data overnight from daylight saving changes
by Richard Curtis-7
::
Rate this Message:
>> Am I missing something here? I did insert a value for 256432400, but
>> it looks like rrdtool is expecting a value for both 01:00 BST and >> 01:00 GMT. > > It needs a value for 1256428800 to keep the sequence intact. > > It doesn't care that one is 01:00 this or 01:00 that. It just cares > that there's a gap of 7200 seconds, which exceeds the heartbeat limit of > 3600. Ah, now it makes sense. My script to collect the data runs via cron which ensures the script only ones once at DST change, so I am missing the data point for 01:00 BST (rather, my loader script assumes the entry for 01:00 on the 25th Oct is GMT so there is nothing inserted. I've fixed in now based on this - I allow a 7200 second gap which allows the graph to continue uninterrupted. Thanks for your help on this! Richard _______________________________________________ rrd-users mailing list rrd-users@... https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users |
|
|
Re: Missing data overnight from daylight saving changes
by Tobias Oetiker-3
::
Rate this Message:
Richard,
Today Richard Curtis wrote: > >> Am I missing something here? I did insert a value for 256432400, but > >> it looks like rrdtool is expecting a value for both 01:00 BST and > >> 01:00 GMT. > > > > It needs a value for 1256428800 to keep the sequence intact. > > > > It doesn't care that one is 01:00 this or 01:00 that. It just cares > > that there's a gap of 7200 seconds, which exceeds the heartbeat limit of > > 3600. > > Ah, now it makes sense. > My script to collect the data runs via cron which ensures the script > only ones once at DST change, so I am missing the data point for 01:00 > BST (rather, my loader script assumes the entry for 01:00 on the 25th > Oct is GMT so there is nothing inserted. > > I've fixed in now based on this - I allow a 7200 second gap which > allows the graph to continue uninterrupted. only deals in GMT/UTC as such it does not care about daylight saving time ... it is merily a presentation thing. cheers tobi > _______________________________________________ > rrd-users mailing list > rrd-users@... > https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users > > -- Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland http://it.oetiker.ch tobi@... ++41 62 775 9902 / sb: -9900 _______________________________________________ rrd-users mailing list rrd-users@... https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users |
| Free embeddable forum powered by Nabble | Forum Help |
