Re: Memory leaks - what causes them?

Just as a follow up to this – I finally had a bit of time to play with this and I think that I found the issue.

 

It turns out that I had three NoExpire triggers set for “common” modules that I had installed at some stage but subsequently deactivated/disabled.  I deleted all three of the redundant triggers and now my leak (hemorrhage would be more appropriate) appears to have stopped.  The modules were “get AP news” (get_ap_breaking.pl), “get top 10 list” (internet_top10.pl), and “get daily news” (news_yahoo.pl).  I deleted all 3 triggers in one hit so I don’t know if it was one particular script that was causing the problem or if it was caused by MH trying to call routines that didn’t exist – the latter seems more likely.

 

It’s now been 40 minutes since I removed the modules and MH hasn’t leaked (apart from a ~2mb leak straight after the reload) – previously I was getting one every 5-6 minutes.

 

 

Hi David,

 

I’ve modified code/common/monitor_memory.pl so that it runs every minute rather than every 10 minutes and I’ve reduced the reporting threshold from 0.5MB to 0.05MB.

 

I am running the weather graphing script and although I haven’t yet done what you suggested, it is looking like it may be the culprit because the leak spikes at hh:01, hh:11, hh:21, etc which suggests that the leak is being caused by something that runs every 10 minutes.

 

However, equally disturbing is the ~2MB leak that occurs after each of the reloads.  It appears that MH reloads itself after each trigger event because the two at 17:20 correlate to me manually reloading the code and the reloads at 17:35, 19:30 and 20:30 correlate with some one-shot triggers that I had set.

 

I searched back through my archive of the mailing list and through the wiki for info on memory leaks and came up with very little.  Definitely keen to see something written up on them and I’m happy to do so if/when we work out what’s causing them.

 

 

Wed 11/02/09 17:00:01  49.0 hours:   0.2 MB in  0.1 hours.  211.2 -> 211.4 at  1.17 MB/hour.  Total:  3.31 MB/hour

Wed 11/02/09 17:01:00  49.0 hours:   0.2 MB in  0.0 hours.  211.4 -> 211.6 at 11.04 MB/hour.  Total:  3.32 MB/hour

Wed 11/02/09 17:02:00  49.0 hours:   0.2 MB in  0.0 hours.  211.6 -> 211.7 at 10.08 MB/hour.  Total:  3.32 MB/hour

Wed 11/02/09 17:11:00  49.2 hours:   0.4 MB in  0.1 hours.  211.7 -> 212.1 at  2.69 MB/hour.  Total:  3.32 MB/hour

Wed 11/02/09 17:20:00  49.3 hours:   0.6 MB in  0.1 hours.  212.1 -> 212.7 at  4.08 MB/hour.  Total:  3.32 MB/hour

Wed 11/02/09 17:20:16    ReLoad

Wed 11/02/09 17:20:25    ReLoad

Wed 11/02/09 17:21:01  49.4 hours:   2.3 MB in  0.0 hours.  212.7 -> 215.0 at 138.00 MB/hour.  Total:  3.36 MB/hour

Wed 11/02/09 17:35:10    ReLoad

Wed 11/02/09 17:36:01  49.6 hours:   1.5 MB in  0.2 hours.  215.0 -> 216.5 at  5.87 MB/hour.  Total:  3.38 MB/hour

Wed 11/02/09 17:51:01  49.9 hours:   0.3 MB in  0.2 hours.  216.5 -> 216.8 at  1.30 MB/hour.  Total:  3.37 MB/hour

Wed 11/02/09 17:58:00  50.0 hours:   0.2 MB in  0.1 hours.  216.8 -> 217.0 at  1.44 MB/hour.  Total:  3.36 MB/hour

Wed 11/02/09 18:01:00  50.0 hours:   0.3 MB in  0.1 hours.  217.0 -> 217.3 at  5.04 MB/hour.  Total:  3.36 MB/hour

Wed 11/02/09 18:02:00  50.0 hours:   0.2 MB in  0.0 hours.  217.3 -> 217.5 at 12.72 MB/hour.  Total:  3.37 MB/hour

Wed 11/02/09 18:10:00  50.2 hours:   0.1 MB in  0.1 hours.  217.5 -> 217.6 at  1.02 MB/hour.  Total:  3.36 MB/hour

Wed 11/02/09 18:11:00  50.2 hours:   0.2 MB in  0.0 hours.  217.6 -> 217.8 at 11.04 MB/hour.  Total:  3.36 MB/hour

Wed 11/02/09 18:12:01  50.2 hours:   0.2 MB in  0.0 hours.  217.8 -> 218.0 at 12.48 MB/hour.  Total:  3.37 MB/hour

Wed 11/02/09 18:20:00  50.3 hours:   0.1 MB in  0.1 hours.  218.0 -> 218.1 at  0.99 MB/hour.  Total:  3.36 MB/hour

Wed 11/02/09 18:21:01  50.4 hours:   0.2 MB in  0.0 hours.  218.1 -> 218.3 at 12.72 MB/hour.  Total:  3.36 MB/hour

Wed 11/02/09 18:22:00  50.4 hours:   0.2 MB in  0.0 hours.  218.3 -> 218.5 at 11.76 MB/hour.  Total:  3.37 MB/hour

Wed 11/02/09 18:31:00  50.5 hours:   0.4 MB in  0.1 hours.  218.5 -> 219.0 at  2.83 MB/hour.  Total:  3.36 MB/hour

Wed 11/02/09 18:38:00  50.6 hours:   0.1 MB in  0.1 hours.  219.0 -> 219.1 at  1.20 MB/hour.  Total:  3.36 MB/hour

Wed 11/02/09 18:41:00  50.7 hours:   0.3 MB in  0.1 hours.  219.1 -> 219.4 at  5.92 MB/hour.  Total:  3.36 MB/hour

Wed 11/02/09 18:43:00  50.7 hours:   0.2 MB in  0.0 hours.  219.4 -> 219.6 at  5.52 MB/hour.  Total:  3.36 MB/hour

Wed 11/02/09 18:51:01  50.9 hours:   0.3 MB in  0.1 hours.  219.6 -> 219.9 at  2.49 MB/hour.  Total:  3.36 MB/hour

Wed 11/02/09 18:52:00  50.9 hours:   0.2 MB in  0.0 hours.  219.9 -> 220.1 at 11.52 MB/hour.  Total:  3.36 MB/hour

Wed 11/02/09 19:01:00  51.0 hours:   0.4 MB in  0.1 hours.  220.1 -> 220.5 at  2.91 MB/hour.  Total:  3.36 MB/hour

Wed 11/02/09 19:08:00  51.1 hours:   0.1 MB in  0.1 hours.  220.5 -> 220.7 at  1.17 MB/hour.  Total:  3.36 MB/hour

Wed 11/02/09 19:11:00  51.2 hours:   0.3 MB in  0.1 hours.  220.7 -> 221.0 at  5.68 MB/hour.  Total:  3.36 MB/hour

Wed 11/02/09 19:13:00  51.2 hours:   0.2 MB in  0.0 hours.  221.0 -> 221.2 at  6.00 MB/hour.  Total:  3.36 MB/hour

Wed 11/02/09 19:21:01  51.4 hours:   0.3 MB in  0.1 hours.  221.2 -> 221.5 at  2.31 MB/hour.  Total:  3.36 MB/hour

Wed 11/02/09 19:22:00  51.4 hours:   0.2 MB in  0.0 hours.  221.5 -> 221.7 at 12.96 MB/hour.  Total:  3.36 MB/hour

Wed 11/02/09 19:30:14    ReLoad

Wed 11/02/09 19:31:00  51.5 hours:   1.9 MB in  0.1 hours.  221.7 -> 223.6 at 12.75 MB/hour.  Total:  3.39 MB/hour

Wed 11/02/09 19:48:01  51.8 hours:   0.1 MB in  0.3 hours.  223.6 -> 223.7 at  0.49 MB/hour.  Total:  3.37 MB/hour

Wed 11/02/09 19:51:01  51.9 hours:   0.3 MB in  0.1 hours.  223.7 -> 224.0 at  5.28 MB/hour.  Total:  3.37 MB/hour

Wed 11/02/09 19:52:00  51.9 hours:   0.2 MB in  0.0 hours.  224.0 -> 224.2 at 12.48 MB/hour.  Total:  3.38 MB/hour

Wed 11/02/09 19:59:00  52.0 hours:   0.1 MB in  0.1 hours.  224.2 -> 224.4 at  1.20 MB/hour.  Total:  3.37 MB/hour

Wed 11/02/09 20:01:00  52.0 hours:   0.2 MB in  0.0 hours.  224.4 -> 224.6 at  6.72 MB/hour.  Total:  3.37 MB/hour

Wed 11/02/09 20:02:00  52.0 hours:   0.2 MB in  0.0 hours.  224.6 -> 224.7 at  9.84 MB/hour.  Total:  3.38 MB/hour

Wed 11/02/09 20:11:00  52.2 hours:   0.4 MB in  0.1 hours.  224.7 -> 225.1 at  2.64 MB/hour.  Total:  3.37 MB/hour

Wed 11/02/09 20:17:00  52.3 hours:   0.2 MB in  0.1 hours.  225.1 -> 225.3 at  1.60 MB/hour.  Total:  3.37 MB/hour

Wed 11/02/09 20:21:01  52.4 hours:   0.3 MB in  0.1 hours.  225.3 -> 225.6 at  4.56 MB/hour.  Total:  3.37 MB/hour

Wed 11/02/09 20:22:00  52.4 hours:   0.2 MB in  0.0 hours.  225.6 -> 225.8 at 10.32 MB/hour.  Total:  3.38 MB/hour

Wed 11/02/09 20:30:01  52.5 hours:   0.1 MB in  0.1 hours.  225.8 -> 225.9 at  1.08 MB/hour.  Total:  3.37 MB/hour

Wed 11/02/09 20:30:13    ReLoad

Wed 11/02/09 20:31:00  52.5 hours:   1.8 MB in  0.0 hours.  225.9 -> 227.8 at 110.16 MB/hour.  Total:  3.40 MB/hour

Wed 11/02/09 20:46:00  52.8 hours:   0.1 MB in  0.2 hours.  227.8 -> 227.9 at  0.54 MB/hour.  Total:  3.39 MB/hour

Wed 11/02/09 20:50:00  52.8 hours:   0.1 MB in  0.1 hours.  227.9 -> 228.0 at  1.98 MB/hour.  Total:  3.39 MB/hour

Wed 11/02/09 20:51:01  52.9 hours:   0.2 MB in  0.0 hours.  228.0 -> 228.2 at 11.04 MB/hour.  Total:  3.39 MB/hour

Wed 11/02/09 20:52:00  52.9 hours:   0.2 MB in  0.0 hours.  228.2 -> 228.4 at 11.28 MB/hour.  Total:  3.39 MB/hour

Wed 11/02/09 21:00:01  53.0 hours:   0.1 MB in  0.1 hours.  228.4 -> 228.5 at  1.05 MB/hour.  Total:  3.39 MB/hour

Wed 11/02/09 21:01:00  53.0 hours:   0.2 MB in  0.0 hours.  228.5 -> 228.7 at 11.04 MB/hour.  Total:  3.39 MB/hour

Wed 11/02/09 21:02:00  53.0 hours:   0.2 MB in  0.0 hours.  228.7 -> 228.9 at 12.00 MB/hour.  Total:  3.39 MB/hour

Wed 11/02/09 21:10:00  53.2 hours:   0.1 MB in  0.1 hours.  228.9 -> 229.0 at  0.99 MB/hour.  Total:  3.39 MB/hour

Wed 11/02/09 21:11:00  53.2 hours:   0.2 MB in  0.0 hours.  229.0 -> 229.3 at 12.48 MB/hour.  Total:  3.39 MB/hour

Wed 11/02/09 21:12:01  53.2 hours:   0.2 MB in  0.0 hours.  229.3 -> 229.4 at 10.80 MB/hour.  Total:  3.39 MB/hour