Wiki event log

[Dave Evans-2]

Already knew that Moin kept an "edit log" (a log of each time a page or
attachment was created/modified/deleted) - apparently it also keeps an "event
log", which includes such trivia as each and every time a Wiki page is viewed.

The edit log is at 5,000 lines (600Kb); that's fine, and it's useful
information.

The event log however is at 200,000 lines (47Mb) and only contains
VIEWPAGE events (useless IMO, and we have web access logs anyway); and
SAVEPAGE events (which I'm guessing correspond to the edit-log entries
anyway).  Plus the daily backup is whining every day that the event-log
changes while it's being backed up (and at that size, I'm not surprised).
Therefore I'd like to "logrotate" the event-log (daily, keep 7 days).

Any objections?  Let me know ASAP if so.

Thanks,

--
Dave Evans

PGP key: http://rudolf.org.uk/pgpkey


_______________________________________________
Musicbrainz-experts mailing list
Musicbrainz-experts@...
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-experts

[DonRedman]

On Sat, 28 Jan 2006 15:47:21 +0100, Dave Evans wrote:

> Already knew that Moin kept an "edit log" (a log of each time a page or
> attachment was created/modified/deleted) - apparently it also keeps an  
> "event
> log", which includes such trivia as each and every time a Wiki page is  
> viewed.
>
> The edit log is at 5,000 lines (600Kb); that's fine, and it's useful
> information.
>
> The event log however is at 200,000 lines (47Mb) and only contains
> VIEWPAGE events (useless IMO, and we have web access logs anyway); and
> SAVEPAGE events (which I'm guessing correspond to the edit-log entries
> anyway).  Plus the daily backup is whining every day that the event-log
> changes while it's being backed up (and at that size, I'm not surprised).
> Therefore I'd like to "logrotate" the event-log (daily, keep 7 days).
>
> Any objections?  Let me know ASAP if so.

I'd really like to have some pageview stats on the wiki. Having the most  
frequently viewed wiki pages could be really helpful for WikiDocs. I would  
do that myself if this is ok with MB privacy policy (if the log contains  
usernames, it might not), and if you are willing to leave the log  
somewhere for me to download.

This might also be useful in the future if (and only if) the logs offer  
some way to differentiate between transcluded page accesses and wiki page  
accesses. WikiDocs is pioneer work and having some useage data could help  
to understand how things evolve.

If however you say that all of this can be done with the apace access  
logs, then go ahead and throw the moin logs away after seven days.

   DonRedman

--
Words that are written in CamelCase refer to WikiPages:
Visit http://wiki.musicbrainz.org/ the best MusicBrainz documentation  
around! :-)
_______________________________________________
Musicbrainz-experts mailing list
Musicbrainz-experts@...
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-experts

[Dave Evans-2]

On Sat, Jan 28, 2006 at 04:00:36PM +0100, Don Redman wrote:
> If however you say that all of this can be done with the apace access  
> logs, then go ahead and throw the moin logs away after seven days.

http://stats.musicbrainz.org/webstats/wiki/ - is that any use to you?  Of
course when I say we have web access logs, what I mean is that we have *some*
web access logs.  Of course, they are rotated too (daily, discard after 7
days).  Any time you just keep everything (never rotating, never deleting)
you're going to have a problem eventually.  Hence moderation_closed was (and
is) a problem, and the wiki event-log is a (much smaller) problem.  The
edit-log will a problem one day, but that day is a long way off yet.

> I'd really like to have some pageview stats on the wiki. Having the most  
> frequently viewed wiki pages could be really helpful for WikiDocs. I would  
> do that myself if this is ok with MB privacy policy (if the log contains  
> usernames, it might not), and if you are willing to leave the log  
> somewhere for me to download.

> This might also be useful in the future if (and only if) the logs offer  
> some way to differentiate between transcluded page accesses and wiki page  
> accesses. WikiDocs is pioneer work and having some useage data could help  
> to understand how things evolve.

Then I think the right way to do this is to perform daily analysis of the
existing web wiki access log (it's easy to identify WikiDocs requests from
the others), and dump daily anonymised aggregate stats into the FTP site (and
rotate and delete those stats just like we do for the access logs themselves).
Then it's up to you (or whoever wants the info) to grab it every day and do
whatever you want with it.  Let me know if you'd like to press ahead with this.

Regards,

--
Dave Evans

PGP key: http://rudolf.org.uk/pgpkey


_______________________________________________
Musicbrainz-experts mailing list
Musicbrainz-experts@...
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-experts

[DonRedman]

On Sat, 28 Jan 2006 16:22:58 +0100, Dave Evans wrote:

> Then I think the right way to do this is to perform daily analysis of the
> existing web wiki access log (it's easy to identify WikiDocs requests  
> from
> the others), and dump daily anonymised aggregate stats into the FTP site  
> (and
> rotate and delete those stats just like we do for the access logs  
> themselves).
> Then it's up to you (or whoever wants the info) to grab it every day and  
> do
> whatever you want with it.  Let me know if you'd like to press ahead  
> with this.

Not press, really. If this is not too much work, then I might be very  
happy about these logs if we encounter problems with WikiDocs that we do  
not understand.

If we never have such troubles, then I'll never need it. And if it is more  
than the matter of writing a simple script, then please let it be.

Sometimes it is better to ask two or three people, instead of analyzing  
obscure stats for hours :-).

   DonRedman

--
Words that are written in CamelCase refer to WikiPages:
Visit http://wiki.musicbrainz.org/ the best MusicBrainz documentation  
around! :-)
_______________________________________________
Musicbrainz-experts mailing list
Musicbrainz-experts@...
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-experts

[Dave Evans-2]

On Sun, Jan 29, 2006 at 06:11:50PM +0100, Don Redman wrote:
> If we never have such troubles, then I'll never need it. And if it is more  
> than the matter of writing a simple script, then please let it be.

OK then.  It's probably 2-3 hours work, all told, to make those sorts of
stats available (assuming really simple analysis - nothing too clever).  For
now I'll set the event-log rotating, and let me know if you need anything more
than what's already available.

--
Dave Evans

PGP key: http://rudolf.org.uk/pgpkey


_______________________________________________
Musicbrainz-experts mailing list
Musicbrainz-experts@...
http://lists.musicbrainz.org/mailman/listinfo/musicbrainz-experts