Re: hg push getting worse?
|
View:
New views
12 Messages
—
Rating Filter:
Alert me
|
 |
|
|
|
Re: hg push getting worse?
by gnu_andrew
::
Rate this Message:
2009/8/11 John Coomes <John.Coomes@...>:
> Andrew John Hughes (gnu_andrew@...) wrote: >> > ... >> > From my experience with the OpenJDK servers, I think it may be jcheck. >> > Â Unfortunately, jcheck itself is not Free Software, so it is >> > impossible for me to say for definite. Â Instead, we seem to be >> > discovering what checks it performs only when failures occur. >> > >> > I do know it does the following: >> > >> > * Checks the use of whitespace in changesets, rejecting the use of >> > tabs and trailing whitespace. >> > * Checks the format of the commit message. Â It must even follow the >> > bugid/summary/reviewer format documented in the developer guide or >> > simply be 'Merge'. >> > * Checks that the bugid used is not used by any other changeset. >> > >> > I've run into all of these with different changesets. Â I can >> > definitely see how the latter could take a while on something like the >> > jdk repository (which dwarfs the others in size). > > I think jcheck is relatively light-weight compared to the zfs snapshot > and auto-push that are also done. But it would be nice to know for > sure. > True. It's hard to tell from the client side as hg appears to hang 'searching for changes' and then suddenly a burst of information appears, either reporting success or failure. I presume this is part of the design of hg unfortunately, but it would be helpful if things were more verbose. I presume hooks were never meant to utilise enough time that this would become an issue. I can see how zfs snapshotting would take longer over time. Although the snapshots are, I presume, relative to the last so that the size remains fairly sane, the sheer number of them is likely to cause a slowdown. Is there a significant benefit to creating these? And are they also created in Maurizio's FX project? >> > jcheck appears to perform its check on the changeset once committed to >> > the remote repository and then performs a rollback if it fails, >> > although you don't see any of this feedback in realtime on the client. >> > Â I presume this also means it has to get Mercurial to generate the >> > changeset (and others for the bugid check) which would take time. > > That's the standard way that mercurial hooks work, and the reason we > have the extra *-gate forests. In more recent versions of mercurial, > the pending changesets aren't visible until the hooks have completed > successfully, so the *-gate forests become unnecessary. We haven't > been able to update because the server-side of the forest extension > doesn't work w/recent mercurial releases. > Is there any further news on whether the forest extension will become a standard part of Mercurial? When I went searching for a client-side version to support newer versions, I had to resort to using a snapshot of their repository for forest. >> > It would be much better if we could perform these sanity checks >> > locally, though we'd still need some way of checking this had been >> > done on the server side (or who knows, we could trust the developers >> > to have done it...). >> >> I should also note that duplicate bugids can appear completely >> legitimately in certain merge cases. This is the current issue with >> updating OpenJDK6's HotSpot.. Fixes for bugs occur both in the >> original (rebased) OpenJDK6 HotSpot and the copy of HotSpot being >> merged from the hs14 repository, resulting in duplicate bugids which >> jcheck rejects. I presume this is to protect against the case that >> someone mistypes the bugid as another legitimate bugid, but it is >> certainly an expensive check for what seems a fairly unlikely >> occurrence. > > The goal is to prevent people from checking in fixes piece-meal, > scattering a bug fix over 2, 3 or more changesets. Having a bug fix > isolated to a single changeset makes backporting it to a different > release much, much easier. We lived with the practice of partial > fixes for quite some time, and I'd rather not go back. > I agree. I'm just not sure that the best way of enforcing this is a technical one. The OpenJDK6 example is fairly convoluted, but I imagine our IcedTea forest would be prone to this too, were it to be jchecked. We often commit a fix there early, and then also receive the same fix in a later pull from the JDK7 forest. All changes have to be approved first, so could not this issue be flagged at that point? The history could also be scanned for duplicates at that point if someone really wanted to be sure. The case is even worse with the whitespace checks. Given jcheck clearly can find the issues, can it not also fix them? Or could some of these checks be applied on the client-side? Worst case, could the whole repo. not just be sanitised on a regular basis? As is, you push a commit, wait several minutes and then get a result from jcheck saying there is an extra space at the end of one line. So, you have to rollback the commit, remove the space, reapply the commit with the same comments and then push again. In all, a lot of time is spent for very little gain. > -John > > -- Andrew :-) Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) Support Free Java! Contribute to GNU Classpath and the OpenJDK http://www.gnu.org/software/classpath http://openjdk.java.net PGP Key: 94EFD9D8 (http://subkeys.pgp.net) Fingerprint: F8EF F1EA 401E 2E60 15FA 7927 142C 2591 94EF D9D8 |
|
|
Re: hg push getting worse?
by john.coomes
::
Rate this Message:
Andrew John Hughes (gnu_andrew@...) wrote:
> 2009/8/11 John Coomes <John.Coomes@...>: > > I think jcheck is relatively light-weight compared to the zfs snapshot > > and auto-push that are also done. Â But it would be nice to know for > > sure. > > True. It's hard to tell from the client side as hg appears to hang > 'searching for changes' and then suddenly a burst of information > appears, either reporting success or failure. I presume this is part > of the design of hg unfortunately, but it would be helpful if things > were more verbose. I presume hooks were never meant to utilise enough > time that this would become an issue. > > I can see how zfs snapshotting would take longer over time. Although > the snapshots are, I presume, relative to the last so that the size > remains fairly sane, the sheer number of them is likely to cause a > slowdown. Is there a significant benefit to creating these? I don't know the openjdk infrastructure, so am just guessing, but I suspect there are only a finite number of snapshots maintained. The benefit is being able to recover :-). AFAIK, we've only used it once or twice, but having the snapshots made it possible. > ... And are > they also created in Maurizio's FX project? I have no idea. > >> > jcheck appears to perform its check on the changeset once committed to > >> > the remote repository and then performs a rollback if it fails, > >> > although you don't see any of this feedback in realtime on the client. > >> > ÃÂ I presume this also means it has to get Mercurial to generate the > >> > changeset (and others for the bugid check) which would take time. > > > > That's the standard way that mercurial hooks work, and the reason we > > have the extra *-gate forests. Â In more recent versions of mercurial, > > the pending changesets aren't visible until the hooks have completed > > successfully, so the *-gate forests become unnecessary. Â We haven't > > been able to update because the server-side of the forest extension > > doesn't work w/recent mercurial releases. > > > > Is there any further news on whether the forest extension will become > a standard part of Mercurial? When I went searching for a client-side > version to support newer versions, I had to resort to using a snapshot > of their repository for forest. I doubt forest will ever become part of mercurial. Mercurial now has an experimental sub-repo feature for dealing with nested repositories. It doesn't have the flexibility of the forest extension, so wouldn't work for openjdk, at least as it stands now. > ... > All changes have to be approved first, so could not this issue be > flagged at that point? The history could also be scanned for > duplicates at that point if someone really wanted to be sure. The > case is even worse with the whitespace checks. Given jcheck clearly > can find the issues, can it not also fix them? Or could some of these > checks be applied on the client-side? Worst case, could the whole > repo. not just be sanitised on a regular basis? As is, you push a > commit, wait several minutes and then get a result from jcheck saying > there is an extra space at the end of one line. So, you have to > rollback the commit, remove the space, reapply the commit with the > same comments and then push again. In all, a lot of time is spent for > very little gain. I think the problem is access to jcheck. I'm not sure why it isn't available externally. We run jcheck on our local repos before pushing, and it's reasonably convenient. Or maybe I should say not too inconvenient :-). -John |
|
|
Re: hg push getting worse?
by jonathan.gibbons
::
Rate this Message:
John Coomes wrote:
Andrew John Hughes (gnu_andrew@...) wrote:2009/8/11 John Coomes John.Coomes@...: What are the issues with the Mercurial sub-repos. It seems close to what we want. What is the lack of flexibility that you have observed that concerns you? -- Jon |
|
|
Re: hg push getting worse?
by gnu_andrew
::
Rate this Message:
2009/8/11 John Coomes <John.Coomes@...>:
> Andrew John Hughes (gnu_andrew@...) wrote: >> 2009/8/11 John Coomes <John.Coomes@...>: >> > I think jcheck is relatively light-weight compared to the zfs snapshot >> > and auto-push that are also done.  But it would be nice to know for >> > sure. >> >> True. It's hard to tell from the client side as hg appears to hang >> 'searching for changes' and then suddenly a burst of information >> appears, either reporting success or failure. I presume this is part >> of the design of hg unfortunately, but it would be helpful if things >> were more verbose. I presume hooks were never meant to utilise enough >> time that this would become an issue. >> >> I can see how zfs snapshotting would take longer over time. Although >> the snapshots are, I presume, relative to the last so that the size >> remains fairly sane, the sheer number of them is likely to cause a >> slowdown. Is there a significant benefit to creating these? > > I don't know the openjdk infrastructure, so am just guessing, Same here. but I > suspect there are only a finite number of snapshots maintained. The > benefit is being able to recover :-). AFAIK, we've only used it once > or twice, but having the snapshots made it possible. > Ah, sounds worth it then ;) >> ... And are >> they also created in Maurizio's FX project? > > I have no idea. > >> >> > jcheck appears to perform its check on the changeset once committed to >> >> > the remote repository and then performs a rollback if it fails, >> >> > although you don't see any of this feedback in realtime on the client. >> >> >  I presume this also means it has to get Mercurial to generate the >> >> > changeset (and others for the bugid check) which would take time. >> > >> > That's the standard way that mercurial hooks work, and the reason we >> > have the extra *-gate forests.  In more recent versions of mercurial, >> > the pending changesets aren't visible until the hooks have completed >> > successfully, so the *-gate forests become unnecessary.  We haven't >> > been able to update because the server-side of the forest extension >> > doesn't work w/recent mercurial releases. >> > >> >> Is there any further news on whether the forest extension will become >> a standard part of Mercurial? When I went searching for a client-side >> version to support newer versions, I had to resort to using a snapshot >> of their repository for forest. > > I doubt forest will ever become part of mercurial. Mercurial now has > an experimental sub-repo feature for dealing with nested repositories. > It doesn't have the flexibility of the forest extension, so wouldn't > work for openjdk, at least as it stands now. > >> ... >> All changes have to be approved first, so could not this issue be >> flagged at that point? The history could also be scanned for >> duplicates at that point if someone really wanted to be sure. The >> case is even worse with the whitespace checks. Given jcheck clearly >> can find the issues, can it not also fix them? Or could some of these >> checks be applied on the client-side? Worst case, could the whole >> repo. not just be sanitised on a regular basis? As is, you push a >> commit, wait several minutes and then get a result from jcheck saying >> there is an extra space at the end of one line. So, you have to >> rollback the commit, remove the space, reapply the commit with the >> same comments and then push again. In all, a lot of time is spent for >> very little gain. > > I think the problem is access to jcheck. I'm not sure why it isn't > available externally. We run jcheck on our local repos before > pushing, and it's reasonably convenient. Or maybe I should say not > too inconvenient :-). > Exactly. Now I know about them (through failed changesets), I could probably come up with my own checks for the whitespace issues I mentioned, but I have no idea what other checks may be being performed by jcheck. It's a black box, and our only evidence of its behaviour is by testing with our changesets as input. > -John > > -- Andrew :-) Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) Support Free Java! Contribute to GNU Classpath and the OpenJDK http://www.gnu.org/software/classpath http://openjdk.java.net PGP Key: 94EFD9D8 (http://subkeys.pgp.net) Fingerprint: F8EF F1EA 401E 2E60 15FA 7927 142C 2591 94EF D9D8 |
|
|
Re: hg push getting worse?
by john.coomes
::
Rate this Message:
Jonathan Gibbons (Jonathan.Gibbons@...) wrote:
> John Coomes wrote: > > Andrew John Hughes (gnu_andrew@...) wrote: > > > >> 2009/8/11 John Coomes <John.Coomes@...>: > >> > >> Is there any further news on whether the forest extension will become > >> a standard part of Mercurial? When I went searching for a client-side > >> version to support newer versions, I had to resort to using a snapshot > >> of their repository for forest. > > > > I doubt forest will ever become part of mercurial. Mercurial now has > > an experimental sub-repo feature for dealing with nested repositories. > > It doesn't have the flexibility of the forest extension, so wouldn't > > work for openjdk, at least as it stands now. > > > What are the issues with the Mercurial sub-repos. It seems close to what > we want. > What is the lack of flexibility that you have observed that concerns you? The main one is the inability to get a partial tree. But there's also the inability to avoid recursing into sub-repos when doing an update. There were a couple of requests for the latter on the mercurial-devel list, but I don't think anything has happened. And it seems to be just an implementation problem, but the location of the sub-repos is kept in a versioned file which can't be overridden from the command line. -John |
|
|
Re: hg push getting worse?
by jonathan.gibbons
::
Rate this Message:
John Coomes wrote:
Jonathan Gibbons (Jonathan.Gibbons@...) wrote: Ouch. Thanks for the update. -- Jon |
|
|
Re: hg push getting worse?
by tim.bell
::
Rate this Message:
> I don't know the openjdk infrastructure, so am just guessing, but I
> suspect there are only a finite number of snapshots maintained. The > benefit is being able to recover :-). AFAIK, we've only used it once > or twice, but having the snapshots made it possible. I was just on the hg server to shake loose a changeset in the jdk6-gate/langtools repo. The most obvious thing is that the push script is spending a lot of time running '/sbin/zfs list -Ho mountpoint,name' Running only that command by hand takes 3 mins 17 secs and produces 11485 lines of output. During that time, truss reports hundreds of these: brk(0x0CC6F000) = 0 ioctl(3, ZFS_IOC_SNAPSHOT_LIST_NEXT, 0x08042E08) = 0 ioctl(3, ZFS_IOC_OBJSET_STATS, 0x08040F68) = 0 ioctl(3, ZFS_IOC_SNAPSHOT_LIST_NEXT, 0x08042E08) = 0 ioctl(3, ZFS_IOC_OBJSET_STATS, 0x08040F68) = 0 ioctl(3, ZFS_IOC_SNAPSHOT_LIST_NEXT, 0x08042E08) = 0 ioctl(3, ZFS_IOC_OBJSET_STATS, 0x08040F68) = 0 ioctl(3, ZFS_IOC_SNAPSHOT_LIST_NEXT, 0x08042E08) = 0 ioctl(3, ZFS_IOC_OBJSET_STATS, 0x08040F68) = 0 ioctl(3, ZFS_IOC_SNAPSHOT_LIST_NEXT, 0x08042E08) = 0 ioctl(3, ZFS_IOC_OBJSET_STATS, 0x08040F68) = 0 ZFS snapshots are supposed to be lightweight and inexpensive. We still have them going back to the beginning of time on hg.ojn. Perhaps we have accumulated enough to expose quadratic (or worse) performance problems. I will do some searching over on OpenSolaris. Tim |
|
|
Re: hg push getting worse?
by tim.bell
::
Rate this Message:
I wrote:
> ZFS snapshots are supposed to be lightweight and inexpensive. We > still have them going back to the beginning of time on hg.ojn. > > Perhaps we have accumulated enough to expose quadratic (or worse) > performance problems. I will do some searching over on OpenSolaris. I spent ten or fifteen minutes searching on this issue. The short version is that there has been a significant amount of ZFS performance work done since "Solaris Express Developer Edition 9/07 snv_70a X86" which is the release hg.ojn is running. That was the best choice back when we set the server up, but now it is pretty old. Here are a few examples - all of these are marked FIXed in builds more recent than 70a. (FYI: OpenSolaris is currently up around b121): 6596190 ""zfs list" is slow due to version property" http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6596190 6386929 "initial "zfs list" is slow" http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6386929 6755389 "Initial run of any zfs command that iterates over datasets can be slow" http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6755389 Here is a message thread related to 'zfs list' being slow: http://mail.opensolaris.org/pipermail/zfs-discuss/2008-July/019956.html hg.ojn has been extremely stable since we set it up - it currently has 652 days of uptime. All the same, it it probably time to start planning an upgrade. Tim |
|
|
Re: hg push getting worse?
by jonathan.gibbons
::
Rate this Message:
Any chance of that upgrade happening? What would it take to make it
happen?
-- Jon Tim Bell wrote: I wrote: |
|
|
Re: hg push getting worse?
by jonathan.gibbons
::
Rate this Message:
Pushes to hg.ojn now take over 10 minutes. What does it take to fix
this before
it's faster to do a Teamware bringover of the j2se repository than to push a couple of files to hg.ojn? -- Jon real 10m7.790s Jonathan Gibbons wrote: Any chance of that upgrade happening? What would it take to make it happen? |
|
|
|
| Free embeddable forum powered by Nabble | Forum Help |
