CVSup src file Edits almost alway triggering Checksum mismatch and Fixup

[John Marshall-2]

A dialup user of our CVSup mirror alerted me to this.

These days, almost every file updated in the src tree triggers a
Checksum mismatch, full file download and Fixup.  I'm not seeing this
for doc, ports, etc.; just src.

For example (from the logs of our synch's with cvsup-master)...

---------------------------------------
 Edit ports/math/jags/Makefile,v
 Edit ports/math/jags/distinfo,v
 Edit src/release/Makefile,v
src/release/Makefile,v: Checksum mismatch -- will transfer entire file
 Create src/release/svnbranch.awk,v
 Edit src/sys/amd64/amd64/local_apic.c,v
src/sys/amd64/amd64/local_apic.c,v: Checksum mismatch -- will transfer entire file
 Edit src/sys/amd64/include/apicvar.h,v
src/sys/amd64/include/apicvar.h,v: Checksum mismatch -- will transfer entire file
 Edit src/sys/amd64/include/pmc_mdep.h,v
src/sys/amd64/include/pmc_mdep.h,v: Checksum mismatch -- will transfer entire file
 Edit src/sys/dev/hwpmc/hwpmc_core.c,v
 Edit src/sys/dev/hwpmc/hwpmc_piv.c,v
src/sys/dev/hwpmc/hwpmc_piv.c,v: Checksum mismatch -- will transfer entire file
 Edit src/sys/dev/hwpmc/hwpmc_ppro.c,v
src/sys/dev/hwpmc/hwpmc_ppro.c,v: Checksum mismatch -- will transfer entire file
 Edit src/sys/dev/hwpmc/hwpmc_x86.c,v
src/sys/dev/hwpmc/hwpmc_x86.c,v: Checksum mismatch -- will transfer entire file
 Edit src/sys/i386/i386/local_apic.c,v
src/sys/i386/i386/local_apic.c,v: Checksum mismatch -- will transfer entire file
 Edit src/sys/i386/include/apicvar.h,v
src/sys/i386/include/apicvar.h,v: Checksum mismatch -- will transfer entire file
 Edit src/sys/i386/include/pmc_mdep.h,v
src/sys/i386/include/pmc_mdep.h,v: Checksum mismatch -- will transfer entire file
 Edit src/sys/pci/agp_i810.c,v -> Attic
src/sys/pci/agp_i810.c,v: Checksum mismatch -- will transfer entire file
---------------------------------------

Followed by...

---------------------------------------
Applying fixups for collection cvs-all/cvs
 Fixup src/release/Makefile,v
 Fixup src/sys/amd64/amd64/local_apic.c,v
 Fixup src/sys/amd64/include/apicvar.h,v
 Fixup src/sys/amd64/include/pmc_mdep.h,v
 Fixup src/sys/dev/hwpmc/hwpmc_piv.c,v
 Fixup src/sys/dev/hwpmc/hwpmc_ppro.c,v
 Fixup src/sys/dev/hwpmc/hwpmc_x86.c,v
 Fixup src/sys/i386/i386/local_apic.c,v
 Fixup src/sys/i386/include/apicvar.h,v
 Fixup src/sys/i386/include/pmc_mdep.h,v
 Fixup src/sys/pci/agp_i810.c,v -> Attic
---------------------------------------

I am not seeing Fixups for anything other than the src tree.  I've
checked back through the past month's logs and the story is the same.
Is this a consequence of SVN export to CVS?  Can it be fixed?  Am I just
doing something wrong?

--
John Marshall

[Andre Albsmeier]

On Sat, 15-Aug-2009 at 11:34:41 +1000, John Marshall wrote:
> A dialup user of our CVSup mirror alerted me to this.
>
> These days, almost every file updated in the src tree triggers a
> Checksum mismatch, full file download and Fixup.  I'm not seeing this
> for doc, ports, etc.; just src.

Me too ;-)

The problem started (for me) a few month ago.

I see the "Checksum mismatch" messages as well. Sometimes two
or three, sometimes 100 or more, sometimes none.

I used CVSup for years. I tried csup, nothing changed. I switched
servers I CVSup from, even tried cvsup.freebsd.org.  I tried
compressing and non-compressing mode.

Like John, I only see the problem on src, not elsewhere.

Any ideas?

Thanks,

        -Andre

>
> For example (from the logs of our synch's with cvsup-master)...
>
> ---------------------------------------
>  Edit ports/math/jags/Makefile,v
>  Edit ports/math/jags/distinfo,v
>  Edit src/release/Makefile,v
> src/release/Makefile,v: Checksum mismatch -- will transfer entire file
>  Create src/release/svnbranch.awk,v
>  Edit src/sys/amd64/amd64/local_apic.c,v
> src/sys/amd64/amd64/local_apic.c,v: Checksum mismatch -- will transfer entire file
>  Edit src/sys/amd64/include/apicvar.h,v
> src/sys/amd64/include/apicvar.h,v: Checksum mismatch -- will transfer entire file
>  Edit src/sys/amd64/include/pmc_mdep.h,v
> src/sys/amd64/include/pmc_mdep.h,v: Checksum mismatch -- will transfer entire file
>  Edit src/sys/dev/hwpmc/hwpmc_core.c,v
>  Edit src/sys/dev/hwpmc/hwpmc_piv.c,v
> src/sys/dev/hwpmc/hwpmc_piv.c,v: Checksum mismatch -- will transfer entire file
>  Edit src/sys/dev/hwpmc/hwpmc_ppro.c,v
> src/sys/dev/hwpmc/hwpmc_ppro.c,v: Checksum mismatch -- will transfer entire file
>  Edit src/sys/dev/hwpmc/hwpmc_x86.c,v
> src/sys/dev/hwpmc/hwpmc_x86.c,v: Checksum mismatch -- will transfer entire file
>  Edit src/sys/i386/i386/local_apic.c,v
> src/sys/i386/i386/local_apic.c,v: Checksum mismatch -- will transfer entire file
>  Edit src/sys/i386/include/apicvar.h,v
> src/sys/i386/include/apicvar.h,v: Checksum mismatch -- will transfer entire file
>  Edit src/sys/i386/include/pmc_mdep.h,v
> src/sys/i386/include/pmc_mdep.h,v: Checksum mismatch -- will transfer entire file
>  Edit src/sys/pci/agp_i810.c,v -> Attic
> src/sys/pci/agp_i810.c,v: Checksum mismatch -- will transfer entire file
> ---------------------------------------
>
> Followed by...
>
> ---------------------------------------
> Applying fixups for collection cvs-all/cvs
>  Fixup src/release/Makefile,v
>  Fixup src/sys/amd64/amd64/local_apic.c,v
>  Fixup src/sys/amd64/include/apicvar.h,v
>  Fixup src/sys/amd64/include/pmc_mdep.h,v
>  Fixup src/sys/dev/hwpmc/hwpmc_piv.c,v
>  Fixup src/sys/dev/hwpmc/hwpmc_ppro.c,v
>  Fixup src/sys/dev/hwpmc/hwpmc_x86.c,v
>  Fixup src/sys/i386/i386/local_apic.c,v
>  Fixup src/sys/i386/include/apicvar.h,v
>  Fixup src/sys/i386/include/pmc_mdep.h,v
>  Fixup src/sys/pci/agp_i810.c,v -> Attic
> ---------------------------------------
>
> I am not seeing Fixups for anything other than the src tree.  I've
> checked back through the past month's logs and the story is the same.
> Is this a consequence of SVN export to CVS?  Can it be fixed?  Am I just
> doing something wrong?
>
> --
> John Marshall

--
Never argue with an idiot. They drag you down to
their level, then beat you with their experience.
_______________________________________________
freebsd-hubs@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hubs
To unsubscribe, send any mail to "freebsd-hubs-unsubscribe@..."

[Simon L. Nielsen]

On 2009.08.15 11:34:41 +1000, John Marshall wrote:

> These days, almost every file updated in the src tree triggers a
> Checksum mismatch, full file download and Fixup.  I'm not seeing this
> for doc, ports, etc.; just src.
>
[...]
>  Edit src/release/Makefile,v
> src/release/Makefile,v: Checksum mismatch -- will transfer entire file

Hey,

This is unfortunatly a known problem.  The problem seems to be that
the actually diff sections in the ,v files are at time re-oredered and
that breaks CVSup's assumptions about the ,v files.

The ,v are still valid in both before and after reorder, the order is
just changed.  Since cvsup's normal acelerated CVS mirror mode works
by knowning how thw ,v files are layed out, cvsup then can just see
that the ,v file does not look like it expects, and it does the safe
thing and transfer/sync the entire file.  I actually suspect if CVSup
didn't do this but just transfered the delta's to the ,v files, the
resulting ,v file on the client would be different, but working and
valid - but this is a guess.

kib@ (AFAIR) first noticed that this happened for security updates
which went to multiple branches at once in the spring, but it was
still only for a few files.

After the upgrade of CVS to 1.12 in base (and therefor on repoman -
the master CVS server) it seems to be that the reason all files are
now have this problem - so CVS 1.12 probably made it worse.

I suspect the reason it happens for src, but not ports or doc, is the
branches.  When there are no branches in a ,v file the ordering is
simply newest -> oldest, but when there are branches I don't think the
order is clear.

So, in short I don't see any easy solutions to this, and especially
given the fact that in the long run CVS/CVSup will probably be
replaced by something else for distributing FreeBSD source, I don't
think it will be that easy to find someone to "fix" either CVSup or
CVS.

PS. Just to be clear CVS/CVSup is not going away any time soon (though
there are no timeline planed yet), but 3-5 years down the line I
really doubt we will be using CVSup.

--
Simon L. Nielsen
Hats: FreeBSD.org cluster, cvs, and svn admin teams
_______________________________________________
freebsd-hubs@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hubs
To unsubscribe, send any mail to "freebsd-hubs-unsubscribe@..."

[Garance A Drosehn]

At 8:47 PM +0200 8/18/09, Simon L. Nielsen wrote:

>On 2009.08.15 11:34:41 +1000, John Marshall wrote:
>
>>  These days, almost every file updated in the src tree triggers a
>>  Checksum mismatch, full file download and Fixup.  I'm not seeing this
>>  for doc, ports, etc.; just src.
>>
>[...]
>>   Edit src/release/Makefile,v
>>  src/release/Makefile,v: Checksum mismatch -- will transfer entire file
>
>Hey,
>
>This is unfortunatly a known problem.  The problem seems to be that
>the actually diff sections in the ,v files are at time re-oredered
>and that breaks CVSup's assumptions about the ,v files.
>
>The ,v are still valid in both before and after reorder, the order
>is just changed.  [...]

>After the upgrade of CVS to 1.12 in base (and therefor on repoman -
>the master CVS server) it seems to be that the reason all files are
>now have this problem - so CVS 1.12 probably made it worse.
>
>I suspect the reason it happens for src, but not ports or doc, is the
>branches.  When there are no branches in a ,v file the ordering is
>simply newest -> oldest, but when there are branches I don't think
>the order is clear.

Hmm.  I've noticed this too, and assumed that it has something to
do with the SVN->CVS conversion process (which is why it would not
effect ports or doc).  When I've skimmed over some of my cvsup logs,
I noticed it doesn't happen for all files.  I've been meaning to
check what is noteable about the changes where the whole file has
to be transfered compared to those where it doesn't.

--
Garance Alistair Drosehn     =               drosehn@...
Senior Systems Programmer               or   gad@...
Rensselaer Polytechnic Institute;             Troy, NY;  USA
_______________________________________________
freebsd-hubs@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hubs
To unsubscribe, send any mail to "freebsd-hubs-unsubscribe@..."

[Perrin, Michael]

Any update on a solution to this problem?

-----Original Message-----
From: owner-freebsd-hubs@...
[mailto:owner-freebsd-hubs@...] On Behalf Of Andre Albsmeier
Sent: Monday, August 17, 2009 8:34 AM
To: freebsd-hubs@...
Subject: Re: CVSup src file Edits almost alway triggering Checksum
mismatch and Fixup

On Sat, 15-Aug-2009 at 11:34:41 +1000, John Marshall wrote:
> A dialup user of our CVSup mirror alerted me to this.
>
> These days, almost every file updated in the src tree triggers a
> Checksum mismatch, full file download and Fixup.  I'm not seeing this
> for doc, ports, etc.; just src.

Me too ;-)

The problem started (for me) a few month ago.

I see the "Checksum mismatch" messages as well. Sometimes two
or three, sometimes 100 or more, sometimes none.

I used CVSup for years. I tried csup, nothing changed. I switched
servers I CVSup from, even tried cvsup.freebsd.org.  I tried
compressing and non-compressing mode.

Like John, I only see the problem on src, not elsewhere.

Any ideas?

Thanks,

        -Andre

>
> For example (from the logs of our synch's with cvsup-master)...
>
> ---------------------------------------
>  Edit ports/math/jags/Makefile,v
>  Edit ports/math/jags/distinfo,v
>  Edit src/release/Makefile,v
> src/release/Makefile,v: Checksum mismatch -- will transfer entire file
>  Create src/release/svnbranch.awk,v
>  Edit src/sys/amd64/amd64/local_apic.c,v
> src/sys/amd64/amd64/local_apic.c,v: Checksum mismatch -- will transfer

entire file
>  Edit src/sys/amd64/include/apicvar.h,v
> src/sys/amd64/include/apicvar.h,v: Checksum mismatch -- will transfer
entire file
>  Edit src/sys/amd64/include/pmc_mdep.h,v
> src/sys/amd64/include/pmc_mdep.h,v: Checksum mismatch -- will transfer
entire file
>  Edit src/sys/dev/hwpmc/hwpmc_core.c,v
>  Edit src/sys/dev/hwpmc/hwpmc_piv.c,v
> src/sys/dev/hwpmc/hwpmc_piv.c,v: Checksum mismatch -- will transfer
entire file
>  Edit src/sys/dev/hwpmc/hwpmc_ppro.c,v
> src/sys/dev/hwpmc/hwpmc_ppro.c,v: Checksum mismatch -- will transfer
entire file
>  Edit src/sys/dev/hwpmc/hwpmc_x86.c,v
> src/sys/dev/hwpmc/hwpmc_x86.c,v: Checksum mismatch -- will transfer
entire file
>  Edit src/sys/i386/i386/local_apic.c,v
> src/sys/i386/i386/local_apic.c,v: Checksum mismatch -- will transfer
entire file
>  Edit src/sys/i386/include/apicvar.h,v
> src/sys/i386/include/apicvar.h,v: Checksum mismatch -- will transfer
entire file
>  Edit src/sys/i386/include/pmc_mdep.h,v
> src/sys/i386/include/pmc_mdep.h,v: Checksum mismatch -- will transfer
entire file
>  Edit src/sys/pci/agp_i810.c,v -> Attic
> src/sys/pci/agp_i810.c,v: Checksum mismatch -- will transfer entire
file

> ---------------------------------------
>
> Followed by...
>
> ---------------------------------------
> Applying fixups for collection cvs-all/cvs
>  Fixup src/release/Makefile,v
>  Fixup src/sys/amd64/amd64/local_apic.c,v
>  Fixup src/sys/amd64/include/apicvar.h,v
>  Fixup src/sys/amd64/include/pmc_mdep.h,v
>  Fixup src/sys/dev/hwpmc/hwpmc_piv.c,v
>  Fixup src/sys/dev/hwpmc/hwpmc_ppro.c,v
>  Fixup src/sys/dev/hwpmc/hwpmc_x86.c,v
>  Fixup src/sys/i386/i386/local_apic.c,v
>  Fixup src/sys/i386/include/apicvar.h,v
>  Fixup src/sys/i386/include/pmc_mdep.h,v
>  Fixup src/sys/pci/agp_i810.c,v -> Attic
> ---------------------------------------
>
> I am not seeing Fixups for anything other than the src tree.  I've
> checked back through the past month's logs and the story is the same.
> Is this a consequence of SVN export to CVS?  Can it be fixed?  Am I

just
> doing something wrong?
>
> --
> John Marshall



--
Never argue with an idiot. They drag you down to
their level, then beat you with their experience.
_______________________________________________
freebsd-hubs@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hubs
To unsubscribe, send any mail to "freebsd-hubs-unsubscribe@..."
_______________________________________________
freebsd-hubs@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hubs
To unsubscribe, send any mail to "freebsd-hubs-unsubscribe@..."