Another i386 distribution build failure. Missing flist entries?

[CivilCK]

=======  10 extra files in DESTDIR  =========
Files in DESTDIR but missing from flist.
File is obsolete or flist is out of date ?
------------------------------------------
./lib/libssp.so
./lib/libssp.so.0
./lib/libssp.so.0.0
./usr/lib/libssp.a
./usr/lib/libssp.so
./usr/lib/libssp.so.0
./usr/lib/libssp.so.0.0
./usr/lib/libssp_p.a
./usr/lib/libssp_pic.a
./usr/libdata/lint/llib-lssp.ln
=========  end of 10 extra files  ===========

Stop.
nbmake: stopped in /usr/src
ERROR: Failed to make distribution




I see above message while building current through CVS checkout. Just an hour ago.

Regards,

[Paul Goyette]

Did you do a build with UPDATE=YES ?

Remove those files from your DESTDIR and all will be well.

On Thu, 31 May 2007, C. K. wrote:

>
>
> =======  10 extra files in DESTDIR  =========
> Files in DESTDIR but missing from flist.
> File is obsolete or flist is out of date ?
> ------------------------------------------
> ./lib/libssp.so
> ./lib/libssp.so.0
> ./lib/libssp.so.0.0
> ./usr/lib/libssp.a
> ./usr/lib/libssp.so
> ./usr/lib/libssp.so.0
> ./usr/lib/libssp.so.0.0
> ./usr/lib/libssp_p.a
> ./usr/lib/libssp_pic.a
> ./usr/libdata/lint/llib-lssp.ln
> =========  end of 10 extra files  ===========
>
> Stop.
> nbmake: stopped in /usr/src
> ERROR: Failed to make distribution
>
>
>
>
> I see above message while building current through CVS checkout. Just an
> hour ago.
>
> Regards,
>
> --
> View this message in context: http://www.nabble.com/Another-i386-distribution-build-failure.-Missing-flist-entries--tf3849749.html#a10904871
> Sent from the current-users mailing list archive at Nabble.com.
>
>

----------------------------------------------------------------------
|   Paul Goyette   | PGP DSS Key fingerprint: |  E-mail addresses:   |
| Customer Service | FA29 0E3B 35AF E8AE 6651 |  paul@...   |
| Network Engineer | 0786 F758 55DE 53BA 7731 | pgoyette@... |
----------------------------------------------------------------------

[CivilCK]

Paul Goyette wrote:

Did you do a build with UPDATE=YES ?

Remove those files from your DESTDIR and all will be well.

Well, i did only type

$ cd /usr
$ cvs checkout -P src

as is said in page http://www.netbsd.org/Documentation/current


Does not "checkout" downloads fresh files (all files with different timestamps)?

[Patrick Welche]

On Fri, Jun 01, 2007 at 02:29:04AM -0700, C. K. wrote:
> Paul Goyette wrote:
> > Did you do a build with UPDATE=YES ?
> > Remove those files from your DESTDIR and all will be well.
...
> $ cd /usr
> $ cvs checkout -P src
...
> Does not "checkout" downloads fresh files (all files with different
> timestamps)?
 
That gives you fresh source, the problem seems to be with stale destination
directory.. (as per Paul's comment UPDATE=YES wouldn't nuke it for you)

Cheers,

Patrick

[Paul Goyette]

I didn't ask about how you retrieved the updated sources.

I asked if you ran build.sh with the UPDATE variable set.



On Fri, 1 Jun 2007, C. K. wrote:

>
>
>
> Paul Goyette wrote:
>>
>> Did you do a build with UPDATE=YES ?
>>
>> Remove those files from your DESTDIR and all will be well.
>>
>>
>
>
> Well, i did only type
>
> $ cd /usr
> $ cvs checkout -P src
>
> as is said in page http://www.netbsd.org/Documentation/current
>
>
> Does not "checkout" downloads fresh files (all files with different
> timestamps)?
>
>
>
> --
> View this message in context: http://www.nabble.com/Another-i386-distribution-build-failure.-Missing-flist-entries--tf3849749.html#a10909590
> Sent from the current-users mailing list archive at Nabble.com.
>
>

----------------------------------------------------------------------
|   Paul Goyette   | PGP DSS Key fingerprint: |  E-mail addresses:   |
| Customer Service | FA29 0E3B 35AF E8AE 6651 |  paul@...   |
| Network Engineer | 0786 F758 55DE 53BA 7731 | pgoyette@... |
----------------------------------------------------------------------

[Julio M. Merino Vidal-4]

On Thu, May 31, 2007 at 07:33:05PM -0700, Paul Goyette wrote:
> Did you do a build with UPDATE=YES ?
>
> Remove those files from your DESTDIR and all will be well.

That might fix the issue for him, but does not resolve a real
problem in the build.  All those files should be marked as
obsolete in the flists so that they are detected and removed
(even when updating a system).

--
Julio M. Merino Vidal <jmmv@...>

[Blair Sadewitz-2]

I believe they are marked as obsolete, aren't they?

[Paul Goyette]

That does not make the build process remove the files from your DESDIR!

Obsolete means that they get cleaned up during an install _from_ the
DESTDIR!

Different process!

On Fri, 1 Jun 2007, Blair Sadewitz wrote:

> I believe they are marked as obsolete, aren't they?
>

----------------------------------------------------------------------
|   Paul Goyette   | PGP DSS Key fingerprint: |  E-mail addresses:   |
| Customer Service | FA29 0E3B 35AF E8AE 6651 |  paul@...   |
| Network Engineer | 0786 F758 55DE 53BA 7731 | pgoyette@... |
----------------------------------------------------------------------

[Valeriy E. Ushakov-2]

On Fri, Jun 01, 2007 at 11:43:42 -0700, Paul Goyette wrote:

> On Fri, 1 Jun 2007, Blair Sadewitz wrote:
>
> > I believe they are marked as obsolete, aren't they?
>
> That does not make the build process remove the files from your DESDIR!

Actually, it does.  See postinstall-fix-obsolete target in the
top-level Makefile.  We do it as part of "distribution" target right
before doing checkflist.

SY, Uwe
--
uwe@...                       |       Zu Grunde kommen
http://snark.ptc.spbu.ru/~uwe/          |       Ist zu Grunde gehen

[Paul Goyette]

On Fri, 1 Jun 2007, Valeriy E. Ushakov wrote:

> On Fri, Jun 01, 2007 at 11:43:42 -0700, Paul Goyette wrote:
>
>> That does not make the build process remove the files from your DESDIR!
>
> Actually, it does.  See postinstall-fix-obsolete target in the
> top-level Makefile.  We do it as part of "distribution" target right
> before doing checkflist.

Yes, but not if you do UPDATE=YES ...  :)

At least, it never has done so for me.

[Julio M. Merino Vidal-4]

On Fri, Jun 01, 2007 at 12:31:39PM -0700, Paul Goyette wrote:

> On Fri, 1 Jun 2007, Valeriy E. Ushakov wrote:
>
> >On Fri, Jun 01, 2007 at 11:43:42 -0700, Paul Goyette wrote:
> >
> >>That does not make the build process remove the files from your DESDIR!
> >
> >Actually, it does.  See postinstall-fix-obsolete target in the
> >top-level Makefile.  We do it as part of "distribution" target right
> >before doing checkflist.
>
> Yes, but not if you do UPDATE=YES ...  :)
>
> At least, it never has done so for me.

I'm fairly sure it actually does.  I typically run builds with -u set, and
they usually succeed even with very old build directories (which surely
have obsolete files).  I say usually because the build very often breaks
because other mistakes, grgrgrgr.

--
Julio M. Merino Vidal <jmmv@...>

[Paul Goyette]

Hmmm, I've frequently had to remove these obsolete files manually (or
else run a much longer non-UPDATE build).  I guess I just got into the
habit.

At least it's an easy cure.  I don't think I've done a non-UPDATE build
for 3 or 4 months now....


On Fri, 1 Jun 2007, Julio M. Merino Vidal wrote:

>>> Actually, it does.  See postinstall-fix-obsolete target in the
>>> top-level Makefile.  We do it as part of "distribution" target right
>>> before doing checkflist.
>>
>> Yes, but not if you do UPDATE=YES ...  :)
>>
>> At least, it never has done so for me.
>
> I'm fairly sure it actually does.  I typically run builds with -u set, and
> they usually succeed even with very old build directories (which surely
> have obsolete files).  I say usually because the build very often breaks
> because other mistakes, grgrgrgr.

[Valeriy E. Ushakov-2]

On Fri, Jun 01, 2007 at 13:30:41 -0700, Paul Goyette wrote:

> On Fri, 1 Jun 2007, Julio M. Merino Vidal wrote:
>
> >>>Actually, it does.  See postinstall-fix-obsolete target in the
> >>>top-level Makefile.  We do it as part of "distribution" target right
> >>>before doing checkflist.
> >>
> >>Yes, but not if you do UPDATE=YES ...  :)
> >>
> >>At least, it never has done so for me.
> >
> >I'm fairly sure it actually does.  I typically run builds with -u set, and
> >they usually succeed even with very old build directories (which surely
> >have obsolete files).  I say usually because the build very often breaks
> >because other mistakes, grgrgrgr.
>
> Hmmm, I've frequently had to remove these obsolete files manually (or
> else run a much longer non-UPDATE build).  I guess I just got into the
> habit.

It certainly does run postinstall-fix-obsolete target, update build or
not.

People forgetting to mark files as obsolete is a different issue :)

SY, Uwe
--
uwe@...                       |       Zu Grunde kommen
http://snark.ptc.spbu.ru/~uwe/          |       Ist zu Grunde gehen

[Valeriy E. Ushakov-2]

[Moving to tech-toolchain.
 Summary: Update builds broken b/c of stale libssp* files in destdir]

On Sat, Jun 02, 2007 at 02:34:42 +0400, Valeriy E. Ushakov wrote:

> It certainly does run postinstall-fix-obsolete target, update build or
> not.
>
> People forgetting to mark files as obsolete is a different issue :)

I've brought back static and lint ssp libs and marked them as
obsolete.

Now, the so's are a different matter.  The problem with .so's is that
there might be locally compiled apps (i.e. not in the base system)
that are linked against the "obsolete" .so's.  If those "obsolete"
.so's are removed, the apps get broken.

postinstall fix obsolete can automatically clean up lib dirs from old
minor versions (so we don't need to have an obsolete entry for each
minor version in the set lists), but it doesn't touch old major
versions.  It has code to do so, but it's not enabled (there's even no
command line option to enable it).

For major version bumps we could 1) remove list entries instead of
marking them obsolete, and 2) run postinstall-fix-obsolete target with
AllLibs enabled.  We only run postinstall-fix-obsolete for builds that
are not in-place, so it should be safe, it will clean the destdir (b/c
of #2), and it will not touch obsolete major versions in an installed
system (b/c of #1).

That still doesn't address the issue of removing a library though
(there's no newer major version to trigger the deletion of the library
in the destdir by postinstall running with AllLibs enabled).


Thoughts?

SY, Uwe
--
uwe@...                       |       Zu Grunde kommen
http://snark.ptc.spbu.ru/~uwe/          |       Ist zu Grunde gehen