Bug#545366: dselect 1.15.4 does not distinguish between new and old packages

[Petr Vandrovec]

Package: dselect
Version: 1.15.4

Since today's update run dselect is completely confused and does not
remember which packages are new and which are old - it lists all
available and not installed packages as new, again and again.  And not
only that, it also marks some of them for installation although I do not
want them (and some are not installable at all - for some reason it
wants to install libsasl2 which is not installable together with
libsasl2-2).  I see that it updates available & status, they seem to
contain valid data, but on load it somehow ignores them.

Besides that it creates some strange /var/lib/dpkg/updates/tmp.i which
is full of lines saying '##padding', but other than that I do not see
what can be wrong, and unfortunately diff between 1.15.3.1 and 1.15.4 is
quite huge.

Thanks, Petr





--
To UNSUBSCRIBE, email to debian-dpkg-bugs-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Raphael Hertzog-3]

On Sun, 06 Sep 2009, Petr Vandrovec wrote:

> Since today's update run dselect is completely confused and does not
> remember which packages are new and which are old - it lists all
> available and not installed packages as new, again and again.  And
> not only that, it also marks some of them for installation although
> I do not want them (and some are not installable at all - for some
> reason it wants to install libsasl2 which is not installable
> together with libsasl2-2).  I see that it updates available &
> status, they seem to contain valid data, but on load it somehow
> ignores them.
>
> Besides that it creates some strange /var/lib/dpkg/updates/tmp.i
> which is full of lines saying '##padding', but other than that I do
> not see what can be wrong, and unfortunately diff between 1.15.3.1
> and 1.15.4 is quite huge.

You can use "git bisect" to isolate the problem more precisely. My bet
would be on the code that auto-cleans up the status database, maybe it
applies by error on the available file as well and thus it believes
that all packages are new everytime?

Cheers,
--
Raphaël Hertzog




--
To UNSUBSCRIBE, email to debian-dpkg-bugs-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Sven Joachim]

On 2009-09-07 09:11 +0200, Raphael Hertzog wrote:

> On Sun, 06 Sep 2009, Petr Vandrovec wrote:
>> Since today's update run dselect is completely confused and does not
>> remember which packages are new and which are old - it lists all
>> available and not installed packages as new, again and again.  And
>> not only that, it also marks some of them for installation although
>> I do not want them (and some are not installable at all - for some
>> reason it wants to install libsasl2 which is not installable
>> together with libsasl2-2).  I see that it updates available &
>> status, they seem to contain valid data, but on load it somehow
>> ignores them.
>>
>> Besides that it creates some strange /var/lib/dpkg/updates/tmp.i
>> which is full of lines saying '##padding', but other than that I do
>> not see what can be wrong, and unfortunately diff between 1.15.3.1
>> and 1.15.4 is quite huge.
>
> You can use "git bisect" to isolate the problem more precisely. My bet
> would be on the code that auto-cleans up the status database,

Certainly.  FWIW, the problem occurs also if only dpkg is upgraded to
1.15.4 and dselect stays at 1.15.3.1.

> maybe it applies by error on the available file as well and thus it
> believes that all packages are new everytime?

Given that the available package is as big as ever and dselect shows
every package as new even after you run "sync-available" (from the
dctrl-tools package), this does not seem to be the case.

Sven




--
To UNSUBSCRIBE, email to debian-dpkg-bugs-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Petr Vandrovec]

Raphael Hertzog napsal(a):
> On Sun, 06 Sep 2009, Petr Vandrovec wrote:
> You can use "git bisect" to isolate the problem more precisely. My bet
> would be on the code that auto-cleans up the status database, maybe it
> applies by error on the available file as well and thus it believes
> that all packages are new everytime?

224f0285abc304bec059e61447781... is first bad commit
commit 224f0285...
Author: Guillem Jover <guillem@...>
Date:   Sun Jul 12 20:11:53 2009 +0200

    Obsolete --forget-old-unavail

    On parse mark not-installed leftover packages for automatic removal from
    the database on next dump ...

    Closes: #333394, #429262


And while we are on it, checkin 7fa96f35... (disable default automake
paths) causes FTBFS on my box.
                                                        Petr






--
To UNSUBSCRIBE, email to debian-dpkg-bugs-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Guillem Jover]

Hi!

On Mon, 2009-09-07 at 09:43:37 +0200, Sven Joachim wrote:
> On 2009-09-07 09:11 +0200, Raphael Hertzog wrote:
> > On Sun, 06 Sep 2009, Petr Vandrovec wrote:
> > > Since today's update run dselect is completely confused and does not
> > > remember which packages are new and which are old - it lists all
> > > available and not installed packages as new, again and again. And

Hrmmmf, ok missed that one when doing the automatic forget change on
dpkg...

> > > not only that, it also marks some of them for installation although
> > > I do not want them (and some are not installable at all - for some
> > > reason it wants to install libsasl2 which is not installable
> > > together with libsasl2-2).  I see that it updates available &
> > > status, they seem to contain valid data, but on load it somehow
> > > ignores them.

The installability problems I assume is due to some transition going
on in unstable right now.

> > > Besides that it creates some strange /var/lib/dpkg/updates/tmp.i
> > > which is full of lines saying '##padding', but other than that I do
> > > not see what can be wrong, and unfortunately diff between 1.15.3.1
> > > and 1.15.4 is quite huge.

The “tmp.i” file is normal, yes.

> > You can use "git bisect" to isolate the problem more precisely. My bet
> > would be on the code that auto-cleans up the status database,
>
> Certainly.  FWIW, the problem occurs also if only dpkg is upgraded to
> 1.15.4 and dselect stays at 1.15.3.1.
>
> > maybe it applies by error on the available file as well and thus it
> > believes that all packages are new everytime?
>
> Given that the available package is as big as ever and dselect shows
> every package as new even after you run "sync-available" (from the
> dctrl-tools package), this does not seem to be the case.

The problem is that dselect used the status file to track not seen
and seen not-installed packages as either want_unknown or want_purge
want states. And was setting all want_unknown packages to want_purge
on normal exit (not using X keyibindig).

This increases the status file, the parsing and processing time when
doing dependency resolutions and package iterations for everyone, and
just to be able to show new packages on dselect. The correct solution
here is not to revert the change, but to store the seen/not-seen
information in another place, in a similar way as how apt/aptitude do
it. And ideally in a unified place which all front-ends can share, so
we avoid duplication.

I'll put it on the pending stuff to discuss with the front-end
developers.

regards,
guillem




--
To UNSUBSCRIBE, email to debian-dpkg-bugs-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Petr Vandrovec]

Guillem Jover napsal(a):
> Hi!
> This increases the status file, the parsing and processing time when
> doing dependency resolutions and package iterations for everyone, and
> just to be able to show new packages on dselect. The correct solution
> here is not to revert the change, but to store the seen/not-seen
> information in another place, in a similar way as how apt/aptitude do
> it. And ideally in a unified place which all front-ends can share, so
> we avoid duplication.

IMHO change should be reverted until old functionality can be achieved
through some other way - unless you can achieve such functionality today.
                                                Petr





--
To UNSUBSCRIBE, email to debian-dpkg-bugs-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Raphael Hertzog-3]

severity 545366 serious
thanks

On Tue, 08 Sep 2009, Petr Vandrovec wrote:
> IMHO change should be reverted until old functionality can be
> achieved through some other way - unless you can achieve such
> functionality today.

I'd like to remind everybody that we're looking for someone to step up and
maintain dselect:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=282283

Here's your chance instead of requesting an instant fix from people who
would rather not have to maintain dselect as part of dpkg.

I'll mark this bug as RC so that it doesn't get into testing, you're free
to downgrade dpkg in the mean time.

Cheers,
--
Raphaël Hertzog




--
To UNSUBSCRIBE, email to debian-dpkg-bugs-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Debian Bug Tracking System]

Processing commands for control@...:

> severity 545366 serious
Bug #545366 [dselect] dselect 1.15.4 does not distinguish between new and old packages
Severity set to 'serious' from 'normal'

> thanks
Stopping processing here.

Please contact me if you need assistance.

Debian bug tracking system administrator
(administrator, Debian Bugs database)


--
To UNSUBSCRIBE, email to debian-dpkg-bugs-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Debian Bug Tracking System]

Your message dated Mon, 19 Oct 2009 17:02:29 +0000
with message-id <E1Mzvcr-0004Zo-EV@...>
and subject line Bug#545366: fixed in dpkg 1.15.4.1
has caused the Debian Bug report #545366,
regarding dselect 1.15.4 does not distinguish between new and old packages
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner@...
immediately.)


--
545366: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=545366
Debian Bug Tracking System
Contact owner@... with problems

Package: dselect
Version: 1.15.4

Since today's update run dselect is completely confused and does not
remember which packages are new and which are old - it lists all
available and not installed packages as new, again and again.  And not
only that, it also marks some of them for installation although I do not
want them (and some are not installable at all - for some reason it
wants to install libsasl2 which is not installable together with
libsasl2-2).  I see that it updates available & status, they seem to
contain valid data, but on load it somehow ignores them.

Besides that it creates some strange /var/lib/dpkg/updates/tmp.i which
is full of lines saying '##padding', but other than that I do not see
what can be wrong, and unfortunately diff between 1.15.3.1 and 1.15.4 is
quite huge.

Thanks, Petr




Source: dpkg
Source-Version: 1.15.4.1

We believe that the bug you reported is fixed in the latest version of
dpkg, which is due to be installed in the Debian FTP archive:

dpkg-dev_1.15.4.1_all.deb
  to pool/main/d/dpkg/dpkg-dev_1.15.4.1_all.deb
dpkg_1.15.4.1.dsc
  to pool/main/d/dpkg/dpkg_1.15.4.1.dsc
dpkg_1.15.4.1.tar.gz
  to pool/main/d/dpkg/dpkg_1.15.4.1.tar.gz
dpkg_1.15.4.1_amd64.deb
  to pool/main/d/dpkg/dpkg_1.15.4.1_amd64.deb
dselect_1.15.4.1_amd64.deb
  to pool/main/d/dpkg/dselect_1.15.4.1_amd64.deb



A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 545366@...,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Guillem Jover <guillem@...> (supplier of updated dpkg package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@...)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.8
Date: Mon, 19 Oct 2009 15:15:17 +0200
Source: dpkg
Binary: dpkg dpkg-dev dselect
Architecture: source amd64 all
Version: 1.15.4.1
Distribution: unstable
Urgency: medium
Maintainer: Dpkg Developers <debian-dpkg@...>
Changed-By: Guillem Jover <guillem@...>
Description:
 dpkg       - Debian package management system
 dpkg-dev   - Debian package development tools
 dselect    - Debian package management front-end
Closes: 545366
Changes:
 dpkg (1.15.4.1) unstable; urgency=medium
 .
   * Do not mark any package as unseen in dselect. This is a workaround
     until it learns how to store such information again. Closes: #545366
Checksums-Sha1:
 529a095ca3c787e9bb4e2444f444691067f4e5eb 1219 dpkg_1.15.4.1.dsc
 bd1c8f8e1d2c9761f3b823cb82b5f0104cb6ac79 7037176 dpkg_1.15.4.1.tar.gz
 1dc5af28f7ec13d7e2c404d464a13190b8c22ce6 2149876 dpkg_1.15.4.1_amd64.deb
 200fcb03ac84330f9005ee85f2a84f6db9bf46be 705026 dselect_1.15.4.1_amd64.deb
 cddbb11dcb2446a62bbf095d57e7327beada393e 690544 dpkg-dev_1.15.4.1_all.deb
Checksums-Sha256:
 715fcf6fb77a52eadd0f00e41503b2423c0a2a253dab38f0cd2f4f9e53a2f0d2 1219 dpkg_1.15.4.1.dsc
 a3b0932748648d0699d28f3327a0d5eb8287f51e6cacecac90c418b197a7da5f 7037176 dpkg_1.15.4.1.tar.gz
 c1bed7400f7cb3591e877f35a4c667c5ac365fed9cf062d56e887d07101ceb2e 2149876 dpkg_1.15.4.1_amd64.deb
 e688c1a91e23d95ab5637c08ccd6a29c779d9260df4ecdf4ac27e7d58ac0dff0 705026 dselect_1.15.4.1_amd64.deb
 0a8cb00b9a23e6cef6e109457abf0d4e22008ddb2ad2b6a02a63061ec8134a83 690544 dpkg-dev_1.15.4.1_all.deb
Files:
 1b3cfefb8ce5d220603bd2aaaf083653 1219 admin required dpkg_1.15.4.1.dsc
 273a83fcd9d2d682585804cb48751e48 7037176 admin required dpkg_1.15.4.1.tar.gz
 0f87728f30161f72c518055e26f8fa1a 2149876 admin required dpkg_1.15.4.1_amd64.deb
 bb80e90b838d2f5689b1799925f9391a 705026 admin optional dselect_1.15.4.1_amd64.deb
 275e2affd0278438f7833ed93cb05efc 690544 utils optional dpkg-dev_1.15.4.1_all.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAkrcj9oACgkQuW9ciZ2SjJtxRACeIsaiOeqMVgtD5IM+qytiiqx+
PC8An1xFjW+NmBNw4nVS+QE6zoyKe3J8
=JxdA
-----END PGP SIGNATURE-----

[Debian Bug Tracking System]

Your message dated Sun, 25 Oct 2009 11:40:41 +0100
with message-id <20091025104041.GE17775@rivendell>
and subject line Re: Bug#545366: dselect 1.15.4 does not distinguish between new and old packages
has caused the Debian Bug report #545366,
regarding dselect 1.15.4 does not distinguish between new and old packages
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner@...
immediately.)


--
545366: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=545366
Debian Bug Tracking System
Contact owner@... with problems

Package: dselect
Version: 1.15.4

Since today's update run dselect is completely confused and does not
remember which packages are new and which are old - it lists all
available and not installed packages as new, again and again.  And not
only that, it also marks some of them for installation although I do not
want them (and some are not installable at all - for some reason it
wants to install libsasl2 which is not installable together with
libsasl2-2).  I see that it updates available & status, they seem to
contain valid data, but on load it somehow ignores them.

Besides that it creates some strange /var/lib/dpkg/updates/tmp.i which
is full of lines saying '##padding', but other than that I do not see
what can be wrong, and unfortunately diff between 1.15.3.1 and 1.15.4 is
quite huge.

Thanks, Petr




Version: 1.15.4.1

On Sun, 25 Oct 2009, Piotr Engelking wrote:
> The only difference between 1.15.4 and 1.15.4.1 is that former marks
> new and old packages as new, and the latter marks new and old packages
> as old.

That's what we wanted. Closing this bug again. The fact that it doesn't
keep track of new packages is tracked in #551638 and is not release
critical for us.

Please use the other bug for further discussion.

Cheers,
--
Raphaël Hertzog