[PATCH] makepkg: rework --skip-integ

[Allan McRae-3]

The current --skip-integ isa bit weird.  It does not skip integrity
checks, but instead does them and prints a warning. Change this
behaviour to actually skipping the checks.

Signed-off-by: Allan McRae <allan@...>
---
 doc/makepkg.8.txt     |    3 +--
 scripts/makepkg.sh.in |   14 +++++++-------
 2 files changed, 8 insertions(+), 9 deletions(-)

diff --git a/doc/makepkg.8.txt b/doc/makepkg.8.txt
index 198aa95..ec02875 100644
--- a/doc/makepkg.8.txt
+++ b/doc/makepkg.8.txt
@@ -87,8 +87,7 @@ Options
  PKGBUILD for source validation using "`makepkg -g >> PKGBUILD`".
 
 *--skipinteg*::
- Do not fail when the PKGBUILD does not contain any integrity checks, just
- print a warning instead.
+ Do not perform any integrity checks, just print a warning instead.
 
 *-h, \--help*::
  Output syntax and command line options.
diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in
index d85f858..f1788b9 100644
--- a/scripts/makepkg.sh.in
+++ b/scripts/makepkg.sh.in
@@ -615,12 +615,8 @@ check_checksums() {
  done
 
  if [ $correlation -eq 0 ]; then
- if [ $SKIPINTEG -eq 1 ]; then
- warning "$(gettext "Integrity checks are missing.")"
- else
- error "$(gettext "Integrity checks are missing.")"
- exit 1 # TODO: error code
- fi
+ error "$(gettext "Integrity checks are missing.")"
+ exit 1 # TODO: error code
  fi
 }
 
@@ -1848,7 +1844,11 @@ elif [ "$REPKG" -eq 1 ]; then
  fi
 else
  download_sources
- check_checksums
+ if [ $SKIPINTEG -eq 0 ]; then
+ check_checksums
+ else
+ warning "$(gettext "Skipping integrity checks.")"
+ fi
  extract_sources
 fi
 
--
1.6.4.4

[Dan McGee]

On Sun, Oct 11, 2009 at 6:59 AM, Allan McRae <allan@...> wrote:
> The current --skip-integ isa bit weird.  It does not skip integrity
> checks, but instead does them and prints a warning. Change this
> behaviour to actually skipping the checks.

I (we?) did this on purpose; we didn't want to skip the following check:

    elif [ ${#integrity_sums[@]} -gt 0 ]; then
      error "$(gettext "Integrity checks (%s) differ in size from the
source array.")" "$integ"

-Dan

[Allan McRae-3]

Dan McGee wrote:

> On Sun, Oct 11, 2009 at 6:59 AM, Allan McRae <allan@...> wrote:
>  
>> The current --skip-integ isa bit weird.  It does not skip integrity
>> checks, but instead does them and prints a warning. Change this
>> behaviour to actually skipping the checks.
>>    
>
> I (we?) did this on purpose; we didn't want to skip the following check:
>
>     elif [ ${#integrity_sums[@]} -gt 0 ]; then
>       error "$(gettext "Integrity checks (%s) differ in size from the
> source array.")" "$integ"
>  

That seems strange to me.  If you are skipping integrity checks, then do
you really care if the array size is wrong?

Allan

[Loui Chang]

On Sun 11 Oct 2009 21:59 +1000, Allan McRae wrote:
> The current --skip-integ isa bit weird.  It does not skip integrity
> checks, but instead does them and prints a warning. Change this
> behaviour to actually skipping the checks.

Why should anyone skip integrity checks?
Do people want to create invalid source packages?

This is a horrible feature in my opinion.

[Ray Rashif]

2009/10/11 Loui Chang <louipc.ist@...>

> Why should anyone skip integrity checks?
> Do people want to create invalid source packages?
>
> This is a horrible feature in my opinion.
>

For --nobuild?

[Dan McGee]

On Sun, Oct 11, 2009 at 7:57 AM, Allan McRae <allan@...> wrote:

> Dan McGee wrote:
>>
>> On Sun, Oct 11, 2009 at 6:59 AM, Allan McRae <allan@...> wrote:
>>
>>>
>>> The current --skip-integ isa bit weird.  It does not skip integrity
>>> checks, but instead does them and prints a warning. Change this
>>> behaviour to actually skipping the checks.
>>>
>>
>> I (we?) did this on purpose; we didn't want to skip the following check:
>>
>>    elif [ ${#integrity_sums[@]} -gt 0 ]; then
>>      error "$(gettext "Integrity checks (%s) differ in size from the
>> source array.")" "$integ"
>>
>
> That seems strange to me.  If you are skipping integrity checks, then do you
> really care if the array size is wrong?

I thought this point got brought up here and no one objected (and I
agreed with Xavier, maybe offline somewhere):
http://bugs.archlinux.org/task/15830

Xavier: "I prefer Profjim's patch, except I would keep the error in
case of incomplete (whats the point of letting an incomplete checksums
array in a pkgbuild...)"

The only change I really think makes sense is only allow --skip-integ
if there are no checksum arrays at all; that way you can never produce
an invalid source package; I would assume we shouldn't allow source
package creation without integrity sums?

-Dan

[Loui Chang]

On Mon 12 Oct 2009 00:10 +0800, Ray Rashif wrote:
> 2009/10/11 Loui Chang <louipc.ist@...>
>
> > Why should anyone skip integrity checks?
> > Do people want to create invalid source packages?
> >
> > This is a horrible feature in my opinion.
>
> For --nobuild?

That isn't making a package. Why shouldn't the integrity of source files
be checked there either?

[Loui Chang]

On Sun 11 Oct 2009 11:14 -0500, Dan McGee wrote:

> On Sun, Oct 11, 2009 at 7:57 AM, Allan McRae <allan@...> wrote:
> > Dan McGee wrote:
> >>
> >> On Sun, Oct 11, 2009 at 6:59 AM, Allan McRae <allan@...> wrote:
> >>
> >>>
> >>> The current --skip-integ isa bit weird.  It does not skip integrity
> >>> checks, but instead does them and prints a warning. Change this
> >>> behaviour to actually skipping the checks.
> >>>
> >>
> >> I (we?) did this on purpose; we didn't want to skip the following check:
> >>
> >>    elif [ ${#integrity_sums[@]} -gt 0 ]; then
> >>      error "$(gettext "Integrity checks (%s) differ in size from the
> >> source array.")" "$integ"
> >>
> >
> > That seems strange to me.  If you are skipping integrity checks,
> > then do you really care if the array size is wrong?
>
> I thought this point got brought up here and no one objected (and I
> agreed with Xavier, maybe offline somewhere):
> http://bugs.archlinux.org/task/15830
>
> Xavier: "I prefer Profjim's patch, except I would keep the error in
> case of incomplete (whats the point of letting an incomplete checksums
> array in a pkgbuild...)"
>
> The only change I really think makes sense is only allow --skip-integ
> if there are no checksum arrays at all; that way you can never produce
> an invalid source package; I would assume we shouldn't allow source
> package creation without integrity sums?

Yeah that makes sense. If you didn't include checksums then makepkg
could assume that they aren't important. It shouldn't explicitly allow
creating invalid packages.

[Allan McRae-3]

Loui Chang wrote:

> On Sun 11 Oct 2009 11:14 -0500, Dan McGee wrote:
>  
>> On Sun, Oct 11, 2009 at 7:57 AM, Allan McRae <allan@...> wrote:
>>    
>>> Dan McGee wrote:
>>>      
>>>> On Sun, Oct 11, 2009 at 6:59 AM, Allan McRae <allan@...> wrote:
>>>>
>>>>        
>>>>> The current --skip-integ isa bit weird.  It does not skip integrity
>>>>> checks, but instead does them and prints a warning. Change this
>>>>> behaviour to actually skipping the checks.
>>>>>
>>>>>          
>>>> I (we?) did this on purpose; we didn't want to skip the following check:
>>>>
>>>>    elif [ ${#integrity_sums[@]} -gt 0 ]; then
>>>>      error "$(gettext "Integrity checks (%s) differ in size from the
>>>> source array.")" "$integ"
>>>>
>>>>        
>>> That seems strange to me.  If you are skipping integrity checks,
>>> then do you really care if the array size is wrong?
>>>      
>> I thought this point got brought up here and no one objected (and I
>> agreed with Xavier, maybe offline somewhere):
>> http://bugs.archlinux.org/task/15830
>>
>> Xavier: "I prefer Profjim's patch, except I would keep the error in
>> case of incomplete (whats the point of letting an incomplete checksums
>> array in a pkgbuild...)"
>>
>> The only change I really think makes sense is only allow --skip-integ
>> if there are no checksum arrays at all; that way you can never produce
>> an invalid source package; I would assume we shouldn't allow source
>> package creation without integrity sums?
>>    
>
> Yeah that makes sense. If you didn't include checksums then makepkg
> could assume that they aren't important. It shouldn't explicitly allow
> creating invalid packages.
>  

It seems a very strange name to me then...  --skip-integ means do
integrity checks but ignore the results when the actual integrity checks
fails but not in the case where the array sizes are different?  How is
the array size being different any worse than a wrong checksum?

Also, when did we start assuming people were stupid.  If I use the
--skip-integ option, then I know there are issues with my md5sums or I
do not want to download the sources to check them.  Either way, I have
gone out of my way not to check them so I really do not want them checked.

Allan

[Dan McGee]

On Sun, Oct 11, 2009 at 5:35 PM, Allan McRae <allan@...> wrote:

> Loui Chang wrote:
>>
>> On Sun 11 Oct 2009 11:14 -0500, Dan McGee wrote:
>>
>>>
>>> On Sun, Oct 11, 2009 at 7:57 AM, Allan McRae <allan@...> wrote:
>>>
>>>>
>>>> Dan McGee wrote:
>>>>
>>>>>
>>>>> On Sun, Oct 11, 2009 at 6:59 AM, Allan McRae <allan@...>
>>>>> wrote:
>>>>>
>>>>>
>>>>>>
>>>>>> The current --skip-integ isa bit weird.  It does not skip integrity
>>>>>> checks, but instead does them and prints a warning. Change this
>>>>>> behaviour to actually skipping the checks.
>>>>>>
>>>>>>
>>>>>
>>>>> I (we?) did this on purpose; we didn't want to skip the following
>>>>> check:
>>>>>
>>>>>   elif [ ${#integrity_sums[@]} -gt 0 ]; then
>>>>>     error "$(gettext "Integrity checks (%s) differ in size from the
>>>>> source array.")" "$integ"
>>>>>
>>>>>
>>>>
>>>> That seems strange to me.  If you are skipping integrity checks,
>>>> then do you really care if the array size is wrong?
>>>>
>>>
>>> I thought this point got brought up here and no one objected (and I
>>> agreed with Xavier, maybe offline somewhere):
>>> http://bugs.archlinux.org/task/15830
>>>
>>> Xavier: "I prefer Profjim's patch, except I would keep the error in
>>> case of incomplete (whats the point of letting an incomplete checksums
>>> array in a pkgbuild...)"
>>>
>>> The only change I really think makes sense is only allow --skip-integ
>>> if there are no checksum arrays at all; that way you can never produce
>>> an invalid source package; I would assume we shouldn't allow source
>>> package creation without integrity sums?
>>>
>>
>> Yeah that makes sense. If you didn't include checksums then makepkg
>> could assume that they aren't important. It shouldn't explicitly allow
>> creating invalid packages.
>>
>
> It seems a very strange name to me then...  --skip-integ means do integrity
> checks but ignore the results when the actual integrity checks fails but not
> in the case where the array sizes are different?  How is the array size
> being different any worse than a wrong checksum?
>
> Also, when did we start assuming people were stupid.  If I use the
> --skip-integ option, then I know there are issues with my md5sums or I do
> not want to download the sources to check them.  Either way, I have gone out
> of my way not to check them so I really do not want them checked.

I only assume people are stupid when they combine --skip-integ with
something that makes no sense. One of these (and I could even concede
this one) is when the *sum array is there but completely the wrong
size. I think my point here would make more sense if --skip-integ
really only worked for times when there were no checksums at all
present.

However the second reason is more serious. If you do --source
--skip-integ, that is a terrible decision. This is something that can
be given to someone else and they have *no idea* this is a completely
broken source package. To me, this is no better than a package with
missing dependencies, etc.

-Dan

[Xavier Chantry-2]

On Mon, Oct 12, 2009 at 12:35 AM, Allan McRae <allan@...> wrote:
>
> It seems a very strange name to me then...  --skip-integ means do integrity
> checks but ignore the results when the actual integrity checks fails but not
> in the case where the array sizes are different?  How is the array size
> being different any worse than a wrong checksum?
>

It was not considered worse. skipinteg did not work with wrong
checksum either. It only works in the case where no checksums are
defined in a pkgbuild.

> Also, when did we start assuming people were stupid.  If I use the
> --skip-integ option, then I know there are issues with my md5sums or I do
> not want to download the sources to check them.  Either way, I have gone out
> of my way not to check them so I really do not want them checked.

And forget what I said in the bug comments, I don't really care. It
might be ok to have skipinteg really skips everything after all.

So my opinion now is :
makepkg: rework --skip-integ -> could be ok
makepkg: allow skipping intergrity checks whem making source package
(Fixes FS#15984) -> less sure

It seems Loui is opposed to the second patch.
And Dan seems to be opposed to both, but especially the second one.

Correct me if I am wrong :)

[Allan McRae-3]

Dan McGee wrote:
> I only assume people are stupid when they combine --skip-integ with
> something that makes no sense. One of these (and I could even concede
> this one) is when the *sum array is there but completely the wrong
> size. I think my point here would make more sense if --skip-integ
> really only worked for times when there were no checksums at all
> present.
>  

I would be happy to make skip-integ only work if there is no md5sum
array.  But then again, the --ignorearch check does not work that way...


> However the second reason is more serious. If you do --source
> --skip-integ, that is a terrible decision. This is something that can
> be given to someone else and they have *no idea* this is a completely
> broken source package. To me, this is no better than a package with
> missing dependencies, etc.
>  

And we do not enforce dependency checking before running makepkg --source.


I'm not saying this is good practice to use the option, but then neither
is "pacman -Syfud" and we allow that.  My point is, if we have an option
called skip-integ, it better skip the integrity checks or be renamed.

Allan

[Allan McRae-3]

Let reanimate this discussion.

For those who need a reminder, this is about two patches on my working
branch:
[1] makepkg: allow skipping intergrity checks when making source package
[2] makepkg: rework --skip-integ

Currently the --skipinteg option results in makepkg outputting a warning
when there are _no_ integrity checks in a PKGBUILD that has a source
array.  In the case where integrity checks are present, it still checks
them and results in an error if they are incorrect.

Patch [2] makes the --skipinteg option to cause makepkg to actually
"skip integ"rity checks and so no error relating to integrity checks
will be raised.  I believe this to be in line with the naming of the
option and similar in spirit to options -a (to skip checking the arch
field) and -d (skips checking deps).

Patch [1] extends the --skipinteg option allow the generation of a
source tarball without requiring the checking of the integrity checks
for the non-local sources (and hence requiring a download of the
sources).  While this does allow someone to distribute a PKGBUILD with
incorrect checksums, the users would have to manually specify the
--skipinteg option (which has no short version), so that is a very
conscious decision by the user.  Similarly, a user could distribute a
PKGBUILD with wrong dependencies and we do not enforce the checking of
those.

Do I sound convincing?  :P

Allan

[1]
http://projects.archlinux.org/users/allan/pacman.git/commit/?h=working&id=636a9754
[2]
http://projects.archlinux.org/users/allan/pacman.git/commit/?h=working&id=3d09cee3

[Jeff-304]

On Tue, Oct 27, 2009 at 05:33:41PM +1000, Allan McRae wrote:
>
> Patch [2] makes the --skipinteg option to cause makepkg to actually
> "skip integ"rity checks and so no error relating to integrity checks

That sounds exactly like what I would expect skipinteg to mean.

> Patch [1] extends the --skipinteg option allow the generation of a
> source tarball without requiring the checking of the integrity checks

You've given the what, but what is the why? If the source integrity is
flawed, then the generated source package is flawed. This seems like
something that should be safeguarded against, IMO.

> Similarly, a user could distribute a PKGBUILD with wrong dependencies
> and we do not enforce the checking of those.

That is programatically much more difficult to enforce, and even if not,
ISTM that a near-equivalent analogy would be saying that since you can't
defend your house from nuclear strike, you might as well not defend it
from break-ins.

--
Jeff

My other computer is an abacus.

[Allan McRae-3]

Jeff wrote:
>> Patch [1] extends the --skipinteg option allow the generation of a
>> source tarball without requiring the checking of the integrity checks
>>    
>
> You've given the what, but what is the why? If the source integrity is
> flawed, then the generated source package is flawed. This seems like
> something that should be safeguarded against, IMO.
>  

I can come up with two use cases:

1) making a PKGBUILD for a snapshot release that is always accessible
from some sort of LATEST release directory symlink.  Many projects use
something like that.  That way the PKGBUILD does not need updated every
time a snapshot is release.  While it may be argued that it is better to
use a svn/cvs/git/etc PKGBUILD, in many cases the snapshots are
generally sanity checked before release.

2) This happens to me occasionally.  Someone sends me a PKGBUILD they
can not get working.  I see an obvious error, fix it and send the
PKGBUILD back saying "try this" because I really do not want to download
the sources/dependencies to check myself.

Allan

[Xavier Chantry-2]

On Thu, Oct 29, 2009 at 5:40 AM, Allan McRae <allan@...> wrote:

> Jeff wrote:
>>>
>>> Patch [1] extends the --skipinteg option allow the generation of a source
>>> tarball without requiring the checking of the integrity checks
>>
>> You've given the what, but what is the why? If the source integrity is
>> flawed, then the generated source package is flawed. This seems like
>> something that should be safeguarded against, IMO.
>>
>
> I can come up with two use cases:
>
> 1) making a PKGBUILD for a snapshot release that is always accessible from
> some sort of LATEST release directory symlink.  Many projects use something
> like that.  That way the PKGBUILD does not need updated every time a
> snapshot is release.  While it may be argued that it is better to use a
> svn/cvs/git/etc PKGBUILD, in many cases the snapshots are generally sanity
> checked before release.
>
> 2) This happens to me occasionally.  Someone sends me a PKGBUILD they can
> not get working.  I see an obvious error, fix it and send the PKGBUILD back
> saying "try this" because I really do not want to download the
> sources/dependencies to check myself.
>

Sounds reasonable to me.

[Xavier Chantry-2]

On Wed, Oct 28, 2009 at 10:40 PM, Jeff <jeff@...> wrote:
>
> That is programatically much more difficult to enforce, and even if not,
> ISTM that a near-equivalent analogy would be saying that since you can't
> defend your house from nuclear strike, you might as well not defend it
> from break-ins.
>

I think a more correct analogy is trying to defend one house with no
walls and one locked door from break-ins.

Why do you bother locking the door ? :)

[Isaac Good-4]

For what it's worth, I'm a fan of being able to skip the integrity check.
 - Isaac

[Loui Chang]

On Thu 29 Oct 2009 14:40 +1000, Allan McRae wrote:

> Jeff wrote:
> >>Patch [1] extends the --skipinteg option allow the generation of
> >>a source tarball without requiring the checking of the integrity
> >>checks
> >
> >You've given the what, but what is the why? If the source integrity is
> >flawed, then the generated source package is flawed. This seems like
> >something that should be safeguarded against, IMO.
>
> I can come up with two use cases:
>
> 1) making a PKGBUILD for a snapshot release that is always accessible
> from some sort of LATEST release directory symlink.  Many projects
> use something like that.  That way the PKGBUILD does not need updated
> every time a snapshot is release.  While it may be argued that it is
> better to use a svn/cvs/git/etc PKGBUILD, in many cases the snapshots
> are generally sanity checked before release.

> 2) This happens to me occasionally.  Someone sends me a PKGBUILD they
> can not get working.  I see an obvious error, fix it and send the
> PKGBUILD back saying "try this" because I really do not want to
> download the sources/dependencies to check myself.

In both cases if you could omit checksums and makepkg could interpret
that as "the packager doesn't really care about integrity, skip checks".

It could print a warning, and you don't need another fancy flag.

[Jeff-304]

On Thu, Oct 29, 2009 at 02:40:53PM +1000, Allan McRae wrote:
>
> While it may be argued that it is better to use a svn/cvs/git/etc
> PKGBUILD, in many cases the snapshots are generally sanity checked
> before release.

Preface: I can't find the words to say to prevent the following from
sounding argumentative. Please be assured it is mere curiosity of
reasoning:

If a sanity check fails, won't the fix be committed to the above
mentioned svn/cvs/git/etc repo and not just exist in the snapshot? This
would seem to put the snapshot argument into a very tiny corner case.

As for a thumbs up/down on the proposal of extending skipinteg to source
packages, you have shown value for such an action, but I wonder if it
should be either a separate option or one that requires an argument such
as --skipinteg=source with the main difference being a big warning
echoed at the end of the process saying something to the effect that the
packaged source tarball is unsafe as it cannot be authenticated easily.
Obviously you cannot restrain laziness, but perhaps this could help with
unintentional mistakes?

--
Jeff

My other computer is an abacus.