gnupg package signing

[Xavier Chantry-2]

Just to let you know that I resurrected the gpg branch there :
http://code.toofishes.net/cgit/xavier/pacman.git/log/?h=gpg

I took Dan's newgpg branch (with a few changes) :
http://code.toofishes.net/cgit/dan/pacman.git/commit/?h=newgpg
then merged the pending patches we had :
http://archlinux.org/pipermail/pacman-dev/2008-December/007808.html
http://archlinux.org/pipermail/pacman-dev/2008-December/007836.html
http://archlinux.org/pipermail/pacman-dev/2008-December/007837.html
and rebased it all on master.

Actually I don't see what else needs to be done on the implementation
side, it looks almost complete to me.

Now the big remaining problem is everything related to key
administration still needs to be figured out, and this is critical in
term of security.
But it might not need additional tool support.

[Allan McRae-3]

Xavier wrote:

> Just to let you know that I resurrected the gpg branch there :
> http://code.toofishes.net/cgit/xavier/pacman.git/log/?h=gpg
>
> I took Dan's newgpg branch (with a few changes) :
> http://code.toofishes.net/cgit/dan/pacman.git/commit/?h=newgpg
> then merged the pending patches we had :
> http://archlinux.org/pipermail/pacman-dev/2008-December/007808.html
> http://archlinux.org/pipermail/pacman-dev/2008-December/007836.html
> http://archlinux.org/pipermail/pacman-dev/2008-December/007837.html
> and rebased it all on master.
>
> Actually I don't see what else needs to be done on the implementation
> side, it looks almost complete to me.
>
> Now the big remaining problem is everything related to key
> administration still needs to be figured out, and this is critical in
> term of security.
> But it might not need additional tool support.
>  

So...   how about we set up a small signed package repo somewhere and
just see how this all goes?  We are not going to know all the issues
until we actually use it.

Allan

[Xavier Chantry-2]

On Tue, Aug 25, 2009 at 12:19 AM, Allan McRae<allan@...> wrote:

> Xavier wrote:
>>
>> Just to let you know that I resurrected the gpg branch there :
>> http://code.toofishes.net/cgit/xavier/pacman.git/log/?h=gpg
>>
>> I took Dan's newgpg branch (with a few changes) :
>> http://code.toofishes.net/cgit/dan/pacman.git/commit/?h=newgpg
>> then merged the pending patches we had :
>> http://archlinux.org/pipermail/pacman-dev/2008-December/007808.html
>> http://archlinux.org/pipermail/pacman-dev/2008-December/007836.html
>> http://archlinux.org/pipermail/pacman-dev/2008-December/007837.html
>> and rebased it all on master.
>>
>> Actually I don't see what else needs to be done on the implementation
>> side, it looks almost complete to me.
>>
>> Now the big remaining problem is everything related to key
>> administration still needs to be figured out, and this is critical in
>> term of security.
>> But it might not need additional tool support.
>>
>
> So...   how about we set up a small signed package repo somewhere and just
> see how this all goes?  We are not going to know all the issues until we
> actually use it.
>

That's probably a good idea.
I wish some people who actually knew how to use gnupg a bit could help though :)

[Dan McGee]

On Mon, Aug 24, 2009 at 5:28 PM, Xavier<shiningxc@...> wrote:

> On Tue, Aug 25, 2009 at 12:19 AM, Allan McRae<allan@...> wrote:
>> Xavier wrote:
>>>
>>> Just to let you know that I resurrected the gpg branch there :
>>> http://code.toofishes.net/cgit/xavier/pacman.git/log/?h=gpg
>>>
>>> I took Dan's newgpg branch (with a few changes) :
>>> http://code.toofishes.net/cgit/dan/pacman.git/commit/?h=newgpg
>>> then merged the pending patches we had :
>>> http://archlinux.org/pipermail/pacman-dev/2008-December/007808.html
>>> http://archlinux.org/pipermail/pacman-dev/2008-December/007836.html
>>> http://archlinux.org/pipermail/pacman-dev/2008-December/007837.html
>>> and rebased it all on master.
>>>
>>> Actually I don't see what else needs to be done on the implementation
>>> side, it looks almost complete to me.
>>>
>>> Now the big remaining problem is everything related to key
>>> administration still needs to be figured out, and this is critical in
>>> term of security.
>>> But it might not need additional tool support.
>>>
>>
>> So...   how about we set up a small signed package repo somewhere and just
>> see how this all goes?  We are not going to know all the issues until we
>> actually use it.
>>
>
> That's probably a good idea.
> I wish some people who actually knew how to use gnupg a bit could help though :)

I did a whole lot of looking and working on this today while sitting
in the jury waiting room (and woo, I got picked to be on a jury, meh).
I've actually worked my way back through the original patches and am
about halfway through what Xavier has on his branch, and I've actually
added another 3 or 4 patches to the mix. I'll try to push the
"results" somewhere public tonight. I do feel the momentum on this
whole thing actually moving in the right direction, however, so that
is awesome.

Hopefully I will be able to continue the patch processing and tidying
and keep looking at this throughout the week.

-Dan

[Dan McGee]

On Mon, Aug 24, 2009 at 6:19 PM, Dan McGee<dpmcgee@...> wrote:

> On Mon, Aug 24, 2009 at 5:28 PM, Xavier<shiningxc@...> wrote:
>> On Tue, Aug 25, 2009 at 12:19 AM, Allan McRae<allan@...> wrote:
>>> Xavier wrote:
>>>>
>>>> Just to let you know that I resurrected the gpg branch there :
>>>> http://code.toofishes.net/cgit/xavier/pacman.git/log/?h=gpg
>>>>
>>>> I took Dan's newgpg branch (with a few changes) :
>>>> http://code.toofishes.net/cgit/dan/pacman.git/commit/?h=newgpg
>>>> then merged the pending patches we had :
>>>> http://archlinux.org/pipermail/pacman-dev/2008-December/007808.html
>>>> http://archlinux.org/pipermail/pacman-dev/2008-December/007836.html
>>>> http://archlinux.org/pipermail/pacman-dev/2008-December/007837.html
>>>> and rebased it all on master.
>>>>
>>>> Actually I don't see what else needs to be done on the implementation
>>>> side, it looks almost complete to me.
>>>>
>>>> Now the big remaining problem is everything related to key
>>>> administration still needs to be figured out, and this is critical in
>>>> term of security.
>>>> But it might not need additional tool support.
>>>>
>>>
>>> So...   how about we set up a small signed package repo somewhere and just
>>> see how this all goes?  We are not going to know all the issues until we
>>> actually use it.
>>>
>>
>> That's probably a good idea.
>> I wish some people who actually knew how to use gnupg a bit could help though :)
>
> I did a whole lot of looking and working on this today while sitting
> in the jury waiting room (and woo, I got picked to be on a jury, meh).
> I've actually worked my way back through the original patches and am
> about halfway through what Xavier has on his branch, and I've actually
> added another 3 or 4 patches to the mix. I'll try to push the
> "results" somewhere public tonight. I do feel the momentum on this
> whole thing actually moving in the right direction, however, so that
> is awesome.
>
> Hopefully I will be able to continue the patch processing and tidying
> and keep looking at this throughout the week.

Remember only half of the patches are there:
http://code.toofishes.net/cgit/dan/pacman.git/log/?h=gpg

[Dan McGee]

On Tue, Aug 25, 2009 at 6:24 AM, Dan McGee <dpmcgee@...> wrote:

> On Mon, Aug 24, 2009 at 6:19 PM, Dan McGee<dpmcgee@...> wrote:
>> On Mon, Aug 24, 2009 at 5:28 PM, Xavier<shiningxc@...> wrote:
>>> On Tue, Aug 25, 2009 at 12:19 AM, Allan McRae<allan@...> wrote:
>>>> Xavier wrote:
>>>>>
>>>>> Just to let you know that I resurrected the gpg branch there :
>>>>> http://code.toofishes.net/cgit/xavier/pacman.git/log/?h=gpg
>>>>>
>>>>> I took Dan's newgpg branch (with a few changes) :
>>>>> http://code.toofishes.net/cgit/dan/pacman.git/commit/?h=newgpg
>>>>> then merged the pending patches we had :
>>>>> http://archlinux.org/pipermail/pacman-dev/2008-December/007808.html
>>>>> http://archlinux.org/pipermail/pacman-dev/2008-December/007836.html
>>>>> http://archlinux.org/pipermail/pacman-dev/2008-December/007837.html
>>>>> and rebased it all on master.
>>>>>
>>>>> Actually I don't see what else needs to be done on the implementation
>>>>> side, it looks almost complete to me.
>>>>>
>>>>> Now the big remaining problem is everything related to key
>>>>> administration still needs to be figured out, and this is critical in
>>>>> term of security.
>>>>> But it might not need additional tool support.
>>>>>
>>>>
>>>> So...   how about we set up a small signed package repo somewhere and just
>>>> see how this all goes?  We are not going to know all the issues until we
>>>> actually use it.
>>>>
>>>
>>> That's probably a good idea.
>>> I wish some people who actually knew how to use gnupg a bit could help though :)
>>
>> I did a whole lot of looking and working on this today while sitting
>> in the jury waiting room (and woo, I got picked to be on a jury, meh).
>> I've actually worked my way back through the original patches and am
>> about halfway through what Xavier has on his branch, and I've actually
>> added another 3 or 4 patches to the mix. I'll try to push the
>> "results" somewhere public tonight. I do feel the momentum on this
>> whole thing actually moving in the right direction, however, so that
>> is awesome.
>>
>> Hopefully I will be able to continue the patch processing and tidying
>> and keep looking at this throughout the week.
>
> Remember only half of the patches are there:
> http://code.toofishes.net/cgit/dan/pacman.git/log/?h=gpg

Soooooo...I finally started looking at this again more tonight. I have
my GPG base rebased, and I see Xavier did the same today as well. My
goal for tonight was to get a better idea of where to head with the
libalpm/pacman side of things, as I am not near as happy with that as
the tooling side of things.

So I did some research, and this is our "competition":
http://bazaar.launchpad.net/~ubuntu-core-dev/apt/ubuntu/annotate/head%3A/methods/gpgv.cc
And the code that calls that executable:
http://bazaar.launchpad.net/~ubuntu-core-dev/apt/ubuntu/annotate/head%3A/apt-pkg/indexcopy.cc

Quick notes:
* They don't use gpgme or any other wrapper; they call gpgv directly
* There is quite a bit of code here, but not an overwhelming amount;
some might be reusable
* I don't believe they do signed packages, just signed repositories
* There is one trusted keyring involved

So some of the next steps:
* Get consensus on whether the script side of the signing stuff is in
a good enough state. This is basically the first 5 patches on my 'gpg'
branch. Does anyone want to raise any objections, suggestions, or have
comments?
* Figure out where we want to move with pacman/libalpm support. I am
feeling less inclined to use gpgme, but I don't really know what the
right answer is yet. I'm hoping things from the above code will help.
* Actually implement the signature checking code.
* Refine the signature checking code.
* Get a test repo set up with signed packages and databases, most
likely with something like pacman-git so we can all test it.

-Dan

[Allan McRae-3]

Dan McGee wrote:
> <snip>
> So some of the next steps:
> * Get consensus on whether the script side of the signing stuff is in
> a good enough state. This is basically the first 5 patches on my 'gpg'
> branch. Does anyone want to raise any objections, suggestions, or have
> comments?
>  

I had a good look at the makepkg/repo-add patches today and I think it
is "in a good enough state".  Despite having no idea what I am doing
with gpg, I took them for a quick spin and they appear to do what is
intended.

My only comment is minor.  In:

makepkg: allow signatures to work with split packages
http://code.toofishes.net/cgit/dan/pacman.git/commit/?h=gpg&id=12ccd781

this naming seems strange:

local pkg_file="$PKGDEST/${nameofpkg}-${pkgver}-${pkgrel}-${PKGARCH}${EXT}"
+ local
zip_file="$PKGDEST/${nameofpkg}-${pkgver}-${pkgrel}-${PKGARCH}${PKGEXT}"

zip_file is actually the package file and pkg_file is the uncompressed
package file.  So how about changing these to tar_file and pkg_file
respectively?

Allan

[Dan McGee]

On Sun, Nov 1, 2009 at 11:50 PM, Allan McRae <allan@...> wrote:

> Dan McGee wrote:
>>
>> <snip>
>> So some of the next steps:
>> * Get consensus on whether the script side of the signing stuff is in
>> a good enough state. This is basically the first 5 patches on my 'gpg'
>> branch. Does anyone want to raise any objections, suggestions, or have
>> comments?
>>
>
> I had a good look at the makepkg/repo-add patches today and I think it is
> "in a good enough state".  Despite having no idea what I am doing with gpg,
> I took them for a quick spin and they appear to do what is intended.
>
> My only comment is minor.  In:
>
> makepkg: allow signatures to work with split packages
> http://code.toofishes.net/cgit/dan/pacman.git/commit/?h=gpg&id=12ccd781
>
> this naming seems strange:
>
> local pkg_file="$PKGDEST/${nameofpkg}-${pkgver}-${pkgrel}-${PKGARCH}${EXT}"
> + local
> zip_file="$PKGDEST/${nameofpkg}-${pkgver}-${pkgrel}-${PKGARCH}${PKGEXT}"
>
> zip_file is actually the package file and pkg_file is the uncompressed
> package file.  So how about changing these to tar_file and pkg_file
> respectively?

I have no objection to that; I'll make the adjustment locally at some
point here.

-Dan