Use a .gem as an archive format
|
View:
New views
6 Messages
—
Rating Filter:
Alert me
|
 |
Use a .gem as an archive format
by Tim Cuthbertson-2
::
Rate this Message:
I am interested in replacing some rubygems dependencies with
zeroinstall, and the task would be much easier if .gem were a supported archive format. I'm nearly there, but need some help. As far as I can tell (by looking at a few, I can't find any documentation on the matter), it looks like .gem is an uncompressed tar containing two gzipped tars. data.tar.gz and metadata.gz. I'm going to assume that they are always in this order. Therefore, I can get a perfectly good gzip file by setting the offset to 512 (the tar header size), and letting gzip ignore the trailing garbage. Zero install, however, is less keen on this idea. When trying to extract it, I get: tar: Child returned status 2 tar: Error is not recoverable: exiting now Which I'm pretty sure is due to gzip, since I also tried the following: $ cat rspec-core-2.4.0.gem | tail -c +513 | gunzip >/dev/null gzip: stdin: decompression OK, trailing garbage ignored $ echo $? 2 Is there any way to get zero install (or tar, or gzip) to ignore the trailing garbage (a.k.a the rest of the top-level .tar file) and just proceed? The gzip man page lists exit code 2 as a warning, so perhaps that could simply be ignored? I'm sure things would blow up later (at the very least in the manifest check) if something were truly amiss... Also, I had a bit of an awkward time getting the manifest digest. Is there any way to get 0store to spit out manifests directly when given an archive? Or do I have to extract it somewhere first, and do the sha1sum/sha256sum myself? Regard, - Tim. ------------------------------------------------------------------------------ Protect Your Site and Customers from Malware Attacks Learn about various malware tactics and how to avoid them. Understand malware threats, the impact they can have on your business, and how you can protect your company and customers by using code signing. http://p.sf.net/sfu/oracle-sfdevnl _______________________________________________ Zero-install-devel mailing list Zero-install-devel@... https://lists.sourceforge.net/lists/listinfo/zero-install-devel |
|
|
Re: Use a .gem as an archive format
by Thomas Leonard-3
::
Rate this Message:
On 19 January 2011 11:50, Tim Cuthbertson <tim@...> wrote:
> I am interested in replacing some rubygems dependencies with > zeroinstall, and the task would be much easier if .gem were a > supported archive format. I'm nearly there, but need some help. > > As far as I can tell (by looking at a few, I can't find any > documentation on the matter), it looks like .gem is an uncompressed > tar containing two gzipped tars. data.tar.gz and metadata.gz. I'm > going to assume that they are always in this order. > > Therefore, I can get a perfectly good gzip file by setting the offset > to 512 (the tar header size), and letting gzip ignore the trailing > garbage. > > Zero install, however, is less keen on this idea. When trying to > extract it, I get: > tar: Child returned status 2 > tar: Error is not recoverable: exiting now > > Which I'm pretty sure is due to gzip, since I also tried the following: > $ cat rspec-core-2.4.0.gem | tail -c +513 | gunzip >/dev/null > gzip: stdin: decompression OK, trailing garbage ignored > $ echo $? > 2 > > Is there any way to get zero install (or tar, or gzip) to ignore the > trailing garbage (a.k.a the rest of the top-level .tar file) and just > proceed? The gzip man page lists exit code 2 as a warning, so perhaps > that could simply be ignored? I'm sure things would blow up later (at > the very least in the manifest check) if something were truly amiss... I don't think so. It would probably be easier to add gem as a separate MIME type with its own extraction code, or to provide support for nested archives. e.g. <recipe> <archive href='.../foo.gem' extract='data.tar.gz'/> <unpack path='data.tar.gz'/> </recipe> > Also, I had a bit of an awkward time getting the manifest digest. Is > there any way to get 0store to spit out manifests directly when given > an archive? Or do I have to extract it somewhere first, and do the > sha1sum/sha256sum myself? Looks like you can't pass an archive to "0store manifest" at the moment, so you'd have to unpack it first. There really should be a "0store digest" command to give you just the digest, not the whole manifest and the final digest. -- Dr Thomas Leonard http://0install.net/ GPG: 9242 9807 C985 3C07 44A6 8B9A AE07 8280 59A5 3CC1 GPG: DA98 25AE CAD0 8975 7CDA BD8E 0713 3F96 CA74 D8BA ------------------------------------------------------------------------------ Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d _______________________________________________ Zero-install-devel mailing list Zero-install-devel@... https://lists.sourceforge.net/lists/listinfo/zero-install-devel |
|
|
Re: Use a .gem as an archive format
by Tim Cuthbertson-2
::
Rate this Message:
On Sun, Jan 23, 2011 at 10:16 PM, Thomas Leonard <talex5@...> wrote:
> On 19 January 2011 11:50, Tim Cuthbertson <tim@...> wrote: >> I am interested in replacing some rubygems dependencies with >> zeroinstall, and the task would be much easier if .gem were a >> supported archive format. I'm nearly there, but need some help. >> >> As far as I can tell (by looking at a few, I can't find any >> documentation on the matter), it looks like .gem is an uncompressed >> tar containing two gzipped tars. data.tar.gz and metadata.gz. I'm >> going to assume that they are always in this order. >> >> Therefore, I can get a perfectly good gzip file by setting the offset >> to 512 (the tar header size), and letting gzip ignore the trailing >> garbage. >> >> Zero install, however, is less keen on this idea. When trying to >> extract it, I get: >> tar: Child returned status 2 >> tar: Error is not recoverable: exiting now >> >> Which I'm pretty sure is due to gzip, since I also tried the following: >> $ cat rspec-core-2.4.0.gem | tail -c +513 | gunzip >/dev/null >> gzip: stdin: decompression OK, trailing garbage ignored >> $ echo $? >> 2 >> >> Is there any way to get zero install (or tar, or gzip) to ignore the >> trailing garbage (a.k.a the rest of the top-level .tar file) and just >> proceed? The gzip man page lists exit code 2 as a warning, so perhaps >> that could simply be ignored? I'm sure things would blow up later (at >> the very least in the manifest check) if something were truly amiss... > > I don't think so. It would probably be easier to add gem as a separate > MIME type with its own extraction code, or to provide support for > nested archives. e.g. > > <recipe> > <archive href='.../foo.gem' extract='data.tar.gz'/> > <unpack path='data.tar.gz'/> > </recipe> Fair point. I don't think nested archives are too common, so I'll have a go at adding a mime type. The problem is, I don't think there's a good mime type for gems. I've asked on the rubygems mailing list, but rubygems,org just serves them up as application/octet-stream. I guess that just means your URLs always have to end in .gem? Cheers, - Tim. ------------------------------------------------------------------------------ Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d _______________________________________________ Zero-install-devel mailing list Zero-install-devel@... https://lists.sourceforge.net/lists/listinfo/zero-install-devel |
|
|
Re: Use a .gem as an archive format
by Thomas Leonard-3
::
Rate this Message:
On 24 January 2011 10:50, Tim Cuthbertson <tim@...> wrote:
> On Sun, Jan 23, 2011 at 10:16 PM, Thomas Leonard <talex5@...> wrote: >> On 19 January 2011 11:50, Tim Cuthbertson <tim@...> wrote: >>> I am interested in replacing some rubygems dependencies with >>> zeroinstall, and the task would be much easier if .gem were a >>> supported archive format. I'm nearly there, but need some help. [...] >> I don't think so. It would probably be easier to add gem as a separate >> MIME type with its own extraction code, or to provide support for >> nested archives. e.g. >> >> <recipe> >> <archive href='.../foo.gem' extract='data.tar.gz'/> >> <unpack path='data.tar.gz'/> >> </recipe> > > Fair point. I don't think nested archives are too common, so I'll have > a go at adding a mime type. The problem is, I don't think there's a > good mime type for gems. I've asked on the rubygems mailing list, but > rubygems,org just serves them up as application/octet-stream. I guess > that just means your URLs always have to end in .gem? You'll still need to make one up, because it uses MIME types internally (in unpack.py). -- Dr Thomas Leonard http://0install.net/ GPG: 9242 9807 C985 3C07 44A6 8B9A AE07 8280 59A5 3CC1 GPG: DA98 25AE CAD0 8975 7CDA BD8E 0713 3F96 CA74 D8BA ------------------------------------------------------------------------------ Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d _______________________________________________ Zero-install-devel mailing list Zero-install-devel@... https://lists.sourceforge.net/lists/listinfo/zero-install-devel |
|
|
Re: Use a .gem as an archive format
by Tim Cuthbertson-2
::
Rate this Message:
On Wed, Jan 26, 2011 at 8:18 AM, Thomas Leonard <talex5@...> wrote:
> On 24 January 2011 10:50, Tim Cuthbertson <tim@...> wrote: >> On Sun, Jan 23, 2011 at 10:16 PM, Thomas Leonard <talex5@...> wrote: >>> On 19 January 2011 11:50, Tim Cuthbertson <tim@...> wrote: >>>> I am interested in replacing some rubygems dependencies with >>>> zeroinstall, and the task would be much easier if .gem were a >>>> supported archive format. I'm nearly there, but need some help. > [...] >>> I don't think so. It would probably be easier to add gem as a separate >>> MIME type with its own extraction code, or to provide support for >>> nested archives. e.g. >>> >>> <recipe> >>> <archive href='.../foo.gem' extract='data.tar.gz'/> >>> <unpack path='data.tar.gz'/> >>> </recipe> >> >> Fair point. I don't think nested archives are too common, so I'll have >> a go at adding a mime type. The problem is, I don't think there's a >> good mime type for gems. I've asked on the rubygems mailing list, but >> rubygems,org just serves them up as application/octet-stream. I guess >> that just means your URLs always have to end in .gem? > > You'll still need to make one up, because it uses MIME types > internally (in unpack.py). Okay, I've pushed this to a branch at github: https://github.com/gfxmonk/zero-install/commit/rubygems (is doing that better or worse than sending a patch? I prefer it more as it gives you a quick diff as well as a clone URL, but the branches probably won't stick around after they get merged into master, so it could lead to broken links in the archives) For now I'm using application/x-ruby-gem Which was the suggestion from someone on the rubygems mailing list; I've sent an email to the maintainer of rubygems.org to see what he thinks. But either way, it won't be too hard to change later if anyone suggests a better mime-type. Cheers, - Tim. ------------------------------------------------------------------------------ Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d _______________________________________________ Zero-install-devel mailing list Zero-install-devel@... https://lists.sourceforge.net/lists/listinfo/zero-install-devel |
|
|
Re: Use a .gem as an archive format
by Thomas Leonard-3
::
Rate this Message:
On 25 January 2011 22:51, Tim Cuthbertson <tim@...> wrote:
[...] > Okay, I've pushed this to a branch at github: > https://github.com/gfxmonk/zero-install/commit/rubygems Applied - thanks! > (is doing that better or worse than sending a patch? I prefer it more > as it gives you a quick diff as well as a clone URL, but the branches > probably won't stick around after they get merged into master, so it > could lead to broken links in the archives) I don't mind. Pulling is slightly easier for me because I can see where you branched from in case there's any problem applying it. Having the patch in the list archives might be nice, but probably if the branch was deleted by the repository owner it was because it got merged. -- Dr Thomas Leonard http://0install.net/ GPG: 9242 9807 C985 3C07 44A6 8B9A AE07 8280 59A5 3CC1 GPG: DA98 25AE CAD0 8975 7CDA BD8E 0713 3F96 CA74 D8BA ------------------------------------------------------------------------------ Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d _______________________________________________ Zero-install-devel mailing list Zero-install-devel@... https://lists.sourceforge.net/lists/listinfo/zero-install-devel |
| Free embeddable forum powered by Nabble | Forum Help |
