Bug#402499: initramfs-tools: depmod wastes space and time, please replace it with a modules.dep file

[Russell Coker]

Package: initramfs-tools
Version: 0.85b
Severity: normal

When compressed a full modules.dep file is larger than a compressed
depmod binary:
# gzip -9 < /lib/modules/2.6.17-2-686/modules.dep |wc -c
15962
# ls -l sbin/depmod
-rwxr-xr-x 1 root root 37736 2006-12-11 07:38 sbin/depmod
# gzip -9 < sbin/depmod | wc -c
16936

When using a modules.dep file with the modules=list option the
uncompressed file is 2601 bytes.

It seems that using depmod instead of a modules.dep file is not going to
save initramfs size for anyone, it just adds another program to be run
(more compute time and another thing which can potentially break) and
does no good.


-- Package-specific info:
-- /proc/cmdline
console=ttyS0,115200 console=tty0 audit=1 selinux=1 ro root=/dev/md0 enforcing=0

-- /proc/filesystems
        cramfs
        ext3

-- lsmod
Module                  Size  Used by
nfsd                  208132  13
exportfs                5440  1 nfsd
lockd                  55752  2 nfsd
nfs_acl                 3392  1 nfsd
sunrpc                134332  9 nfsd,lockd,nfs_acl
ipt_REJECT              5120  91
ipt_LOG                 5760  2
ipv6                  222976  40
quota_v2                8448  6
dm_mirror              18832  0
softdog                 5716  2
iptable_filter          2816  1
ip_tables              17536  3 ipt_REJECT,ipt_LOG,iptable_filter
via_agp                 9344  1
agpgart                31496  1 via_agp
psmouse                32516  0
serio_raw               6596  0
via686a                15368  0
rtc                    11508  0
uhci_hcd               28304  0
shpchp                 39872  0
i2c_isa                 4672  1 via686a
usbcore               113924  2 uhci_hcd
pci_hotplug            24884  1 shpchp
i2c_core               19408  2 via686a,i2c_isa
parport_pc             32324  0
floppy                 54916  0
tulip                  45088  0
parport                31880  1 parport_pc
pcspkr                  1732  0
joydev                  8960  0
evdev                   8896  0
mousedev               10592  1
thermal                13512  0
processor              22976  1 thermal
fan                     4676  0
dm_snapshot            16348  0
dm_mod                 51512  6 dm_mirror,dm_snapshot
raid1                  18176  3
md_mod                 61652  4 raid1
ext3                  118664  5
jbd                    48724  1 ext3
mbcache                 8516  1 ext3
ide_disk               15936  10
ide_generic             1216  0 [permanent]
via82cxxx               8260  0 [permanent]
ide_core              112928  3 ide_disk,ide_generic,via82cxxx

-- kernel-img.conf
do_symlinks=yes
do_initrd=yes


-- System Information:
Debian Release: 4.0
  APT prefers testing
  APT policy: (500, 'testing')
Architecture: i386 (i686)
Shell:  /bin/sh linked to /bin/bash
Kernel: Linux 2.6.15-1-k7
Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968)

Versions of packages initramfs-tools depends on:
ii  busybox                       1:1.1.3-3  Tiny utilities for small and embed
ii  cpio                          2.6-17     GNU cpio -- a program to manage ar
ii  klibc-utils                   1.4.29-1   small statically-linked utilities
ii  module-init-tools             3.2.2-3    tools for managing Linux kernel mo
ii  udev                          0.100-2.2  /dev/ and hotplug management daemo

initramfs-tools recommends no packages.

-- no debconf information


--
To UNSUBSCRIBE, email to debian-bugs-dist-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Maximilian Attems-3]

On Mon, Dec 11, 2006 at 07:51:24AM +1100, Russell Coker wrote:

>
> When compressed a full modules.dep file is larger than a compressed
> depmod binary:
> # gzip -9 < /lib/modules/2.6.17-2-686/modules.dep |wc -c
> 15962
> # ls -l sbin/depmod
> -rwxr-xr-x 1 root root 37736 2006-12-11 07:38 sbin/depmod
> # gzip -9 < sbin/depmod | wc -c
> 16936
>
> When using a modules.dep file with the modules=list option the
> uncompressed file is 2601 bytes.
>
> It seems that using depmod instead of a modules.dep file is not going to
> save initramfs size for anyone, it just adds another program to be run
> (more compute time and another thing which can potentially break) and
> does no good.

nice idea, but in initramfs the depmod call is blazingly fast!
this can be easily timed in a qemu image.
so nacked.

i really don't see your antipathy on BUSYBOX=n
there you can get rid of glibc which is taking a _huge_
place on initramfs.
klibc-utils already ships dash as rescue shell.

best regards

--
maks


--
To UNSUBSCRIBE, email to debian-bugs-dist-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Russell Coker]

On Monday 11 December 2006 08:23, maximilian attems <maks@...>
wrote:
> nice idea, but in initramfs the depmod call is blazingly fast!
> this can be easily timed in a qemu image.
> so nacked.

The execution time is a minor issue.  The major issue is that the depmod
program is simply larger than a full modules.dep file when compressed.  Given
that most sane configurations will not have the complete set of kernel
modules (I use modules=list for a very small set of modules) the modules.dep
file will often be smaller than depmod when uncompressed.

Using the modules.dep file saves initramfs size in every case, and saves
memory in some common cases.

> i really don't see your antipathy on BUSYBOX=n
> there you can get rid of glibc which is taking a _huge_
> place on initramfs.


I use lvm root over a software RAID-1 on all my important machines (apart from
laptops which use LVM-root but no RAID).  Therefore glibc is required in the
initramfs for both LVM and RAID.  If I'm going to have glibc there anyway
then I might as well take advantages of the benefits that busybox offers!

If there were klibc versions of mdadm and vgchange then things would be
different.

Given that the quality of support for root on LVM and RAID has increased
significantly for etch (was it even in the installer before etch?) and that
cryptsetup (which is dynamically linked) is now supported for root it seems
likely that a very large portion of etch users will have glibc in their
initramfs, in fact a majority of users might have it.


--
To UNSUBSCRIBE, email to debian-bugs-dist-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Maximilian Attems-3]

On Mon, 11 Dec 2006, Russell Coker wrote:

> On Monday 11 December 2006 08:23, maximilian attems <maks@...>
> wrote:
> > nice idea, but in initramfs the depmod call is blazingly fast!
> > this can be easily timed in a qemu image.
> > so nacked.
>
> The execution time is a minor issue.

it was brought forward multiple times and showed to be false.

> The major issue is that the depmod
> program is simply larger than a full modules.dep file when compressed.  Given
> that most sane configurations will not have the complete set of kernel
> modules (I use modules=list for a very small set of modules) the modules.dep
> file will often be smaller than depmod when uncompressed.
>
> Using the modules.dep file saves initramfs size in every case, and saves
> memory in some common cases.

first of all MODULES=most is the default, so that is the common case!
the case you brought up had a small 1k difference,
so that's not the place to optimize.
 

> > i really don't see your antipathy on BUSYBOX=n
> > there you can get rid of glibc which is taking a _huge_
> > place on initramfs.
>
>
> I use lvm root over a software RAID-1 on all my important machines (apart from
> laptops which use LVM-root but no RAID).  Therefore glibc is required in the
> initramfs for both LVM and RAID.  If I'm going to have glibc there anyway
> then I might as well take advantages of the benefits that busybox offers!
>
> If there were klibc versions of mdadm and vgchange then things would be
> different.

hmm ok, i thought you were squeezing for some embedded board.
there i would consider also aboves possibility
othere than that i don't see an improvement.
 
> Given that the quality of support for root on LVM and RAID has increased
> significantly for etch (was it even in the installer before etch?) and that
> cryptsetup (which is dynamically linked) is now supported for root it seems
> likely that a very large portion of etch users will have glibc in their
> initramfs, in fact a majority of users might have it.

sure by default any etch user will have glibc on initramfs due to
busysbox, udev and module-init-tools.

--
maks


--
To UNSUBSCRIBE, email to debian-bugs-dist-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Russell Coker]

On Monday 11 December 2006 20:57, maximilian attems <maks@...>
wrote:

> > The major issue is that the depmod
> > program is simply larger than a full modules.dep file when compressed.
> > Given that most sane configurations will not have the complete set of
> > kernel modules (I use modules=list for a very small set of modules) the
> > modules.dep file will often be smaller than depmod when uncompressed.
> >
> > Using the modules.dep file saves initramfs size in every case, and saves
> > memory in some common cases.
>
> first of all MODULES=most is the default, so that is the common case!
> the case you brought up had a small 1k difference,
> so that's not the place to optimize.

MODULES=most will give a smaller modules.dep than the one that lives
under /lib/modules.  The modules.dep that comes from /lib/modules will take
up less space when compressed than depmod, so the MODULES=most case will save
even more space.

> > > i really don't see your antipathy on BUSYBOX=n
> > > there you can get rid of glibc which is taking a _huge_
> > > place on initramfs.
> >
> > I use lvm root over a software RAID-1 on all my important machines (apart
>
> hmm ok, i thought you were squeezing for some embedded board.

Just generally squeezing because I don't like wasting space.

> sure by default any etch user will have glibc on initramfs due to
> busysbox, udev and module-init-tools.

OK, as the default requires glibc therefore doing everything in busybox will
save space.  Long-term we should consider getting modprobe and other programs
included in busybox (modprobe used to be there for 2.4.x, we can do it
again).



--
To UNSUBSCRIBE, email to debian-bugs-dist-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Debian Bug Tracking System]

Your message dated Sun, 8 Nov 2009 22:19:24 +0100
with message-id <20091108211924.GF20676@...>
and subject line Re: initramfs-tools: depmod wastes space and time, please replace it with a modules.dep file
has caused the Debian Bug report #402499,
regarding initramfs-tools: depmod wastes space and time, please replace it with a modules.dep file
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner@...
immediately.)


--
402499: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=402499
Debian Bug Tracking System
Contact owner@... with problems

Package: initramfs-tools
Version: 0.85b
Severity: normal

When compressed a full modules.dep file is larger than a compressed
depmod binary:
# gzip -9 < /lib/modules/2.6.17-2-686/modules.dep |wc -c
15962
# ls -l sbin/depmod
-rwxr-xr-x 1 root root 37736 2006-12-11 07:38 sbin/depmod
# gzip -9 < sbin/depmod | wc -c
16936

When using a modules.dep file with the modules=list option the
uncompressed file is 2601 bytes.

It seems that using depmod instead of a modules.dep file is not going to
save initramfs size for anyone, it just adds another program to be run
(more compute time and another thing which can potentially break) and
does no good.


-- Package-specific info:
-- /proc/cmdline
console=ttyS0,115200 console=tty0 audit=1 selinux=1 ro root=/dev/md0 enforcing=0

-- /proc/filesystems
        cramfs
        ext3

-- lsmod
Module                  Size  Used by
nfsd                  208132  13
exportfs                5440  1 nfsd
lockd                  55752  2 nfsd
nfs_acl                 3392  1 nfsd
sunrpc                134332  9 nfsd,lockd,nfs_acl
ipt_REJECT              5120  91
ipt_LOG                 5760  2
ipv6                  222976  40
quota_v2                8448  6
dm_mirror              18832  0
softdog                 5716  2
iptable_filter          2816  1
ip_tables              17536  3 ipt_REJECT,ipt_LOG,iptable_filter
via_agp                 9344  1
agpgart                31496  1 via_agp
psmouse                32516  0
serio_raw               6596  0
via686a                15368  0
rtc                    11508  0
uhci_hcd               28304  0
shpchp                 39872  0
i2c_isa                 4672  1 via686a
usbcore               113924  2 uhci_hcd
pci_hotplug            24884  1 shpchp
i2c_core               19408  2 via686a,i2c_isa
parport_pc             32324  0
floppy                 54916  0
tulip                  45088  0
parport                31880  1 parport_pc
pcspkr                  1732  0
joydev                  8960  0
evdev                   8896  0
mousedev               10592  1
thermal                13512  0
processor              22976  1 thermal
fan                     4676  0
dm_snapshot            16348  0
dm_mod                 51512  6 dm_mirror,dm_snapshot
raid1                  18176  3
md_mod                 61652  4 raid1
ext3                  118664  5
jbd                    48724  1 ext3
mbcache                 8516  1 ext3
ide_disk               15936  10
ide_generic             1216  0 [permanent]
via82cxxx               8260  0 [permanent]
ide_core              112928  3 ide_disk,ide_generic,via82cxxx

-- kernel-img.conf
do_symlinks=yes
do_initrd=yes


-- System Information:
Debian Release: 4.0
  APT prefers testing
  APT policy: (500, 'testing')
Architecture: i386 (i686)
Shell:  /bin/sh linked to /bin/bash
Kernel: Linux 2.6.15-1-k7
Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968)

Versions of packages initramfs-tools depends on:
ii  busybox                       1:1.1.3-3  Tiny utilities for small and embed
ii  cpio                          2.6-17     GNU cpio -- a program to manage ar
ii  klibc-utils                   1.4.29-1   small statically-linked utilities
ii  module-init-tools             3.2.2-3    tools for managing Linux kernel mo
ii  udev                          0.100-2.2  /dev/ and hotplug management daemo

initramfs-tools recommends no packages.

-- no debconf information


> When compressed a full modules.dep file is larger than a compressed
> depmod binary:
> # gzip -9 < /lib/modules/2.6.17-2-686/modules.dep |wc -c
> 15962
> # ls -l sbin/depmod
> -rwxr-xr-x 1 root root 37736 2006-12-11 07:38 sbin/depmod
> # gzip -9 < sbin/depmod | wc -c
> 16936

the size difference is minimal.
the depmod in ramfs is blazing fast.
boot speed improvements were not demonstrated since, thus closing.