Maximum number of available partitions on Debian GNU/Linux (lenny).

[Teodor-Adrian]

Hi,

I'm trying to setup a mail server with flexible storage allocation.
The solution I was thinking about was to create 10 logical volumes
(LV) with the corresponding volume groups (VG), one for each major
mailbox storage (/home/a, /home/c, /home/d.. /home/m..). From the
start each of these will have only 60GB, but if the disk usage will be
over 80% I'll add a new logical partition to the VG and later resize
the LV. This schema could give a better utilization of disk space
(smaller partitions, >50% utilization, different sizes as needed,
flexibility to expand on demand).

So far the plan was good, in practice I've discovered that the Linux
kernel cannot access more than 15 partitions (all the partitions >=16
do NOT have a corresponding block device in /dev). I could create up
to 60 partitions using 'fdisk'. However, in /proc/partitions and /dev
only the first ones up to 15 are available.
The only references to this subject that I've found are these:
  http://www.justlinux.com/forum/showthread.php?t=152404
  http://osdir.com/ml/anaconda-devel-list/2009-04/threads.html#00438

Not being able to use more than 10 partitions for the mailboxes in
/home kind of ruins the above plan.
Does anyone from this list knows if there is a way to activate these
partitions? Probably I won't need more than 25-30 but only 10 logical
partitions are not enough in this case.

Thanks


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Keith Edmunds-2]

Why do you want separate partitions? Why not just one big partition with
separate directories? Then you don't need to worry about (re-)allocating
space between one directory and another.

--
Keith Edmunds

+-------------------------------------------------------------------------+
|    Tiger Computing Ltd    |  Helping businesses make the most of Linux  |
|  "The Linux Specialists"  |       http://www.tiger-computing.co.uk      |
+-------------------------------------------------------------------------+


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[hpw]

Teodor wrote:

> I'm trying to setup a mail server with flexible storage allocation.
> The solution I was thinking about was to create 10 logical volumes
> (LV) with the corresponding volume groups (VG), one for each major
> mailbox storage

Consider using only one VG and create all logical volumes in this volume
group.

> Not being able to use more than 10 partitions for the mailboxes in
> /home kind of ruins the above plan.

Of course you need only one Partition over all available space on each
hard disk. You use this partition to create a physical volume. All
physical volumes go into your one and only mail volume group.

HTH,

Hans Peter


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Teodor-Adrian]

On Fri, Oct 23, 2009 at 8:58 PM, Hans Peter Wiedau <hpw@...> wrote:
> Consider using only one VG and create all logical volumes in this volume
> group.
>
>> Not being able to use more than 10 partitions for the mailboxes in
>> /home kind of ruins the above plan.
>
> Of course you need only one Partition over all available space on each
> hard disk. You use this partition to create a physical volume. All
> physical volumes go into your one and only mail volume group.

Yes, this is an options too. I was trying to avoid this implementation
because there
is a greater risk of data loss with 1TB partitions than it is with
smaller ones. Also, if
the physical volume (PV) is corrupted I'm not sure if any of the LVs can be used
anymore or all data is lost. With smaller partitions the chances of
losing all mails
are not that high. Yes, I'm having two disks on software RAID1 too.

Thanks to Hans and Keith


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Keith Edmunds-2]

On Fri, 23 Oct 2009 21:40:36 +0300, mteodor@... said:

> I was trying to avoid this implementation because there is a greater
> risk of data loss with 1TB partitions than it is with smaller ones.

So run two HA servers with DRDB and RAID-1. Unlikely that you'd suffer any
data loss short of a B-737 mistaking your data centre for the intended
runway.

--
Keith Edmunds

+-------------------------------------------------------------------------+
|    Tiger Computing Ltd    |  Helping businesses make the most of Linux  |
|  "The Linux Specialists"  |       http://www.tiger-computing.co.uk      |
+-------------------------------------------------------------------------+


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Teodor-Adrian]

On Fri, Oct 23, 2009 at 10:50 PM, Keith Edmunds <kae@...> wrote:
> So run two HA servers with DRDB and RAID-1. Unlikely that you'd suffer any
> data loss short of a B-737 mistaking your data centre for the intended
> runway.

That's just too much for <100 email accounts. A simple RAID1 is enough for us.

Back to the original question, does anyone know if more than 15
partitions can be used in Linux?

Thanks


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Thomas Goirand]

Teodor wrote:

> So far the plan was good, in practice I've discovered that the Linux
> kernel cannot access more than 15 partitions (all the partitions >=16
> do NOT have a corresponding block device in /dev). I could create up
> to 60 partitions using 'fdisk'. However, in /proc/partitions and /dev
> only the first ones up to 15 are available.
> Does anyone from this list knows if there is a way to activate these
> partitions? Probably I won't need more than 25-30 but only 10 logical
> partitions are not enough in this case.
>
> Thanks

If I'm not mistaking, the hard limit to 16 partition is in the MSDOS
partitioning format, this is not a kernel limitation.

But really, I don't see what is your point in using so many partition
when you have LVM available. I really don't see how this would increase
safety of your data.

> Also, if the physical volume (PV) is corrupted I'm not sure if any
> of the LVs can be used anymore or all data is lost.
> With smaller partitions the chances of losing all mails
> are not that high.

While creating many partition can have some benefits (like reducing the
risk that a single user uses all the inodes for example...), I don't see
your point in trying to making physical partitions rather than using one
big PV.

Why do you think that there is a greater risk of having your PV
corrupted rather than having your partition table corrupted? This makes
absolutely no sense at all to me. Just use one big PV and create your
partitions in it, and don't try to make things too complicated for no
reasons...

Thomas


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Keith Edmunds-2]

On Fri, 23 Oct 2009 23:05:34 +0300, mteodor@... said:

> That's just too much for <100 email accounts. A simple RAID1 is enough
> for us.
>
> Back to the original question, does anyone know if more than 15
> partitions can be used in Linux?

Oh come on! You have fewer than 100 email accounts and you want more than
15 partitions?! If a simple dual-server setup is overkill for <100 email
accounts, how is 10 partitions not overkill?

Start from your requirements. not the solution. What, in business terms,
do you need to achieve?

--
Keith Edmunds

+-------------------------------------------------------------------------+
|    Tiger Computing Ltd    |  Helping businesses make the most of Linux  |
|  "The Linux Specialists"  |       http://www.tiger-computing.co.uk      |
+-------------------------------------------------------------------------+


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Thomas Goirand]

Teodor wrote:
> That's just too much for <100 email accounts. A simple RAID1 is enough for us.

WHAT? All this for less than 100 email accounts? Man ... this is insane!
Forget even about your idea of doing that a...z directories structure.

People do that when they have THOUSANDS of accounts, because otherwise,
ext3 is too slow (ever tried to do a "ls" on a directory with 100 000
files in it?). But that's the only reason...

Thomas


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Seth Mattinen]

Thomas Goirand wrote:
>
> People do that when they have THOUSANDS of accounts, because otherwise,
> ext3 is too slow (ever tried to do a "ls" on a directory with 100 000
> files in it?). But that's the only reason...
>

And even then one should consider using something other than ext3.

~Seth


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[hpw]

Thomas Goirand schrieb:

> If I'm not mistaking, the hard limit to 16 partition is in the MSDOS
> partitioning format, this is not a kernel limitation.

All SCSI disks are using the same major device number. Which device is
addressd is determined by the minor device number. This minor device
number is increased by 16 for every disk:

   disk     minor device number
   sda       0
   sdb      16
   sdc      32

The minor device number of partitions is calculated by adding the
partition number and the minor device number of the disk. So for sdc3 it
is 3 + 32 = 35.

So you are able to address 15 partitions, not more.

HTH,

Hans Peter


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Michelle Konzack-2]

Am 2009-10-23 22:22:15, schrieb Keith Edmunds:
> Oh come on! You have fewer than 100 email accounts and you want more than
> 15 partitions?! If a simple dual-server setup is overkill for <100 email
> accounts, how is 10 partitions not overkill?

A seperated partition could be usefull, if there are accounts with  MANY
messages in IMAP.

My personal message count is:

    X-TDMailSerialnumber: 9420324

This header is attached to any messages coming into my account.  Arround
37 GByte of messages of the last 14 years.  And per day I get ~2500 more

The total ammount of messages on my server are  ~280 million  with  over
8 TByte of storage and currently I am switching from two big servers Sun
Fire X4200 to 10 small X4100M2 and use the Courier IMAP Proxy.

The X4100 are inexpensive, and it just work from scratch using Debian.

Thanks, Greetings and nice Day/Evening
    Michelle Konzack
    Systemadministrator
    Tamay Dogan Network
    Debian GNU/Linux Consultant

--
Linux-User #280138 with the Linux Counter, http://counter.li.org/
##################### Debian GNU/Linux Consultant #####################
<http://www.tamay-dogan.net/>                 Michelle Konzack
<http://www.can4linux.org/>                   Apt. 917
<http://www.flexray4linux.org/>               50, rue de Soultz
Jabber linux4michelle@...           67100 Strabourg/France
IRC    #Debian (irc.icq.com)                  Tel. DE: [phone]
ICQ    #328449886                             Tel. FR: [phone]

[Michelle Konzack-2]

Am 2009-10-24 05:13:06, schrieb Thomas Goirand:
> But really, I don't see what is your point in using so many partition
> when you have LVM available. I really don't see how this would increase
> safety of your data.

A server crash with a 1 TByte volume will give headache for one week  if
you are forced to run fsck.  exspecialy if you 50 million mesages there.

I was already runing into this problem for some years on my courier-imap
and it was running arround 10 days to restore the volume.

> While creating many partition can have some benefits (like reducing the
> risk that a single user uses all the inodes for example...),

Even if ext3/reiserfs/xfs support several billion inodes, having  50 mio
of messages and more in a maildir partition...

Thanks, Greetings and nice Day/Evening
    Michelle Konzack
    Systemadministrator
    Tamay Dogan Network
    Debian GNU/Linux Consultant

--
Linux-User #280138 with the Linux Counter, http://counter.li.org/
##################### Debian GNU/Linux Consultant #####################
<http://www.tamay-dogan.net/>                 Michelle Konzack
<http://www.can4linux.org/>                   Apt. 917
<http://www.flexray4linux.org/>               50, rue de Soultz
Jabber linux4michelle@...           67100 Strabourg/France
IRC    #Debian (irc.icq.com)                  Tel. DE: [phone]
ICQ    #328449886                             Tel. FR: [phone]

[Thomas Goirand]

Michelle Konzack wrote:

> Am 2009-10-24 05:13:06, schrieb Thomas Goirand:
>> But really, I don't see what is your point in using so many partition
>> when you have LVM available. I really don't see how this would increase
>> safety of your data.
>
> A server crash with a 1 TByte volume will give headache for one week  if
> you are forced to run fsck.  exspecialy if you 50 million mesages there.
>
> I was already runing into this problem for some years on my courier-imap
> and it was running arround 10 days to restore the volume.

Yes, but here we are talking about less than 100 accounts...

Thomas


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...

[Teodor-Adrian]

On Sat, Oct 24, 2009 at 12:03 PM, Hans Peter Wiedau <hpw@...> wrote:

> All SCSI disks are using the same major device number. Which device is
> addressd is determined by the minor device number. This minor device
> number is increased by 16 for every disk:
>
>   disk     minor device number
>   sda       0
>   sdb      16
>   sdc      32
>
> The minor device number of partitions is calculated by adding the
> partition number and the minor device number of the disk. So for sdc3 it
> is 3 + 32 = 35.
>
> So you are able to address 15 partitions, not more.

Thank you for the technical explanation.

I've decided to follow the advice some of you gave me and create one
big VG with smaller LVs of 64GB or 128GB where appropriate.

Thanks


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@...
with a subject of "unsubscribe". Trouble? Contact listmaster@...