Nabble - DRBD

"Bendita a nação cujo Deus é o SENHOR!"

On 11/26/2009 07:33 PM, gilberto nunes wrote:

Well...

I never use it!.. :-)

You are aware that only the init script takes care of the
become-primary-on logic?

Well if you weren't, you are now. :)

Cheers,
Florian

_______________________________________________
drbd-user mailing list
drbd-user@...
http://lists.linbit.com/mailman/listinfo/drbd-user

On Thu, Nov 26, 2009 at 04:17:36PM -0200, gilberto nunes wrote:

Now I thing that I need zeroing the partition used to DRBD....

But this is a destruction procedure...
I do not want lost the data record on it!

any light!!

Don't use Gentoo ;)

No, actually I meant,
double check your init script.

Hello DRBD users,

We used DRBD with version 8.2.6 ( internal metadata ) and we found a
slowdown disk resources on DRBD partitions after have upgrade DRBD
release to 8.3.1 or 8.3.6 ( external metadata on
the same disk ). We didn't understand
why DRBD performance between older and newer version have declined and
the difference between real partition and DRBD partition performance ! 

We have trying to increase activity log size without best performance.
We have trying also to disable cache with more performance... If
someone have an idea.

Please to read it.

regards

Resources and program used :

- tstDrbd is a program
that write lines to a file.

- hdparm

- iostat

- iftop

We made following tests :

DRBD 8.2.6 ( C protocol ) using 2.6.17 kernel with CPU ( VIA Nehemiah /
1002.462 Mhz / 2006.94 bogomips )

Network interface using to DRBD : 10/100Mbs

> hdparm -ctT /dev/hda

/dev/hda:

 IO_support   =  1 (32-bit)

 Timing cached reads:   258 MB in  2.01 seconds = 128.19 MB/sec

 Timing buffered disk reads:   72 MB in  3.07 seconds =  23.46 MB/sec

-- TEST with 5000000
lines --

/root partition :
tstDrbd -f
/root/tstDrbd.txt -n 5000000
Time to write 5000000 lines (hh:mm:ss) :
00:00:18.                    <=============\

/drbd partition :
tstDrbd -f
/drbd/tstDrbd.txt -n 5000000
Time to write 5000000 lines (hh:mm:ss) : 00:00:20.                
   <============== ~ Same performance

-- TEST with 7000000
lines --
/root partition :
tstDrbd -f
/root/tstDrbd.txt -n 7000000
Time to write 7000000 lines (hh:mm:ss) :
00:00:26.                    <==============\

/drbd partition :
tstDrbd -f
/root/tstDrbd.txt -n 7000000
Time to write 7000000 lines (hh:mm:ss) : 00:00:40.

   <============== ~ DRBD slowness

If we used both partition test in the same time, the /root test
increased the delay to write !!!

During the test Writing program we launched iostat to analyzed
IO resources. About IO disk stats :

avg-cpu:  %user   %nice %system %iowait  %steal   %idle

          25,87    0,00   74,13    0,00    0,00    0,00

Device:         rrqm/s   wrqm/s   r/s   w/s   rsec/s   wsec/s avgrq-sz
avgqu-sz   await  svctm  %util

hda               0,00   548,26  0,00 58,71     0,00 39761,19  
677,29     3,05   54,27  11,05  64,88

drbd0             0,00     0,00  0,00 605,47     0,00 38276,62   
63,22  1827,88 1649,08   1,64  99,50

The DRBD driver to used at 100% and hard disk only at 65%.

During this test, network bandwidth using to synchronized data used 90
% of 100 Mbs during all test ( iftop on the selected interface )

DRBD setup :

DRBD network parameters :
 resource drbd {
            protocol C;

            net {      

                shared-secret "NUMLOG";

                ....

            }

        syncer {
            rate 40M;
            al-extents 257;
       }

        disk {

            on-io-error call-local-io-error;

        }

        on sv1 {

            device    /dev/drbd0;

            disk      /dev/hda8;

            address   192.168.20.1:7789;

            meta-disk  internal;

        }

        on sv2 {

            device    /dev/drbd0;

            disk      /dev/hda7;

            address   192.168.20.2:7789;

            meta-disk  internal;

        }

---------------------------------------------------------------------------------------

DRBD 8.3.6 or 8.3.1 ( B protocol ) using kernel 2.6.29.1 with CPU (
Intel(R) Atom(TM) CPU 230 1.60GHz / 3192.12 
bogomips )
Network interface using to DRBD : 10/100Mbs

> hdparm -ctT /dev/hda

/dev/hda:

 IO_support   =  0 (default 16-bit)

 Timing cached reads:   1264 MB in  2.00 seconds = 632.16 MB/sec

 Timing buffered disk reads:   84 MB in  3.01 seconds =  27.89 MB/sec

-- TEST with 5000000 lines --

/root partition :
tstDrbd -f
/root/tstDrbd.txt -n 5000000

Time to write
 5000000 lines
(hh:mm:ss) : 00:00:19.

/drbd partition : 

tstDrbd -f
/drbd/tstDrbd.txt -n
5000000                                                  

Time to write 5000000 lines (hh:mm:ss) : 00:00:33.                  
<==============
~ DRBD slowness

-- TEST with 7000000
lines --
/root partition :

tstDrbd -f /root/tstDrbd.txt -n 7000000

Time to write 7000000 lines (hh:mm:ss) : 00:00:25.

/drbd partition :

tstDrbd -f /drbd/tstDrbd.txt -n 7000000

Time to write
 7000000 lines
(hh:mm:ss) : 00:01:07.

<============== ~ DRBD slowness

During the test Writing program we launched iostat to analyzed IO
resources. About IO disk stats :

avg-cpu:  %user   %nice %system %iowait  %steal   %idle

           1.49    0.00    2.99   47.26    0.00   48.26

Device:         rrqm/s   wrqm/s   r/s   w/s   rsec/s  
wsec/s avgrq-sz avgqu-sz   await  svctm  %util

hda7              0.00   123.50  0.00 68.50     0.00 
8428.00   123.04     0.49    6.57   2.80  19.20      /dev/hda9

hda9              0.00     0.00  0.00  1.00     0.00    
2.00     2.00     0.15  152.00 152.00  15.20      Metadata

drbd0             0.00     0.00  0.00 192.50     0.00 
8656.00    44.97    23.10  108.75   5.21 100.20     /dev/drbd0

The DRBD driver to used at 100% and hard disk only at 20%.

During this test, network bandwidth using to synchronized data
used 20 to 40 % of 100 Mbs during all test ( iftop on the selected
interface )

DRBD setup :

DRBD network parameters :
 resource drbd {
            protocol B;

            net {      

                shared-secret "NUMLOG";

                ....

            }

        syncer {
            rate 40M;
            al-extents 257;
       }

        disk {

            on-io-error call-local-io-error;

            no-disk-flushes;

        }

        on sv1 {

            device    /dev/drbd0;

            disk      /dev/hda7;

            address   192.168.20.1:7789;

            meta-disk  /dev/hda9[0];

        }

        on sv2 {

            device    /dev/drbd0;

            disk      /dev/hda7;

            address   192.168.20.2:7789;

            meta-disk  /dev/hda9[0];

        }

-- 
Fabrice LE CREURER
Développement / Support technique EDTI FT-MASTER
Developer engineer / Helpdesk FT-MASTER product
NUMLOG - Internet : http://www.numlog.fr
Tel : (+33) 1 30 79 16 16 - Fax: (+33) 1 30 81 92 86

I've been using HP NC510C PCIe 10 gigabit nic(netxen)
since early 2009 in
a drbd setup between two DL380G5. We did experience
hanging issues with 
the card but this was related to driver versions(HP
psp support packs).
We ended up opening a case with HP and are currently
running on an "older" 
version of the nx_nic driver. If you want I will send
you the specifics offline.
BTW I just purchased some DL380G6 with NC522SFP(with
BLc copper)and will be setting them
    up in the New Year.
> Message: 4

> Date: Wed, 25 Nov 2009 11:45:23 -0700

> From: Mike Lovell <mike@...>

> Subject: Re: [DRBD-user] 8.3.5 Stalling on sync

> To: James Larcombe <jim@...>

> Cc: drbd-user@...

> Message-ID: <4B0D7B43.3060302@...>

> Content-Type: text/plain; charset="iso-8859-1"; Format="flowed"

> hrm. i thought i had heard of someone using drbd
over 10 gig with netxen

> cards. i went looking for a few minutes and didn't find anything though.

> my recommendation would be try newer drivers either through compiling

> the drivers for you existing kernel or using a newer kernel. i don't

> have details on how to do that for your cards cause i have never used

> any 10 gig from hp or netxen. other than that, my only recommendation
is

> new nics.

> good luck

> mike

> James Larcombe wrote:

> >

> > Hi Mike,

> >

> >

> >

> > The cards I'm using are HP NC522SFP Dual Port 10GbE Server Adapters

> > with HP BLc 10Gb SR SFP+ Fiber Transceivers. I could try running
these

> > with 1GB Fiber cables instead of 10GB.

> >

> >

> >

> > James

> >

> >

> >

> > *From:* Mike Lovell [mailto:mike@...]

> > *Sent:* 25 November 2009 16:01

> > *To:* James Larcombe

> > *Cc:* drbd-user@...

> > *Subject:* Re: [DRBD-user] 8.3.5 Stalling on sync

> >

> >

> >

> > nothing i tried tweaking in drbd.conf worked. the only thing
that did

> > was changing the 10gig interfaces. what cards are you using?
i was

> > using ones with an intel chip. the cards that i did get it to
work

> > with were from chelsio. in my previous thread on the list, someone

> > mentioned that they had neterion cards working.

> >

> > mike

> >

> > James Larcombe wrote:

> >

> > Hi Mike,

> >

> >

> >

> > Thanks for the quick response. Yes you are correct we are using
10gig

> > fibre cards. I'm not sure we could change them though as the
fibre

> > modules used in them cost over ?400 each.

> >

> >

> >

> > Is there anything I can tweak in the drbd.conf file to get these
to work.

> >

> >

> >

> > James

> >

> >

> >

> > *From:* Mike Lovell [mailto:mike@...]

> > *Sent:* 24 November 2009 17:49

> > *To:* James Larcombe

> > *Cc:* drbd-user@... <mailto:drbd-user@...>

> > *Subject:* Re: [DRBD-user] 8.3.5 Stalling on sync

> >

> >

> >

> > James Larcombe wrote:

> >

> > Hi List,

> >

> >

> >

> > Please help. I have installed drbd 8.3.5 on Open Suse 11.1 (Kernel

> > 2.6.27.29-0.1).

> >

> >

> >

> > I have run drbdadm create-md dbms-test on one node and create-md

> > dbms-test2 on the other node. I then ran drbdadm up all on both
nodes.

> > I then ran drbdadm -- --overwrite-data-of-my-peer primary dbms-test
on

> > the first node and the same with dbms-test2 on the other node.
They

> > then run for a short while before stalling. I have tried older
version

> > without success and turning the sync rate down does not make
any

> > difference. Downing the resources and bringing back up starts
the sync

> > again but this then stalls quickly.

> >

> >

> >

> > I have attached /proc/drbd, /etc/drbd.conf and a section from

> > /var/log/messages. Any pointers would be greatly appreciated.

> >

> >

> >

> > version: 8.3.5 (api:88/proto:86-91)

> >

> > GIT-hash: ded8cdf09b0efa1460e8ce7a72327c60ff2210fb build by

> > root@hp-tm-40, 2009-11-24 12:21:46

> >

> >  0: cs:SyncSource ro:Secondary/Secondary ds:UpToDate/Inconsistent
C r----

> >

> >     ns:160896 nr:0 dw:0 dr:160896 al:0 bm:9 lo:1 pe:0
ua:0 ap:0 ep:1

> > wo:b oos:926694296

> >

> >         [>.] sync'ed:  0.1% (905040/905132)M
     4972

> >

> >         stalled

> >

> >  1: cs:SyncTarget ro:Secondary/Secondary ds:Inconsistent/UpToDate
C r----

> >

> >     ns:0 nr:2173248 dw:2173248 dr:0 al:0 bm:132 lo:0
pe:29878 ua:0

> > ap:0 ep:1 wo:b oos:777971256

> >

> >         [>.] sync'ed:  0.3% (759736/761856)M

> >

> >         Stalled

> >

> >

> > what kind of network are you using between the two servers? this
is

> > almost the exact same behavior i had when i was trying to get
drbd to

> > work over 10gig ethernet. turned out to be something in drbd
didn't

> > like something about the 10gig cards i had. i eventually had
to change

> > my network cards. what cards are you using? 1gig? 10gig? have
you

> > tried other cards? that is where i would look.

> >

> > mike

> >

> >

> >

Nabble - DRBD

Re: Primary on both nodes (SOLVED)

Feature suggestion for primary/primary configurations

Re: Primary on both nodes (SOLVED)

Re: Primary on both nodes (SOLVED)

Primary not disconnecting Secondary with IO problems

Re: Primary on both nodes (SOLVED)

Re: Primary on both nodes (SOLVED)

Re: Primary on both nodes (SOLVED)

Re: Primary on both nodes

Re: Primary on both nodes

Re: Primary on both nodes

Re: Primary on both nodes

Re: Primary on both nodes

Re: Primary on both nodes

Re: Primary on both nodes

Re: Primary on both nodes

Re: Primary on both nodes

Lower performance between DRBD 8.26 and release 8.3.1 or 8.3.6

Re: Primary on both nodes

Re: Primary on both nodes

Primary on both nodes

A problem with drbd 8.3.4

Re: Problems updating to 8.3.6 - could not load drbd module

Re: Problems updating to 8.3.6 - could not load drbd module

Re: Problems updating to 8.3.6 - could not load drbd module

Problems updating to 8.3.6 - could not load drbd module

Re: 8.3.5 Stalling on sync (Mike Lovell)

Re: 8.3.5 Stalling on sync

Re: Errors compiling drbd-8.3.6 on SLES 11

Re: 8.3.5 Stalling on sync

Errors compiling drbd-8.3.6 on SLES 11

Re: WAN based replication...

Re: 8.3.5 Stalling on sync

Re: WAN based replication...

Re: 8.3.5 Stalling on sync