Re: Performance increase

Well there are a couple of papers on these two subjects I will send out on a follow-up, but the jist of it is that flow-control interferes with TCP/IP congestion control and sliding window calculations by artificially limiting packet loss and increasing latency, while jumbo frames increase latency due to pipelining (cannot start to send data till whole 8k frame assembled, cannot on sender, cannot receive other frames while whole 8k frame is being received on the receiver).

Having said that though, there are situations where they are necessary.

1) Flow-control is necessary if the amount of data being received saturates the PCI bus thus TCP/IP is artificially hampered in a bad way which will negatively impact other PCI bus devices. Mostly seen with 10Gbe or a whole lot of 1Gbe interfaces running full blast at once.

2) Jumbo frames are useful if the number of frames and the amount of interrupts they generate are overwhelming the CPU and it cannot achieve the throughput desired. You will use this for 1Gbe for io requests 64k or > or with 10Gbe on a CPU not quite up to snuff.

For general 1Gbe file system io which usually is 4-8k io request size, the use of these will reduce throughput by increasing latency.

If you do a lot of large block io then think about Jumbo Frames, but you still will not need flow-control.

-Ross


-----Original Message-----
From: iscsitarget-devel-bounces@... <iscsitarget-devel-bounces@...>
To: 'Pasi Kärkkäinen' <pasik@...>
CC: iscsitarget-devel@... <iscsitarget-devel@...>
Sent: Fri Apr 27 00:49:21 2007
Subject: Re: [Iscsitarget-devel] Performance increase

Hello,

 With iSCSI it's all about flowcontrol.  Flowcontrol tells the sender to
stop sending when a device, typically the initiator, can't handle any more
packets.  W/O flowcontrol packets will start being dropped and have to be
retransmitted. There is a penalty for that.  Jumbo Frames won't alleviate
that congestion issue.  If you're familiar with serial communications you
can easily see what happens as you increase the baud rate w/o flowcontrol on
a terminal. Garbage characters. Add XON/XOFF and you can run at much higher
speeds.

 iSCSI SANs tend to have very 'bursty' traffic patterns.  Jumbo frames also
more quickly use up buffers in the switch.  Low-end switches tend to have a
very small amount of buffers.  I.e. there are Cisco's that have up to
2MB/port.  I've seen switches as low as 32K/port.  With a good switch you
can run both Jumbo Frames and Flowcontrol.  If not, then stick strictly to
flowcontrol.  When troubleshooting start with flowcontrol on and jumbo
frames off.  Run some tests, ie. IOmeter, then turn on Jumbo frames.  Re-run
the test.

 Regards,
 Don


-----Original Message-----
From: Pasi Kärkkäinen [pasik@...]
Sent: Friday, April 27, 2007 12:25 AM
To: Don Williams
Cc: 'Paul Roberts'; iscsitarget-devel@...
Subject: Re: [Iscsitarget-devel] Performance increase

On Wed, Apr 25, 2007 at 04:58:36PM -0400, Don Williams wrote:
> Do you have a switch available?   Performance with a switch is better than
> direct.  Switches add buffering and flowcontrol.  The latter being
critical
> for iSCSI SANs. 
>
>

Hi!

Can you comment on this a bit.. some people say jumbo frames are important,
and some people say flow control is important.. Do you have deeper knowledge
or analysis of these and the effects of them?

-- Pasi
 


-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Iscsitarget-devel mailing list
Iscsitarget-devel@...
https://lists.sourceforge.net/lists/listinfo/iscsitarget-devel

---

This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof.
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Iscsitarget-devel mailing list
Iscsitarget-devel@...
https://lists.sourceforge.net/lists/listinfo/iscsitarget-devel