This is the mail archive of the cygwin-patches mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Fix performance on 10Gb networks


 You are right, of course. We initially thought it has to be a
multiple of page_size but it doesn't. I just re-tested with 63k and it
gives good perf too.
We get 600Mbits/second compared with 10Mb for the old default.
Attached the modified patch.

On Tue, Nov 18, 2014 at 8:43 PM, Corinna Vinschen
<corinna-cygwin@cygwin.com> wrote:
> Hi Iuliu,
>
> On Nov 18 19:30, Iuliu Rus wrote:
>> Hello,
>> Google is running Cygwin apps on its 10Gb networks and we are seeing
>> extremely bad performance in a couple of cases. For example, iperf
>> with the defaults results in only 10Mbits/sec.
>> We tracked this down to a combination of non-blocking sockets with
>> Nagle+delayed ack kicking in, since the apps eventually end up sending
>> a very small packets (2 bytes).
>> We have a case open against Microsoft but since everything is moving
>> very slow we would like to work around by picking socket buffers that
>> are multiple of 4k.
>
> Thanks for the patch.  One question:
>
>> Change log:
>> 2014-11-18 Iuliu Rus <rus.iuliu@gmail.com>
>>
>> * net.cc Change default values for socket buffers to fix performance
>> on 10Gb networks.
>>
>> Index: winsup/cygwin/net.cc
>> ===================================================================
>> RCS file: /cvs/src/src/winsup/cygwin/net.cc,v
>> retrieving revision 1.320
>> diff -u -p -r1.320 net.cc
>> --- winsup/cygwin/net.cc      13 Oct 2014 08:18:18 -0000      1.320
>> +++ winsup/cygwin/net.cc      18 Nov 2014 19:12:00 -0000
>> @@ -621,13 +621,16 @@ fdsock (cygheap_fdmanip& fd, const devic
>>       this is no problem on 64 bit.  So we set the default buffer size to
>>       the default values in current 3.x Linux versions.
>>
>> -     (*) Maximum normal TCP window size.  Coincidence?  */
>> +     (*) Maximum normal TCP window size.  Coincidence?
>> +
>> +     NOTE 3. Setting the window size to 65535 results in extremely
>> bad performance for apps that send data in multiples of Kb, as they
>> eventually end up sending 1 byte on the network and naggle + delay ack
>> kicks in. For example, iperf on a 10Gb network gives only 10 Mbits/sec
>> with a 65535 send buffer. We want this to be a multiple of PAGE_SIZE,
>> but since 64k breaks WSADuplicateSocket we use 60Kb.
>
> We do?  See below.
>
>> +*/
>>  #ifdef __x86_64__
>>    ((fhandler_socket *) fd)->rmem () = 212992;
>>    ((fhandler_socket *) fd)->wmem () = 212992;
>>  #else
>> -  ((fhandler_socket *) fd)->rmem () = 65535;
>> -  ((fhandler_socket *) fd)->wmem () = 65535;
>> +  ((fhandler_socket *) fd)->rmem () = 63488;
>> +  ((fhandler_socket *) fd)->wmem () = 63488;
>
> This is 62K, certainly not a multiple of the native PAGE_SIZE of 4K.
> And this makes me wonder.  Did you intend to use 60K and ended up with
> 62K for a reason?  And then, why not 63K as a multiple of 1K?
>
>
> Corinna
>
> --
> Corinna Vinschen                  Please, send mails regarding Cygwin to
> Cygwin Maintainer                 cygwin AT cygwin DOT com
> Red Hat

Attachment: net_patch
Description: Binary data


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]