[ruby-core:23053] [Bug #1325] fiber tests kill windows

[Shyouhei Urabe]

Bug #1325: fiber tests kill windows
http://redmine.ruby-lang.org/issues/show/1325

Author: Charlie Savage
Status: Open, Priority: Normal
Target version: 1.9.1
ruby -v: ruby 1.9.2dev (2009-03-29) [i386-mswin32_90]

If run on its own, test_fiber.rb runs fine on windows (VC2008 as compiler).  But when run as part of the full test_suite, it eventually uses so much memory to cause windows to stop responding, start disk swapping, and finally to kill the ruby process.  This consistently happens.

Reducing the number of created fibers solves the issue (patch attached), but that's surely masking some deeper issue.


----------------------------------------
http://redmine.ruby-lang.org

[Shyouhei Urabe]

Issue #1325 has been updated by Roger Pack.


I run into this too--except it just segfaults.  (mingw)
----------------------------------------
http://redmine.ruby-lang.org/issues/show/1325

----------------------------------------
http://redmine.ruby-lang.org

[Shyouhei Urabe]

Issue #1325 has been updated by _ wanabe.

File noalloc_fiber.patch added

It's probably because the following situation.

1. fiber_init() (== Fiber#initialize) raises NoMemoryError.
2. next_fiber == NULL || next_fiber == NULL || local_storage == NULL ||
   saved_thread.stack is not allocated, but keeps original th->stack.
3. fiber_free() cause SEGV.

In this case, added Patch may prevent SEGV.
(I'm afraid that disk swapping is unavoidable.)
----------------------------------------
http://redmine.ruby-lang.org/issues/show/1325

----------------------------------------
http://redmine.ruby-lang.org

[Nobuyoshi Nakada-2]

Hi,

At Wed, 21 Oct 2009 00:45:15 +0900,
_ wanabe wrote in [ruby-core:26184]:
> It's probably because the following situation.

Seems reasonable.

> 1. fiber_init() (== Fiber#initialize) raises NoMemoryError.
> 2. next_fiber == NULL || next_fiber == NULL || local_storage == NULL ||
>    saved_thread.stack is not allocated, but keeps original th->stack.
> 3. fiber_free() cause SEGV.

In another case, when st_init_numtable() to initialize
local_storage fails, it might be left pointing the current
thread's local_storage, and cause double-free.  So I suspect
fib->saved_thread.local_storage should be cleared after
cont_init() in fiber_t_alloc().

Also, setting prev_fiber and next_fiber in root_fiber_alloc()
will be no longer needed.

--
Nobu Nakada

[Shyouhei Urabe]

Issue #1325 has been updated by _ wanabe.


Thank you, Nobu. I rewrote the patch in accordance with your advise.

And finally, I found which makes disk-swapping.
File.open('large.png', 'r:ascii-8bit') in test/cgi/test_cgi_multipart.rb does.
It does realloc many times and increases malloc_limit in gc.c.
Interestingly, mode 'rb:ascii-8bit' gets rid of the problem.

This is another example that enlargement of malloc_limit obstructs GC.

$ cat test.rb
require 'benchmark'
a = []
10.times do |i|
  ARGV[0].to_i.times do
    a[10000] = nil
    a.clear
  end
end

GC.start
GC::Profiler.enable

Benchmark.bm do |bm|
  bm.report do
    100_000.times do
      Fiber.new{}.resume
    end
  end
end

gc_result = GC::Profiler.result.split(/\n/)
gc_result[4..-3] = "(snip)"
puts nil, gc_result

$ ./ruby -v test.rb 0
ruby 1.9.2dev (2009-10-30 trunk 25566) [i386-mingw32]
      user     system      total        real
  4.516000   1.844000   6.360000 (  7.109375)

GC 294 invokes.
Index    Invoke Time(sec)       Use Size(byte)     Total Size(byte)         Total Object                    GC Time(ms)
    1               0.109                70704               229376                 9548         0.00000000000000000000
    2               0.109                70776               229376                 9548        15.62500000000000000000
(snip)
  292               4.594                70776               229376                 9548         0.00000000000000000000
  293               4.609                70776               229376                 9548         0.00000000000000000000


$ ./ruby -v test.rb 10000
ruby 1.9.2dev (2009-10-30 trunk 25566) [i386-mingw32]
      user     system      total        real
  6.406000   3.406000   9.812000 (108.718750)

GC 6 invokes.
Index    Invoke Time(sec)       Use Size(byte)     Total Size(byte)         Total Object                    GC Time(ms)
    1               5.375                70728               245760                10230         0.00000000000000000000
    2               5.562                70752               425984                17732         0.00000000000000000000
(snip)
    4               6.844                70848              1343488                55924         0.00000000000000000000
    5               9.266                70872              2392064                99572         0.00000000000000000000
----------------------------------------
http://redmine.ruby-lang.org/issues/show/1325

----------------------------------------
http://redmine.ruby-lang.org