|
View:
New views
20 Messages
—
Rating Filter:
Alert me
|
| < Prev | 1 - 2 | Next > |
|
|
[ruby-core:26361] [Feature #2294] [PATCH] ruby_bind_stack() to embed Ruby in coroutineFeature #2294: [PATCH] ruby_bind_stack() to embed Ruby in coroutine
http://redmine.ruby-lang.org/issues/show/2294 Author: Suraj Kurapati Status: Open, Priority: Normal Category: core, Target version: 1.9.2 Hi, I am attaching a "ruby_bind_stack.patch" patch file that adds a ruby_bind_stack() function to the Ruby C API. This function allows me to inform the GC about the stack boundaries of the coroutine inside which Ruby is embedded: void ruby_bind_stack(void *lower, void *upper); I am also attaching tarballs containing code examples that embed Ruby inside two different coroutine environments: UNIX System V contexts[1] and libpcl[2] coroutines. Each tarball has an "output.log" file which contains the result of running `script -c ./run.sh output.log` on my machine: Linux yantram 2.6.31-ARCH #1 SMP PREEMPT Tue Oct 13 13:36:23 CEST 2009 i686 Intel(R) Pentium(R) D CPU 3.00GHz GenuineIntel GNU/Linux The last section in "output.log" corresponds to Ruby @ SVN trunk that is patched with the "ruby_bind_stack.patch" patch file that is attached to this issue. Thanks for your consideration. [1]: http://www.gnu.org/s/libc/manual/html_node/System-V-contexts.html [2]: http://www.xmailserver.org/libpcl.html See also: * http://redmine.ruby-lang.org/issues/show/2258 * http://redmine.ruby-lang.org/issues/show/2126 ---------------------------------------- http://redmine.ruby-lang.org |
|
|
[ruby-core:26376] Re: [Feature #2294] [PATCH] ruby_bind_stack() to embed Ruby in coroutineHi,
At Wed, 28 Oct 2009 02:03:01 +0900, Suraj Kurapati wrote in [ruby-core:26361]: > I am attaching a "ruby_bind_stack.patch" patch file > that adds a ruby_bind_stack() function to the Ruby C API. This patch does not work with multithreading at all. -- Nobu Nakada |
|
|
[ruby-core:26380] [Feature #2294] [PATCH] ruby_bind_stack() to embed Ruby in coroutineIssue #2294 has been updated by Suraj Kurapati.
File ruby_bind_stack.patch added Hi, Nobu Nakada wrote: > This patch does not work with multithreading at all. Thank you for pointing out this problem. I have updated my patch accordingly and am reattaching it to this issue. Here is my approach for solving this problem: (Please correct me if I am wrong.) Since Ruby 1.9 threads are native kernel threads, they dynamically allocate and manage their own stacks. So the ruby_bind_stack() GC marking restriction must only be applied to the main Ruby thread---which isn't really a thread at all; it runs on the native C program stack. Thanks for your consideration. ---------------------------------------- http://redmine.ruby-lang.org/issues/show/2294 ---------------------------------------- http://redmine.ruby-lang.org |
|
|
[ruby-core:26384] [Feature #2294] [PATCH] ruby_bind_stack() to embed Ruby in coroutineIssue #2294 has been updated by Suraj Kurapati.
File ruby-ucontext-thread.tgz added File ruby-libpcl-thread.tgz added Hi, I am attaching two updated code examples which test the multi-threading support of my updated "ruby_bind_stack.patch" patch file. One example uses UNIX System V contexts and the other uses libpcl for embedding Ruby in coroutine. Thanks for your consideration. ---------------------------------------- http://redmine.ruby-lang.org/issues/show/2294 ---------------------------------------- http://redmine.ruby-lang.org |
|
|
[ruby-core:26387] [Feature #2294] [PATCH] ruby_bind_stack() to embed Ruby in coroutineIssue #2294 has been updated by Roman Shterenzon.
I'm embedding a Ruby 1.9.1 in my app, and it would die with segmentation fault, and I was suspicious about the stack, as it's multithreaded. I applied your patch and it looks fine so far. I'm using the following code (and assume that stack grows upward): static void pthread_get_stack(void **stack_begin, void **stack_end) { size_t stack_size; #if defined(HAVE_STACKADDR_NP) && defined(HAVE_GET_STACKSIZE_NP) /* MacOS X */ pthread_t t_id = pthread_self(); *stack_begin = pthread_get_stackaddr_np(t_id); stack_size = pthread_get_stacksize_np(t_id); #else /* Linux */ pthread_attr_t attr; pthread_getattr_np(pthread_self(), &attr); pthread_attr_getstack(&attr, stack_begin, &stack_size); #endif *stack_end = *stack_begin + stack_size; } ---------------------------------------- http://redmine.ruby-lang.org/issues/show/2294 ---------------------------------------- http://redmine.ruby-lang.org |
|
|
[ruby-core:26391] [Feature #2294] [PATCH] ruby_bind_stack() to embed Ruby in coroutineIssue #2294 has been updated by Suraj Kurapati.
Hi Roman, I did not understand your code example. Where do you call ruby_bind_stack() ? Without that call, I don't see how my patch can make any difference to your program. Thanks. ---------------------------------------- http://redmine.ruby-lang.org/issues/show/2294 ---------------------------------------- http://redmine.ruby-lang.org |
|
|
[ruby-core:26396] [Feature #2294] [PATCH] ruby_bind_stack() to embed Ruby in coroutineIssue #2294 has been updated by Roman Shterenzon.
Sorry for the lack of explanation, I thought that it was implicit and apparent. After I get thread's stack_begin and stack_end I'm calling your bind stack function, of course. ---------------------------------------- http://redmine.ruby-lang.org/issues/show/2294 ---------------------------------------- http://redmine.ruby-lang.org |
|
|
[ruby-core:26398] [Feature #2294] [PATCH] ruby_bind_stack() to embed Ruby in coroutineIssue #2294 has been updated by Suraj Kurapati.
Roman, thanks for clarifying. ---------------------------------------- http://redmine.ruby-lang.org/issues/show/2294 ---------------------------------------- http://redmine.ruby-lang.org |
|
|
[ruby-core:26418] [Feature #2294] [PATCH] ruby_bind_stack() to embed Ruby in coroutineIssue #2294 has been updated by Suraj Kurapati.
File ruby_bind_stack.patch added Hi, I am attaching an updated "ruby_bind_stack.patch" file which adds: * API documentation for the ruby_bind_stack() function in ruby.h * an assertion to ensure that upper > lower inside ruby_bind_stack() Thanks for your consideration. ---------------------------------------- http://redmine.ruby-lang.org/issues/show/2294 ---------------------------------------- http://redmine.ruby-lang.org |
|
|
[ruby-core:26444] [Feature #2294] [PATCH] ruby_bind_stack() to embed Ruby in coroutineIssue #2294 has been updated by Suraj Kurapati.
File ruby_bind_stack_after_refactoring.patch added Hi, To reduce your risk of applying (or even considering) this patch, I moved the refactoring of *duplicated* machine stack calculation code into a new "get_machine_stack_bounds.patch" file on this issue: http://redmine.ruby-lang.org/issues/show/2315 I am attaching a new "ruby_bind_stack_after_refactoring.patch" which basically contains the result of "ruby_bind_stack.patch" *minus* the changes in the "get_machine_stack_bounds.patch" mentioned above. Thanks for your consideration. ---------------------------------------- http://redmine.ruby-lang.org/issues/show/2294 ---------------------------------------- http://redmine.ruby-lang.org |
|
|
[ruby-core:26452] [Feature #2294] [PATCH] ruby_bind_stack() to embed Ruby in coroutineIssue #2294 has been updated by Suraj Kurapati.
File ruby_bind_stack.patch added File ruby_bind_stack_after_refactoring.patch added Hi, I'm attaching updated patches that contain better API documentation: /* * Binds the stack of Ruby's main thread to the region of memory that spans * inclusively from the given lower boundary to the given upper boundary: * * lower boundary <= stack pointer of Ruby's main thread <= upper boundary * * These boundaries *do not* protect Ruby's main thread against stack * overflow and they *do not* apply to non-main Ruby threads (whose stacks * are dynamically allocated and managed by the native Operating System). */ void ruby_bind_stack(void *lower_boundary, void *upper_boundary); Thanks for your consideration. ---------------------------------------- http://redmine.ruby-lang.org/issues/show/2294 ---------------------------------------- http://redmine.ruby-lang.org |
|
|
[ruby-core:26454] [Feature #2294] [PATCH] ruby_bind_stack() to embed Ruby in coroutineIssue #2294 has been updated by Suraj Kurapati.
File ruby_bind_stack.patch added File ruby_bind_stack_after_refactoring.patch added Hi, I'm attaching updated patches that reduce the runtime overhead of stack bound correction. Thanks for your consideration. ---------------------------------------- http://redmine.ruby-lang.org/issues/show/2294 ---------------------------------------- http://redmine.ruby-lang.org |
|
|
[ruby-core:26502] [Feature #2294] [PATCH] ruby_bind_stack() to embed Ruby in coroutineIssue #2294 has been updated by Suraj Kurapati.
File ruby_bind_stack_r25604.patch added Hi, Since my refactoring patch (from issue #2315) was accepted in r25604, I am attaching a new "ruby_bind_stack_r25604.patch" file accordingly. Thanks for your consideration. ---------------------------------------- http://redmine.ruby-lang.org/issues/show/2294 ---------------------------------------- http://redmine.ruby-lang.org |
|
|
[ruby-core:26550] [Feature #2294] [PATCH] ruby_bind_stack() to embed Ruby in coroutineIssue #2294 has been updated by Suraj Kurapati.
Hi, According to Matz's suggestion in [ruby-core:25139], I wrote a detailed explanation of the problem this patch solves. I hope this explanation is helpful. Please do not hesitate to ask for clarifications or to correct any misunderstandings. Thanks for your thoughtful consideration. == Introduction The patch adds a ruby_bind_stack() function to the Ruby C API. This function allows the person who is embedding Ruby to tell the Ruby GC about the stack boundaries of the embedded environment: void ruby_bind_stack(VALUE *lower_bound, VALUE *upper_bound); In order to understand why this function is important, please consider the following two modes of operation: normal & embedded. == Normal operation: Ruby runs in a C program's main() Initially, Ruby assumes that the stack of Ruby's main thread exists in a high memory address range, like this: (high memory address) 0xc1bff1f0 Ruby's stack upper boundary 0xbffff1f0 Ruby's stack lower boundary (low memory address) As Ruby runs, the lower boundary is adjusted (by the SET_STACK_END macro) to reflect the machine stack pointer: (high memory address) 0xc1bff1f0 Ruby's stack upper boundary (not changed) 0xc0ff1e80 Ruby's stack lower boundary (after update by SET_STACK_END) (low memory address) == Embedded operation: Ruby runs inside a C coroutine Initially, Ruby assumes that the stack of Ruby's main thread exists in a high memory address range, like this: (high memory address) 0xc1bff1f0 Ruby's stack upper boundary 0xbffff1f0 Ruby's stack lower boundary (low memory address) However, the stack of the C coroutine (which runs Ruby) exists at a low memory address range, because it is statically allocated: (high memory address) 0xc1bff1f0 Ruby's stack upper boundary 0xbffff1f0 Ruby's stack lower boundary 0x086032a0 System V context's stack upper boundary 0x082032a0 System V context's stack lower boundary (low memory address) As Ruby runs, the lower boundary is adjusted (by the SET_STACK_END macro) to reflect the machine stack pointer: (high memory address) 0xc1bff1f0 Ruby's stack upper boundary 0x086032a0 System V context's stack upper boundary 0x08601680 Ruby's stack lower boundary (after update by SET_STACK_END) 0x082032a0 System V context's stack lower boundary (low memory address) See the problem? Ruby's stack and the C coroutine stack do not agree. They overlap! This situation becomes worse (and causes a segfault) when the Ruby GC runs: it marks VALUEs in the Ruby stack, which currently contains all of the heap memory! Somewhere in the vast heap memory, it finds and dereferences a NULL value and BOOM! a segfault occurs. :-) To solve this problem, the ruby_bind_stack() function corrects Ruby's stack to reflect the stack boundaries of the C coroutine: (high memory address) 0x086032a0 Ruby's stack upper boundary and also System V context's stack upper boundary 0x08601680 Ruby's stack lower boundary (after update by SET_STACK_END) 0x082032a0 System V context's stack lower boundary (low memory address) Now, when the Ruby GC runs, it marks VALUEs in the correct memory region. It does not travel into heap memory and cause a segfault. That is all. Thanks for reading! ---------------------------------------- http://redmine.ruby-lang.org/issues/show/2294 ---------------------------------------- http://redmine.ruby-lang.org |
|
|
[ruby-core:26661] Re: [Feature #2294] [PATCH] ruby_bind_stack() to embed Ruby in coroutineHi,
I would like to say that without applying this patch, my ruby interpreter, embedded in a pthread, would cause a segmentation fault as soon as GC was invoked. I would like to see this applied to 1.9.1 as well as http://redmine.ruby-lang.org/issues/show/2279 . Without these, it's hardly possible to have ruby 1.9.1 embedded in a useful way. Thanks, --Roman ----- Original Message ---- From: Suraj Kurapati <redmine@...> To: ruby-core@... Sent: Thu, November 5, 2009 9:27:17 AM Subject: [ruby-core:26550] [Feature #2294] [PATCH] ruby_bind_stack() to embed Ruby in coroutine Issue #2294 has been updated by Suraj Kurapati. Hi, According to Matz's suggestion in [ruby-core:25139], I wrote a detailed explanation of the problem this patch solves. I hope this explanation is helpful. Please do not hesitate to ask for clarifications or to correct any misunderstandings. Thanks for your thoughtful consideration. == Introduction The patch adds a ruby_bind_stack() function to the Ruby C API. This function allows the person who is embedding Ruby to tell the Ruby GC about the stack boundaries of the embedded environment: void ruby_bind_stack(VALUE *lower_bound, VALUE *upper_bound); In order to understand why this function is important, please consider the following two modes of operation: normal & embedded. == Normal operation: Ruby runs in a C program's main() Initially, Ruby assumes that the stack of Ruby's main thread exists in a high memory address range, like this: (high memory address) 0xc1bff1f0 Ruby's stack upper boundary 0xbffff1f0 Ruby's stack lower boundary (low memory address) As Ruby runs, the lower boundary is adjusted (by the SET_STACK_END macro) to reflect the machine stack pointer: (high memory address) 0xc1bff1f0 Ruby's stack upper boundary (not changed) 0xc0ff1e80 Ruby's stack lower boundary (after update by SET_STACK_END) (low memory address) == Embedded operation: Ruby runs inside a C coroutine Initially, Ruby assumes that the stack of Ruby's main thread exists in a high memory address range, like this: (high memory address) 0xc1bff1f0 Ruby's stack upper boundary 0xbffff1f0 Ruby's stack lower boundary (low memory address) However, the stack of the C coroutine (which runs Ruby) exists at a low memory address range, because it is statically allocated: (high memory address) 0xc1bff1f0 Ruby's stack upper boundary 0xbffff1f0 Ruby's stack lower boundary 0x086032a0 System V context's stack upper boundary 0x082032a0 System V context's stack lower boundary (low memory address) As Ruby runs, the lower boundary is adjusted (by the SET_STACK_END macro) to reflect the machine stack pointer: (high memory address) 0xc1bff1f0 Ruby's stack upper boundary 0x086032a0 System V context's stack upper boundary 0x08601680 Ruby's stack lower boundary (after update by SET_STACK_END) 0x082032a0 System V context's stack lower boundary (low memory address) See the problem? Ruby's stack and the C coroutine stack do not agree. They overlap! This situation becomes worse (and causes a segfault) when the Ruby GC runs: it marks VALUEs in the Ruby stack, which currently contains all of the heap memory! Somewhere in the vast heap memory, it finds and dereferences a NULL value and BOOM! a segfault occurs. :-) To solve this problem, the ruby_bind_stack() function corrects Ruby's stack to reflect the stack boundaries of the C coroutine: (high memory address) 0x086032a0 Ruby's stack upper boundary and also System V context's stack upper boundary 0x08601680 Ruby's stack lower boundary (after update by SET_STACK_END) 0x082032a0 System V context's stack lower boundary (low memory address) Now, when the Ruby GC runs, it marks VALUEs in the correct memory region. It does not travel into heap memory and cause a segfault. That is all. Thanks for reading! ---------------------------------------- http://redmine.ruby-lang.org/issues/show/2294 ---------------------------------------- http://redmine.ruby-lang.org |
|
|
[ruby-core:26797] [Feature #2294] [PATCH] ruby_bind_stack() to embed Ruby in coroutineIssue #2294 has been updated by Suraj Kurapati.
Hi, Sorry to be impatient, but has there been any further decision or consideration about this patch? The only feedback I've received so far is that: * An early version of this patch did not support multi-threading (thanks to Mr. Nobu). * A later version of this patch worked for embedding Ruby 1.9 inside a pthread (thanks to Mr. Roman). The silent suspense is "killing" me, so to speak. :-) Thanks for your consideration. ---------------------------------------- http://redmine.ruby-lang.org/issues/show/2294 ---------------------------------------- http://redmine.ruby-lang.org |
|
|
[ruby-core:26799] Re: [Feature #2294] [PATCH] ruby_bind_stack() to embed Ruby in coroutineHi,
At Wed, 18 Nov 2009 15:55:07 +0900, Suraj Kurapati wrote in [ruby-core:26797]: > Sorry to be impatient, but has there been any > further decision or consideration about this patch? Sorry to be late. > The only feedback I've received so far is that: > > * An early version of this patch did not support > multi-threading (thanks to Mr. Nobu). > > * A later version of this patch worked for embedding > Ruby 1.9 inside a pthread (thanks to Mr. Roman). Switching stack using setcontext() can't work on all platforms. For instance, on NetBSD and older LinuxThread stack address is tightly bound to thread, and can't be changed. That is, your strategy is not portable. Why don't you simply use a thread instead? -- Nobu Nakada |
|
|
[ruby-core:26801] [Feature #2294](Closed) [PATCH] ruby_bind_stack() to embed Ruby in coroutineIssue #2294 has been updated by Nobuyoshi Nakada.
Status changed from Open to Closed % Done changed from 0 to 100 This issue was solved with changeset r25842. Suraj, thank you for reporting this issue. Your contribution to Ruby is greatly appreciated. May Ruby be with you. ---------------------------------------- http://redmine.ruby-lang.org/issues/show/2294 ---------------------------------------- http://redmine.ruby-lang.org |
|
|
[ruby-core:26802] [Feature #2294](Open) [PATCH] ruby_bind_stack() to embed Ruby in coroutineIssue #2294 has been updated by Nobuyoshi Nakada.
Status changed from Closed to Open Sorry, mistaken. ---------------------------------------- http://redmine.ruby-lang.org/issues/show/2294 ---------------------------------------- http://redmine.ruby-lang.org |
|
|
[ruby-core:26803] [Feature #2294](Rejected) [PATCH] ruby_bind_stack() to embed Ruby in coroutineIssue #2294 has been updated by Nobuyoshi Nakada.
Status changed from Open to Rejected I don't think it's good idea to it as public API. ---------------------------------------- http://redmine.ruby-lang.org/issues/show/2294 ---------------------------------------- http://redmine.ruby-lang.org |
| < Prev | 1 - 2 | Next > |
| Free embeddable forum powered by Nabble | Forum Help |