Patch for term_to_binary() stack overflow

[Jan Lehnardt-3]

Dear OTP Team,

https://issues.apache.org/jira/browse/COUCHDB-275 describes an issue
with Erlang's term_to_binary() BIF and includes a patch. The core of the
issue is that term_to_binary() is implemented in a recursive C function
that blows the stack when converting deeply nested term structures. We
see this when people try to read deeply nested JSON structures that we
convert from Erlang terms to binaries representing JSON for delivery.

The easiest way to reproduce this is

term_to_binary(lists:foldl(fun(E,A) -> [E, A] end, [], lists:seq(1,  
100000))).

See https://issues.apache.org/jira/secure/attachment/12401866/term_to_binary_fix.diff

for a non-recursive version of term_to_binary().

How do other projects deal with this issue? If there is no workaround,  
is
it possible to apply the patch in the next OTP release?

On behalf of Damien Katz and the CouchDB team,

Jan Lehnardt
--

_______________________________________________
erlang-patches mailing list
erlang-patches@...
http://www.erlang.org/mailman/listinfo/erlang-patches

[Bjorn Gustavsson-2]

On Tue, Mar 10, 2009 at 11:40 PM, Jan Lehnardt <jan@...> wrote:

> Dear OTP Team,
>
> https://issues.apache.org/jira/browse/COUCHDB-275 describes an issue
> with Erlang's term_to_binary() BIF and includes a patch. The core of the
> issue is that term_to_binary() is implemented in a recursive C function
> that blows the stack when converting deeply nested term structures. We
> see this when people try to read deeply nested JSON structures that we
> convert from Erlang terms to binaries representing JSON for delivery.
>
> The easiest way to reproduce this is
>
> term_to_binary(lists:foldl(fun(E,A) -> [E, A] end, [], lists:seq(1,
> 100000))).
>
> See https://issues.apache.org/jira/secure/attachment/12401866/term_to_binary_fix.diff
>
> for a non-recursive version of term_to_binary().
>
> How do other projects deal with this issue? If there is no workaround,
> is
> it possible to apply the patch in the next OTP release?

We will not include the suggested patch in OTP.

However, we do intend to fix the problem as soon as possible,
in a way that degrades performance as little as possible,
possibly already in the R13B release.

/Bjorn
--
Björn Gustavsson, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-patches mailing list
erlang-patches@...
http://www.erlang.org/mailman/listinfo/erlang-patches

[Bjorn Gustavsson-2]

On Tue, Mar 10, 2009 at 11:40 PM, Jan Lehnardt <jan@...> wrote:

> Dear OTP Team,
>
> https://issues.apache.org/jira/browse/COUCHDB-275 describes an issue
> with Erlang's term_to_binary() BIF and includes a patch. The core of the
> issue is that term_to_binary() is implemented in a recursive C function
> that blows the stack when converting deeply nested term structures. We
> see this when people try to read deeply nested JSON structures that we
> convert from Erlang terms to binaries representing JSON for delivery.
>
> The easiest way to reproduce this is
>
> term_to_binary(lists:foldl(fun(E,A) -> [E, A] end, [], lists:seq(1,
> 100000))).
>

We have now removed deep recursion from the implementation of term_to_binary/1
(will appear in the R13B release). Seems to be somewhat faster than
before, at least
for deeply nested terms.

/Bjorn
--
Björn Gustavsson, Erlang/OTP, Ericsson AB
_______________________________________________
erlang-patches mailing list
erlang-patches@...
http://www.erlang.org/mailman/listinfo/erlang-patches