Nabble - Erlang Questions

An English word stemmer

2009-11-10T13:18:10Z

Hi,

I've dusted off an English word stemmer [1] I made
quite a while ago. I've seen at least one other erlang
stemmer before, and that's one by Hans Nilsson which
was included in Joe's erlang book as well as in Ulf
Wiger's rdbms.

This stemmer is an implementation of the Porter2
stemming algorithm [2] and does its job fairly quickly
at approximately 400,000 words per second on my 2.53
GHz Core 2 Duo MacBook Pro. I've tested it on the
example vocabulary (available from [2]) and all words
are stemmed according to the already prepared list of
stemmed words.

Code (and tests only a few clicks away) here:

http://github.com/klajo/hacks/blob/master/stem/src/stem_en.erl

There's also a Swedish version of the algorithm [3] in
the repository which I whipped together just for the
fun of it. The English algorithm does a better job at
stemming words though.

In case someone's interested... :-)

Cheers,
Klas

[1] Read more on stemming for example here:
http://en.wikipedia.org/wiki/Stemming

[2] The Porter2 stemming algorithm
http://snowball.tartarus.org/algorithms/english/stemmer.html

[3] The Swedish stemming algorithm
http://snowball.tartarus.org/algorithms/swedish/stemmer.html

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

Re: Why Beam.smp crashes when memory is over?

2009-11-10T12:22:12Z

Dropping messages, suspending processes, and crashing processes is
just a bad idea.

Erlang's messaging is not "left-guarded" in the sense described by
Hoare. That means that any behavior that suspends processes when the
remote mailbox is full can either exhaust memory (i.e. what we have
now) or arbitrarily deadlock the system. The only requirement for a
deadlock is a loop in messaging anywhere, and we have more than a few
of those.

Dropping messages is probably the least disruptive, but (in
applications that use OTP behaviors at least) it would just translate
into gen_server:call timeouts, and we're back to defensive programming
again.

I'm all for a heuristic that sends a signal to a program when it's
getting close (i.e. memory allocation failed and "reserved" memory is
starting to be consumed so trigger an alarm of some sort), but
anything proposed so far compromises the current behavior of the VM in
ways that are awfully unpredictable.

On Nov 10, 2009, at 4:45 AM, Joe Armstrong wrote:

> This is a very interesting problem. if processes have quotas, then how
> could you set the
> quota value?
>
> A perfectly correct process might just have a very deep stack, just
> once in its
> life and otherwise be fine. Whether to crash this process or not would
> depend upon
> what the other processes in the system happened to be doing at the
> time. This would
> be very unfortunate - it's like your program being hit by a cosmic ray
> - nasty. It creates a random non-deterministic coupling between things
> that are supposed to be independent.
>
> A possibility that just occurred to me might be to suspend processes
> that appear to be
> running wild until such a time as the overall memory situation looks
> good.
>
> Image two scheduler queues. One for well behaved programs. One for
> programs whose
> stacks and heaps are growing too fast. If memory is no problem we run
> programs from both queues. If memory is tight we run processes in the
> "problem" queue less often and with frequent garbs.
>
> Killing a program with a large stack and heap, just because their
> happens to be a
> temporary memory problem seems horrible, especially since the problem
> might go away
> if we wait a few milliseconds.
>
> Suspending a memory hungry process for a while, until memory is
> available seems
> less objectionable. Perhapse it could be swapped out to disk and
> pulled in a lot later.
> Killing things at random in the hope it might help sounds like a
> really bad idea.
> Process migration could solve this - move it to a machine that has got
> more memory.
>
> Suspending things seems ok - you might even suspend an errant
> process forever
> and reclaim the memory - but not kill it. Some other process could
> detect that the processes
> is not responding and kill it and thus all the semantics of the
> application would be obeyed
> (processes are allowed to be unresponsive, that's fine) and the
> semantics of the
> error recovery should say what to do in this case.
>
> Just killing processes when they have done nothing wrong is not a
> good idea.
>
> /Joe
>
>
>
> On Tue, Nov 10, 2009 at 1:10 PM, Ulf Wiger
> <ulf.wiger@...> wrote:
>> Richard O'Keefe wrote:
>>>>
>>>> One way would be to let the user set a memory quota on a process
>>>> with
>>>> options at spawn time. When the process reaches it quota it can be
>>>> automatically killed or the user can
>>>> be notified in some way and take actions.
>>>
>>> One of the reasons this hasn't been done is, I presume, the fact
>>> that
>>> it is quite difficult for a programmer to determine what the memory
>>> quota should be. It depends on
>>> ...
>>
>> I implemented resource limits in erlhive - at the Erlang level rather
>> than in the VM. The purpose was to be able to run foreign code safely
>> in a hosted environment. Eliminating the possibility to do damage
>> through traditional side-effects was relatively easy with a code
>> transform, but two ways of staging a DoS attack would be to gobble
>> RAM or CPU capacity. I approached this by inserting calls to a check
>> function that sampled heap size, and started a "watchdog" process
>> that
>> would unceremoniously kill the program after a certain time.
>>
>> In short, I can see a need for such limits, and would like to include
>> a reduction ceiling. The limits could be set after careful testing
>> and high enough that they protect against runaway processes. A
>> reduction
>> limit could be checked at the end of each slice, perhaps.
>>
>> In my experience, per-process memory usage is fairly predictable in
>> erlang. Does anyone have a different experience?
>>
>> BR,
>> Ulf W
>> --
>> Ulf Wiger
>> CTO, Erlang Training & Consulting Ltd
>> http://www.erlang-consulting.com
>>
>> ________________________________________________________________
>> erlang-questions mailing list. See http://www.erlang.org/faq.html
>> erlang-questions (at) erlang.org
>>
>>
>
> ________________________________________________________________
> erlang-questions mailing list. See http://www.erlang.org/faq.html
> erlang-questions (at) erlang.org
>

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

Re: package namespaces question

2009-11-10T11:58:05Z

Gracias Enrique!

But how can i import all system modules and apps?

Has gs become lib.gs or something alike?

Well, the URLS from 2004 were bit old ive just found the current version at http://www.erlang.org/doc/man/packages.html

I saw no prior examples of packcages used on the list certainly not back in 2008/2009 so an " almost deprecated" notice at the top
of the page is all i need to stop at here and continue reading other things....

/Angel

El Martes, 10 de Noviembre de 2009 Enrique Marcote escribió:

> Hola Angel,
>
> Try importing gs.
>
> Saludos,
>
> Enrique
>
>
> El 10/11/2009, a las 20:10, Angel Alvarez escribió:
>
> > Hi Guys!
> >
> > Im currently reading erlang archives from 2004
> >
> > On 10/05/2004 Someone talked about package namespaces ("Is erlang too small" thread) so i decided to just test this "new2 erlang knowledge.
> >
> > iv just borrowed some little code from a gs article... :-)
> >
> > the module starts like this:
> >
> >
> > -module(gui).
> > -export([init/0]).
> >
> > init() ->
> > S = gs:start(),
> > Win = gs:create(window, S, [{width, 350}, {height, 100}]),
> > gs:create(button, quit, Win, [{label, {text, "Quit"}}, {x, 0}]),
> > ...
> >
> > so on the erlang shell gui:init(). just runs the example..
> > So i changed the module to use package style names...
> >
> >
> > -module(angel.tests.gui).
> > -export([init/0]).
> >
> > init() ->
> > S = gs:start(),
> > Win = gs:create(window, S, [{width, 350}, {height, 100}]),
> > gs:create(button, quit, Win, [{label, {text, "Quit"}}, {x, 0}]),
> > ...
> >
> > and moved the gui.erl to $PWD/angel/test/gui.erl
> >
> >
> >> sinosuke@sushiba:~/Datos/Docs/Personal/Erlang/Code/angel/test> erlc gui.erl
> >> cd ../../
> >> sinosuke@sushiba:~/Datos/Docs/Personal/Erlang/Code/
> >
> > but now firing at the shell results in :
> >
> > sinosuke@sushiba:~/Datos/Docs/Personal/Erlang/Code> erl
> > Erlang R13B01 (erts-5.7.2) [source] [rq:1] [async-threads:0] [hipe] [kernel-poll:false]
> >
> > Eshell V5.7.2 (abort with ^G)
> > 1> angel.test.gui:init().
> > ** exception error: undefined function 'angel.test.gs':start/0
> > in function 'angel.test.gui':init/0
> >
> > What's wrong?
> > Is needed the same convention on the rest of modules
> > after you start using package name styles?
> >
> >
> > From http://www.erlang.se/publications/packages.html
> >
> > "m2:g(X) becomes a call to foo.bar.m2
> >
> > If this is not what was intended, the call can be written explicitly, as in
> > -module(foo.bar.m1).
> > -export([f/1]).
> >
> > f(X) -> fee.fie.foe.m2:g(X)."
> >
> > it seems bit embarrasing, i think is why perhaps no one is using this feature 5 years later...
> >
> >
> > Thanks Angel
> > --
> > No imprima este correo si no es necesario. El medio ambiente está en nuestras manos.
> > ->>-----------------------------------------------
> > Clist UAH a.k.a Angel
> > ---------------------------------[www.uah.es]-<<--
> >
> > Tú lo compras, yo lo copio. Todo legal.
> >
> > ________________________________________________________________
> > erlang-questions mailing list. See http://www.erlang.org/faq.html
> > erlang-questions (at) erlang.org
> >
>
>

--
Agua para todo? No, Agua para Todos.
->>-----------------------------------------------
Clist UAH a.k.a Angel
---------------------------------[www.uah.es]-<<--

No le daría Cocacola Zero, ni a mi peor enemigo. Para eso está el gas Mostaza que es mas piadoso.

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

Re: package namespaces question

2009-11-10T11:39:32Z

Hola Angel,

Try importing gs.

Saludos,

Enrique

El 10/11/2009, a las 20:10, Angel Alvarez escribió:

> Hi Guys!
>
> Im currently reading erlang archives from 2004
>
> On 10/05/2004 Someone talked about package namespaces ("Is erlang too small" thread) so i decided to just test this "new2 erlang knowledge.
>
> iv just borrowed some little code from a gs article... :-)
>
> the module starts like this:
>
>
> -module(gui).
> -export([init/0]).
>
> init() ->
> S = gs:start(),
> Win = gs:create(window, S, [{width, 350}, {height, 100}]),
> gs:create(button, quit, Win, [{label, {text, "Quit"}}, {x, 0}]),
> ...
>
> so on the erlang shell gui:init(). just runs the example..
> So i changed the module to use package style names...
>
>
> -module(angel.tests.gui).
> -export([init/0]).
>
> init() ->
> S = gs:start(),
> Win = gs:create(window, S, [{width, 350}, {height, 100}]),
> gs:create(button, quit, Win, [{label, {text, "Quit"}}, {x, 0}]),
> ...
>
> and moved the gui.erl to $PWD/angel/test/gui.erl
>
>
>> sinosuke@sushiba:~/Datos/Docs/Personal/Erlang/Code/angel/test> erlc gui.erl
>> cd ../../
>> sinosuke@sushiba:~/Datos/Docs/Personal/Erlang/Code/
>
> but now firing at the shell results in :
>
> sinosuke@sushiba:~/Datos/Docs/Personal/Erlang/Code> erl
> Erlang R13B01 (erts-5.7.2) [source] [rq:1] [async-threads:0] [hipe] [kernel-poll:false]
>
> Eshell V5.7.2 (abort with ^G)
> 1> angel.test.gui:init().
> ** exception error: undefined function 'angel.test.gs':start/0
> in function 'angel.test.gui':init/0
>
> What's wrong?
> Is needed the same convention on the rest of modules
> after you start using package name styles?
>
>
> From http://www.erlang.se/publications/packages.html
>
> "m2:g(X) becomes a call to foo.bar.m2
>
> If this is not what was intended, the call can be written explicitly, as in
> -module(foo.bar.m1).
> -export([f/1]).
>
> f(X) -> fee.fie.foe.m2:g(X)."
>
> it seems bit embarrasing, i think is why perhaps no one is using this feature 5 years later...
>
>
> Thanks Angel
> --
> No imprima este correo si no es necesario. El medio ambiente está en nuestras manos.
> ->>-----------------------------------------------
> Clist UAH a.k.a Angel
> ---------------------------------[www.uah.es]-<<--
>
> Tú lo compras, yo lo copio. Todo legal.
>
> ________________________________________________________________
> erlang-questions mailing list. See http://www.erlang.org/faq.html
> erlang-questions (at) erlang.org
>

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

package namespaces question

2009-11-10T11:10:13Z

Hi Guys!

Im currently reading erlang archives from 2004

On 10/05/2004 Someone talked about package namespaces ("Is erlang too small" thread) so i decided to just test this "new2 erlang knowledge.

iv just borrowed some little code from a gs article... :-)

the module starts like this:

-module(gui).
-export([init/0]).

init() ->
S = gs:start(),
Win = gs:create(window, S, [{width, 350}, {height, 100}]),
gs:create(button, quit, Win, [{label, {text, "Quit"}}, {x, 0}]),
...

so on the erlang shell gui:init(). just runs the example..
So i changed the module to use package style names...

-module(angel.tests.gui).
-export([init/0]).

init() ->
S = gs:start(),
Win = gs:create(window, S, [{width, 350}, {height, 100}]),
gs:create(button, quit, Win, [{label, {text, "Quit"}}, {x, 0}]),
...

and moved the gui.erl to $PWD/angel/test/gui.erl

> sinosuke@sushiba:~/Datos/Docs/Personal/Erlang/Code/angel/test> erlc gui.erl
> cd ../../
> sinosuke@sushiba:~/Datos/Docs/Personal/Erlang/Code/

but now firing at the shell results in :

sinosuke@sushiba:~/Datos/Docs/Personal/Erlang/Code> erl
Erlang R13B01 (erts-5.7.2) [source] [rq:1] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.2 (abort with ^G)
1> angel.test.gui:init().
** exception error: undefined function 'angel.test.gs':start/0
in function 'angel.test.gui':init/0

What's wrong?
Is needed the same convention on the rest of modules
after you start using package name styles?

From http://www.erlang.se/publications/packages.html

"m2:g(X) becomes a call to foo.bar.m2

If this is not what was intended, the call can be written explicitly, as in
-module(foo.bar.m1).
-export([f/1]).

f(X) -> fee.fie.foe.m2:g(X)."

it seems bit embarrasing, i think is why perhaps no one is using this feature 5 years later...

Thanks Angel
--
No imprima este correo si no es necesario. El medio ambiente está en nuestras manos.
->>-----------------------------------------------
Clist UAH a.k.a Angel
---------------------------------[www.uah.es]-<<--

Tú lo compras, yo lo copio. Todo legal.

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

Re: Best way to implement a simple cache

2009-11-10T09:45:51Z

On Tue, Nov 10, 2009 at 11:25 AM, Jachym Holecek <freza@...> wrote:

> Hello,
>
> # Jan Vincent 2009-11-10:
>> I wanted your take on how to implement a simple small key-value
>> cache, maybe holding around a few hundred tuples. The thing is, I
>> don't want the cache to be consuming all my memory so that entries
>> in the cache expires if it isn't read for some time or some maximum
>> size is met.
>
> An ETS table owned by a gen_server that runs periodic cleanup on it?
>

I find myself writing purpose built gen_servers that maintain the
cache their state for this sort of thing. To 'expire' items in the
cache, you could run another process as a timer that calls an expire
method on the gen_server. There are more moving parts here, but
they're decoupled and avoid using ETS for what's a pretty simple
caching requirement.

Garrett

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

Re: Best way to implement a simple cache

2009-11-10T09:25:40Z

Hello,

# Jan Vincent 2009-11-10:
> I wanted your take on how to implement a simple small key-value
> cache, maybe holding around a few hundred tuples. The thing is, I
> don't want the cache to be consuming all my memory so that entries
> in the cache expires if it isn't read for some time or some maximum
> size is met.

An ETS table owned by a gen_server that runs periodic cleanup on it?

-- Jachym

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

Re: Re: SCTP accept?

2009-11-10T08:09:46Z

On the contrary to TCP where a client socket corresponds to a single
communication channel to be used between the server/client pair, with
SCTP (when using one-to-many style interface, which is what's currently
supported in the SCTP driver) you have one SCTP socket to work with and
one association per client end-point.

You can send replies from more than one Erlang server-side worker
process as long as you pass it the Socket::sctp_socket() and
AssocID::assoc_id() (the AssocID comes in the #sctp_assoc_change{} event
when a new client association is established). Then you can use
gen_sctp:send/3 or gen_sctp:send/4 functions to send your HTTP payload.

Serge

Steve Davis wrote:

> I must be incredibly dense! Many words later, I'm still 100% unclear
> as to how to use gen_sctp effectively to do "http over sctp".
>
> The documentation shows an
> "Example of an Erlang SCTP Server which receives SCTP messages and
> prints them on the standard output."
> ...but does not really address how you reply unless you do everything
> from the one server process.
>
> To have any chance of scaling I would think that you'd want to process
> client requests in a number of worker processes at the server side
> (obviously)?
>
> With tcp you hand over the socket you get from accept -- but I'm
> entirely unclear what the equivalent of controlling_process(Socket,
> Pid) would be for associations?
>
> Any education would be very gratefully "accepted"!
>
> Best,
> Steve
>
> ________________________________________________________________
> erlang-questions mailing list. See http://www.erlang.org/faq.html
> erlang-questions (at) erlang.org
>

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

Re: Why Beam.smp crashes when memory is over?

2009-11-10T05:40:52Z

Joe Armstrong wrote:
>
> Just killing processes when they have done nothing wrong is not a good idea.

Well, it's optional, of course. :)

Imagine, OTOH, a well-tested system where memory characteristics
have been well charted for the foreseeable cases. It might be
defensible to set resource limits so that everything we expect
to see falls well within the limit, and stuff that we don't
expect might trigger the killing of some process. If this is
done on temporary processes, we should be able to accept it
as long as the number of spurious kills is low.

This is not much stranger than things that we do routinely in
other cases:

- If dets or disk_log notice that a file hasn't been properly
closed, it 'repairs' the file - that is, it repairs the index.
Corrupt objects are simply discarded, not repaired.

- Replication in the AXD 301 and similar products was asynchronous
with a bulking factor. Some failure cases could lead to dropped
calls, but as long as they were few, it was acceptable.

- Some complex state machines would bail out for unexpected
sequences (I showed an example of this in my Structured Network
Programming talk at EUC). This was a form of "complexity
overload", and hugely unfair to the poor process running the
code, as it was probably not a real failure case.

- Mnesia's deadlock prevention algorithm, or indeed any deadlock
prevention algo, will restart transactions if there is even
the smallest chance of deadlock. Granted, this should be
transparent if the transaction fun is well written, but there
will be false positives, and this will affect performance.

On the other hand, there can be situations where a rogue process
gobbles up all available memory, rendering the VM unresponsive
for several minutes (e.g. due to the infamous "loss of sharing"),
or cases where a number of unexpectedly large processes "gang up"
and kill the VM in one big memory spike. Or a difficult-to-reproduce
bug that sends some application into an infinite retry situation
rendering the system unusable. In all these cases, killing off
the poor culprits, guilty or not, may well result in a less deadly
disturbance for the system as a whole.

BR,
Ulf W
--
Ulf Wiger
CTO, Erlang Training & Consulting Ltd
http://www.erlang-consulting.com

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

Re: Why Beam.smp crashes when memory is over?

2009-11-10T05:22:52Z

Tony Rogvall wrote:
> Not yet.
>
> What kind of flavor do you have in mind?
>
> One proposal is to let sender crash when receiver in box is full.
>
> I would like to test a blocking version as well. This may sound like
> utterly crazy, but we already have the infamous busy_port ;-)

...and the punishing of senders if the receiver has a long
message queue. I think these are abominations in the multicore
world, and should be phased out.

> A suggestion from Patrik Winroth is to be able to set thresholds on
> resource limits, say in percent from max limit.

This could be done as an extension of erlang:system_monitor().
It already has the ability to trigger on large heap and long
GC, etc.

BR,
Ulf W
--
Ulf Wiger
CTO, Erlang Training & Consulting Ltd
http://www.erlang-consulting.com

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

Re: Why Beam.smp crashes when memory is over?

2009-11-10T05:19:55Z

Tony Rogvall wrote:
> Not yet.
>
> What kind of flavor do you have in mind?
>
> One proposal is to let sender crash when receiver in box is full.

I would prefer it if messages are simply dropped. This would be
symmetric with the distributed case - you can hardly crash a
remote process, or a port in active mode. :)

BR,
Ulf W
--
Ulf Wiger
CTO, Erlang Training & Consulting Ltd
http://www.erlang-consulting.com

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

Re: Why Beam.smp crashes when memory is over?

2009-11-10T05:13:02Z

On Tue, Nov 10, 2009 at 1:45 PM, Joe Armstrong <erlang@...> wrote:
> This is a very interesting problem. if processes have quotas, then how
> could you set the
> quota value?

If you for example have processes representing subscribers och
half-calls or mobile phones
it is pretty easy to know a reasonable size of such a process by measurement.
It should be easy to say if it can grow up to 100 kwords, 1Mwords, 10
Mwords etc.
Of course you should not try to set quota as tight as possible. The
quota on process
size is just a way to protect the system from total disaster if one of
these processes
happens to grow over all reasonable limits because of a bug.

/Kenneth Erlang/OTP Ericsson

>
> A perfectly correct process might just have a very deep stack, just once in its
> life and otherwise be fine. Whether to crash this process or not would
> depend upon
> what the other processes in the system happened to be doing at the
> time. This would
> be very unfortunate - it's like your program being hit by a cosmic ray
> - nasty. It creates a random non-deterministic coupling between things
> that are supposed to be independent.
>
> A possibility that just occurred to me might be to suspend processes
> that appear to be
> running wild until such a time as the overall memory situation looks good.
>
> Image two scheduler queues. One for well behaved programs. One for
> programs whose
> stacks and heaps are growing too fast. If memory is no problem we run
> programs from both queues. If memory is tight we run processes in the
> "problem" queue less often and with frequent garbs.
>
> Killing a program with a large stack and heap, just because their
> happens to be a
> temporary memory problem seems horrible, especially since the problem
> might go away
> if we wait a few milliseconds.
>
> Suspending a memory hungry process for a while, until memory is available seems
> less objectionable. Perhapse it could be swapped out to disk and
> pulled in a lot later.
> Killing things at random in the hope it might help sounds like a
> really bad idea.
> Process migration could solve this - move it to a machine that has got
> more memory.
>
> Suspending things seems ok - you might even suspend an errant process forever
> and reclaim the memory - but not kill it. Some other process could
> detect that the processes
> is not responding and kill it and thus all the semantics of the
> application would be obeyed
> (processes are allowed to be unresponsive, that's fine) and the semantics of the
> error recovery should say what to do in this case.
>
> Just killing processes when they have done nothing wrong is not a good idea.
>
> /Joe
>
>
>
> On Tue, Nov 10, 2009 at 1:10 PM, Ulf Wiger
> <ulf.wiger@...> wrote:
>> Richard O'Keefe wrote:
>>>>
>>>> One way would be to let the user set a memory quota on a process with
>>>> options at spawn time. When the process reaches it quota it can be
>>>> automatically killed or the user can
>>>> be notified in some way and take actions.
>>>
>>> One of the reasons this hasn't been done is, I presume, the fact that
>>> it is quite difficult for a programmer to determine what the memory
>>> quota should be. It depends on
>>> ...
>>
>> I implemented resource limits in erlhive - at the Erlang level rather
>> than in the VM. The purpose was to be able to run foreign code safely
>> in a hosted environment. Eliminating the possibility to do damage
>> through traditional side-effects was relatively easy with a code
>> transform, but two ways of staging a DoS attack would be to gobble
>> RAM or CPU capacity. I approached this by inserting calls to a check
>> function that sampled heap size, and started a "watchdog" process that
>> would unceremoniously kill the program after a certain time.
>>
>> In short, I can see a need for such limits, and would like to include
>> a reduction ceiling. The limits could be set after careful testing
>> and high enough that they protect against runaway processes. A reduction
>> limit could be checked at the end of each slice, perhaps.
>>
>> In my experience, per-process memory usage is fairly predictable in
>> erlang. Does anyone have a different experience?
>>
>> BR,
>> Ulf W
>> --
>> Ulf Wiger
>> CTO, Erlang Training & Consulting Ltd
>> http://www.erlang-consulting.com
>>
>> ________________________________________________________________
>> erlang-questions mailing list. See http://www.erlang.org/faq.html
>> erlang-questions (at) erlang.org
>>
>>
>
> ________________________________________________________________
> erlang-questions mailing list. See http://www.erlang.org/faq.html
> erlang-questions (at) erlang.org
>
>

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

Re: Why Beam.smp crashes when memory is over?

2009-11-10T04:45:17Z

This is a very interesting problem. if processes have quotas, then how
could you set the
quota value?

A perfectly correct process might just have a very deep stack, just once in its
life and otherwise be fine. Whether to crash this process or not would
depend upon
what the other processes in the system happened to be doing at the
time. This would
be very unfortunate - it's like your program being hit by a cosmic ray
- nasty. It creates a random non-deterministic coupling between things
that are supposed to be independent.

A possibility that just occurred to me might be to suspend processes
that appear to be
running wild until such a time as the overall memory situation looks good.

Image two scheduler queues. One for well behaved programs. One for
programs whose
stacks and heaps are growing too fast. If memory is no problem we run
programs from both queues. If memory is tight we run processes in the
"problem" queue less often and with frequent garbs.

Killing a program with a large stack and heap, just because their
happens to be a
temporary memory problem seems horrible, especially since the problem
might go away
if we wait a few milliseconds.

Suspending a memory hungry process for a while, until memory is available seems
less objectionable. Perhapse it could be swapped out to disk and
pulled in a lot later.
Killing things at random in the hope it might help sounds like a
really bad idea.
Process migration could solve this - move it to a machine that has got
more memory.

Suspending things seems ok - you might even suspend an errant process forever
and reclaim the memory - but not kill it. Some other process could
detect that the processes
is not responding and kill it and thus all the semantics of the
application would be obeyed
(processes are allowed to be unresponsive, that's fine) and the semantics of the
error recovery should say what to do in this case.

Just killing processes when they have done nothing wrong is not a good idea.

/Joe

On Tue, Nov 10, 2009 at 1:10 PM, Ulf Wiger
<ulf.wiger@...> wrote:

> Richard O'Keefe wrote:
>>>
>>> One way would be to let the user set a memory quota on a process with
>>> options at spawn time. When the process reaches it quota it can be
>>> automatically killed or the user can
>>> be notified in some way and take actions.
>>
>> One of the reasons this hasn't been done is, I presume, the fact that
>> it is quite difficult for a programmer to determine what the memory
>> quota should be. It depends on
>> ...
>
> I implemented resource limits in erlhive - at the Erlang level rather
> than in the VM. The purpose was to be able to run foreign code safely
> in a hosted environment. Eliminating the possibility to do damage
> through traditional side-effects was relatively easy with a code
> transform, but two ways of staging a DoS attack would be to gobble
> RAM or CPU capacity. I approached this by inserting calls to a check
> function that sampled heap size, and started a "watchdog" process that
> would unceremoniously kill the program after a certain time.
>
> In short, I can see a need for such limits, and would like to include
> a reduction ceiling. The limits could be set after careful testing
> and high enough that they protect against runaway processes. A reduction
> limit could be checked at the end of each slice, perhaps.
>
> In my experience, per-process memory usage is fairly predictable in
> erlang. Does anyone have a different experience?
>
> BR,
> Ulf W
> --
> Ulf Wiger
> CTO, Erlang Training & Consulting Ltd
> http://www.erlang-consulting.com
>
> ________________________________________________________________
> erlang-questions mailing list. See http://www.erlang.org/faq.html
> erlang-questions (at) erlang.org
>
>

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

Best way to implement a simple cache

2009-11-10T04:35:17Z

Hi guys,

I wanted your take on how to implement a simple small key-value cache,
maybe holding around a few hundred tuples. The thing is, I don't want
the cache to be consuming all my memory so that entries in the cache
expires if it isn't read for some time or some maximum size is met.

Jan Vincent Liwanag
jvliwanag@...

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

Re: Why Beam.smp crashes when memory is over?

2009-11-10T04:25:54Z

2009/11/10 Tony Rogvall <tony@...>

> Not yet.
>
> What kind of flavor do you have in mind?
>
> One proposal is to let sender crash when receiver in box is full.
>

A few strategies to choose from would be useful.

* Crash sender
* Crash receiver
* Discard message

cheers
Chandru

I would like to test a blocking version as well. This may sound like utterly
> crazy, but we already have the infamous busy_port ;-)
> Busy ports do block senders (mostly distribution stuff).
>
An other flavor is to drop the overflow message. This can however be done

> by catch (Pid ! Message) if the crash semantics is implemented.
>
> A suggestion from Patrik Winroth is to be able to set thresholds on
> resource limits,
> say in percent from max limit. When threshold is reached then a warning
> message
> is sent to a logger process.
>
> The max_memory limit currently control number of words! that can be
> allocated for heaps.
> This is the sum of sizes of all heaps for the process and it's
> "sub"-processes
> (spawned by process that had the limit set in the first place). It is
> implementation specific
> if the message queue and/or the messages them selfs are located on the
> process heap.
> In the case where messages are none shared and located on a separate heap
> fragments
> this should be accounted for.
>
> /Tony
>
>
>
>
>
> On 10 nov 2009, at 12.18, Chandru wrote:
>
> 2009/11/9 Tony Rogvall <tony@...>
>>
>> Interesting discussion!
>>>
>>> I have been working on a resource system for Erlang for nearly two years
>>> now.
>>> I have a working (tm) prototype where you can set resource limits like
>>> max_processes/max_ports/max_memory/max_time/max_reductions ...
>>>
>>>
>> You don't happen to have a max_message_q_size option do you? Or does
>> max_memory implicitly enforce this?
>>
>> Chandru
>>
>
>

Re: Why Beam.smp crashes when memory is over?

2009-11-10T04:10:24Z

Richard O'Keefe wrote:
>> One way would be to let the user set a memory quota on a process with
>> options at spawn time. When the process reaches it quota it can be
>> automatically killed or the user can
>> be notified in some way and take actions.
>
> One of the reasons this hasn't been done is, I presume, the fact that
> it is quite difficult for a programmer to determine what the memory
> quota should be. It depends on
> ...

I implemented resource limits in erlhive - at the Erlang level rather
than in the VM. The purpose was to be able to run foreign code safely
in a hosted environment. Eliminating the possibility to do damage
through traditional side-effects was relatively easy with a code
transform, but two ways of staging a DoS attack would be to gobble
RAM or CPU capacity. I approached this by inserting calls to a check
function that sampled heap size, and started a "watchdog" process that
would unceremoniously kill the program after a certain time.

In short, I can see a need for such limits, and would like to include
a reduction ceiling. The limits could be set after careful testing
and high enough that they protect against runaway processes. A reduction
limit could be checked at the end of each slice, perhaps.

In my experience, per-process memory usage is fairly predictable in
erlang. Does anyone have a different experience?

BR,
Ulf W
--
Ulf Wiger
CTO, Erlang Training & Consulting Ltd
http://www.erlang-consulting.com

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

Re: Why Beam.smp crashes when memory is over?

2009-11-10T04:04:48Z

Not yet.

What kind of flavor do you have in mind?

One proposal is to let sender crash when receiver in box is full.

I would like to test a blocking version as well. This may sound like
utterly
crazy, but we already have the infamous busy_port ;-)
Busy ports do block senders (mostly distribution stuff).

An other flavor is to drop the overflow message. This can however be
done
by catch (Pid ! Message) if the crash semantics is implemented.

A suggestion from Patrik Winroth is to be able to set thresholds on
resource limits,
say in percent from max limit. When threshold is reached then a
warning message
is sent to a logger process.

The max_memory limit currently control number of words! that can be
allocated for heaps.
This is the sum of sizes of all heaps for the process and it's "sub"-
processes
(spawned by process that had the limit set in the first place). It is
implementation specific
if the message queue and/or the messages them selfs are located on the
process heap.
In the case where messages are none shared and located on a separate
heap fragments
this should be accounted for.

/Tony

On 10 nov 2009, at 12.18, Chandru wrote:

> 2009/11/9 Tony Rogvall <tony@...>
>
>> Interesting discussion!
>>
>> I have been working on a resource system for Erlang for nearly two
>> years
>> now.
>> I have a working (tm) prototype where you can set resource limits
>> like
>> max_processes/max_ports/max_memory/max_time/max_reductions ...
>>
>
> You don't happen to have a max_message_q_size option do you? Or does
> max_memory implicitly enforce this?
>
> Chandru

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

Re: Why Beam.smp crashes when memory is over?

2009-11-10T03:18:04Z

2009/11/9 Tony Rogvall <tony@...>

> Interesting discussion!
>
> I have been working on a resource system for Erlang for nearly two years
> now.
> I have a working (tm) prototype where you can set resource limits like
> max_processes/max_ports/max_memory/max_time/max_reductions ...
>

You don't happen to have a max_message_q_size option do you? Or does
max_memory implicitly enforce this?

Chandru

Re: Why Beam.smp crashes when memory is over?

2009-11-10T01:08:22Z

El Martes, 10 de Noviembre de 2009 05:36:44 Richard O'Keefe escribió:

>
> On Nov 9, 2009, at 10:53 PM, Kenneth Lundin wrote:
> > What we could do is to make it easier for the user to prevent OOM
> > situations and also to
> > let him take the decision when it occurs or rather before it occurs.
> >
> > One way would be to let the user set a memory quota on a process with
> > options at spawn time. When the process reaches it quota it can be
> > automatically killed or the user can
> > be notified in some way and take actions.
>
> One of the reasons this hasn't been done is, I presume, the fact that
> it is quite difficult for a programmer to determine what the memory
> quota should be. It depends on
> - the nature of the code being run
> - the data it is processing (data loads tend to increase with the
> years)
> - the cleverness of the compiler
> - the data representation used by the Erlang system (e.g., whether
> pointers are 32 bits or 64, whether there are special tags for small
> arrays
> - the data representation used by library modules the code calls
> (change the representation of ordered sets and a bound that might
> have worked might not any more)
> - a system policy on whether it's better to be tight on memory and
> tolerate some processes crashing and needing to be restarted or
> whether it's best to keep more processing running at the cost of
> using more memory, a thing that might change at run time.
>
> I have long bemoaned the (measured!) fact that the size of a stack
> frame in C can vary by a factor of 10, so that determining the sizes
> for POSIX thread stacks is a game of Russian Roulette.
>
> You can determine suitable sizes for a particular release by running
> experiments, but the sizes thus determined are ONLY reliable for the
> specific release or releases you tried on the machine or machines
> that you tried it or them on. Set a limit, and you can *EXPECT*
> working processes to be crashed as a *NORMAL* outcome.
>
> Once some feature goes in, I expect to see messages in this mailing
> list "My program gets lots of killed processes due to Out-Of-Memory
> but it used to work and I haven't changed it."

I agree with You Richard

Most of our production java code throws millions of nullpointer exceptions messages
and management staff, and developers dont care, if users are happy.

I presume many subtle errors came from those nullpointer errors but people got used
to have those messages when the moved from other platforms to java and now is
the norm among java developers.

Its not acceptable for a platform that focuses on highly scalable, survivable and error tolerant
systems that programmers were acustommed to see a lot of memory errors.

Everyone here knows how to measure memory available at program life-time (im new but, i think i knows ), there is no need
to cover such poor memory management with artificial limits, this is no C and the like.

Virtual memory and Garbage collection should garantee enough memory provided the programmers know something about the algorithms
and data structures (good unit testing).

Imagine someone asks a question on the list, provides the offending code and everyother who is helping sees lot of memory errors
on his code..

Should i waste my time helping someone when his code spurts so much memory errors and hi/her doesnt care?

Should newbies like me be exposed to such poor desing and coding style?

What's better? counting bits properly or just set the limit and wait the process for a memory crash??

How do you manage diferent compilers code size, or 32/64bits overhead? IMHO its up to the programmer concise
memory usage expectation and this work has to be done on desing fase, and not let some black magic, i pressume
this wil not be portable across many erlang versions and setups...

Clearly for memory battles on process/threads and the like we have already many good languages like C
despite most people here seems to want the VM to became a better OS...

Classic behavior (let the VM crash) should be preserved for legacy systems and for academic and marketing stuff...

I agree, Shielding the VM againts many error condictions and you probably will generate as much poor programmers.

/Angel

PS: I agree with Joe, Erlang best feature is a new mental model of dealing with faulty conditions more than the "multicore" promises
> ________________________________________________________________
> erlang-questions mailing list. See http://www.erlang.org/faq.html
> erlang-questions (at) erlang.org
>
>

--
Este correo no tiene dibujos. Las formas extrañas en la pantalla son letras.
__________________________________________

Clist UAH a.k.a Angel
__________________________________________
HONEY BUNNY - Any of you *uckin' pricks move and I'll execute every mother*ucking last one of you.

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

Re: Why Beam.smp crashes when memory is over?

2009-11-09T20:36:44Z

On Nov 9, 2009, at 10:53 PM, Kenneth Lundin wrote:
> What we could do is to make it easier for the user to prevent OOM
> situations and also to
> let him take the decision when it occurs or rather before it occurs.
>
> One way would be to let the user set a memory quota on a process with
> options at spawn time. When the process reaches it quota it can be
> automatically killed or the user can
> be notified in some way and take actions.

One of the reasons this hasn't been done is, I presume, the fact that
it is quite difficult for a programmer to determine what the memory
quota should be. It depends on
- the nature of the code being run
- the data it is processing (data loads tend to increase with the
years)
- the cleverness of the compiler
- the data representation used by the Erlang system (e.g., whether
pointers are 32 bits or 64, whether there are special tags for small
arrays
- the data representation used by library modules the code calls
(change the representation of ordered sets and a bound that might
have worked might not any more)
- a system policy on whether it's better to be tight on memory and
tolerate some processes crashing and needing to be restarted or
whether it's best to keep more processing running at the cost of
using more memory, a thing that might change at run time.

I have long bemoaned the (measured!) fact that the size of a stack
frame in C can vary by a factor of 10, so that determining the sizes
for POSIX thread stacks is a game of Russian Roulette.

You can determine suitable sizes for a particular release by running
experiments, but the sizes thus determined are ONLY reliable for the
specific release or releases you tried on the machine or machines
that you tried it or them on. Set a limit, and you can *EXPECT*
working processes to be crashed as a *NORMAL* outcome.

Once some feature goes in, I expect to see messages in this mailing
list "My program gets lots of killed processes due to Out-Of-Memory
but it used to work and I haven't changed it."

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

Re: Mnesia stuck in transaction

2009-11-09T17:52:50Z

Not sure what actually happened. But I found the work-around.

changed the table type to ram_copies
Stopped mnesia
Started mnesia

Once gen_server message queue is drained, changed the table type of disc_copies

-- baliga
"Point of view is worth 80 IQ points" --Alan Kay

http://dudefrommangalore.blogspot.com/

________________________________
From: Yogish Baliga <yogishb@...>
To: Erlang Questions <erlang-questions@...>
Sent: Mon, November 9, 2009 5:15:52 PM
Subject: [erlang-questions] Mnesia stuck in transaction

Hello Mnesia experts,

I have a process hung with the following backtrace. What is the possible solution? This gen_server (workflow_manager_logs) is backed up with lot of messages delivered via RabbitMQ. Either I need a way to cancel this transaction or fix the database so that it will continue processing all messages.

Thanx,
-- baliga

(workflow_gateway@...)11> io:format("~s", [erlang:binary_to_list(element(2, erlang:process_info(whereis(workflow_manager_logs), backtrace)))]).
Program counter: 0xf0717490 (mnesia_tm:rec/2 + 28)
CP: 0x00000000 (invalid)
arity = 0

0xefa18404 Return addr 0xf071b7c0 (mnesia_tm:execute_outer/6 + 68)
y(0) #Ref<0.0.0.1407>
y(1) <0.104.0>

0xefa18410 Return addr 0xf07fe6e8 (mnesia:wrap_trans/6 + 20)
y(0) async
y(1) infinity
y(2) 1
y(3) []
y(4) #Fun<workflow_manager_logs.4.58301755>
y(5) mnesia_frag

0xefa1842c Return addr 0xf13b0448 (workflow_manager_logs:handle_info/2 + 312)

0xefa18430 Return addr 0xf7e13124 (gen_server:handle_msg/5 + 932)
y(0) []
y(1) Catch 0xf13b0448 (workflow_manager_logs:handle_info/2 + 312)
y(2) []
y(3) {<0.264.0>,<0.274.0>,<<33 bytes>>}

0xefa18444 Return addr 0xf7db1c6c (proc_lib:init_p_do_apply/3 + 28)
y(0) workflow_manager_logs
y(1) {<0.264.0>,<0.274.0>,<<33 bytes>>}
y(2) workflow_manager_logs
y(3) <0.38.0>
y(4) {{'basic.deliver',<<33 bytes>>,7,false,<<15 bytes>>,<<24 bytes>>},{content,60,none,<<29 bytes>>,[<<75 bytes>>]}}
y(5) Catch 0xf7e13124 (gen_server:handle_msg/5 + 932)

0xefa18460 Return addr 0x08236c00 (<terminate process normally>)
y(0) Catch 0xf7db1c7c (proc_lib:init_p_do_apply/3 + 44)

"Point of view is worth 80 IQ points" --Alan Kay

http://dudefrommangalore.blogspot.com/

Mnesia stuck in transaction

2009-11-09T17:15:52Z

Hello Mnesia experts,

I have a process hung with the following backtrace. What is the possible solution? This gen_server (workflow_manager_logs) is backed up with lot of messages delivered via RabbitMQ. Either I need a way to cancel this transaction or fix the database so that it will continue processing all messages.

Thanx,
-- baliga

(workflow_gateway@...)11> io:format("~s", [erlang:binary_to_list(element(2, erlang:process_info(whereis(workflow_manager_logs), backtrace)))]).
Program counter: 0xf0717490 (mnesia_tm:rec/2 + 28)
CP: 0x00000000 (invalid)
arity = 0

0xefa18404 Return addr 0xf071b7c0 (mnesia_tm:execute_outer/6 + 68)
y(0) #Ref<0.0.0.1407>
y(1) <0.104.0>

0xefa18410 Return addr 0xf07fe6e8 (mnesia:wrap_trans/6 + 20)
y(0) async
y(1) infinity
y(2) 1
y(3) []
y(4) #Fun<workflow_manager_logs.4.58301755>
y(5) mnesia_frag

0xefa1842c Return addr 0xf13b0448 (workflow_manager_logs:handle_info/2 + 312)

0xefa18430 Return addr 0xf7e13124 (gen_server:handle_msg/5 + 932)
y(0) []
y(1) Catch 0xf13b0448 (workflow_manager_logs:handle_info/2 + 312)
y(2) []
y(3) {<0.264.0>,<0.274.0>,<<33 bytes>>}

0xefa18444 Return addr 0xf7db1c6c (proc_lib:init_p_do_apply/3 + 28)
y(0) workflow_manager_logs
y(1) {<0.264.0>,<0.274.0>,<<33 bytes>>}
y(2) workflow_manager_logs
y(3) <0.38.0>
y(4) {{'basic.deliver',<<33 bytes>>,7,false,<<15 bytes>>,<<24 bytes>>},{content,60,none,<<29 bytes>>,[<<75 bytes>>]}}
y(5) Catch 0xf7e13124 (gen_server:handle_msg/5 + 932)

0xefa18460 Return addr 0x08236c00 (<terminate process normally>)
y(0) Catch 0xf7db1c7c (proc_lib:init_p_do_apply/3 + 44)

"Point of view is worth 80 IQ points" --Alan Kay

http://dudefrommangalore.blogspot.com/

Re: Dialyzer problem on OSX 10.6 (Erlang R13B02-1)

2009-11-09T14:50:04Z

Hi,

I have been misinforming the mailing list here (unintentionally, lack
of information is the reason).

The problem is not with aggressive optimisation or it depends how you
look at it as homebrew on Snow Leopard uses the new llvm compiler as
the default c compiler.
This in case of erlang did not go too well. If somebody has the urge
to get llvm compiled erlang working this is one issue which has to be
solved.

Tamas Nagy
Erlang Training & Consulting
http://www.erlang-consulting.com

On 9 Nov 2009, at 21:42, Tamas Nagy wrote:

> Hi everyone,
>
> The problem has been solved in homebrew. Aggressive optimisation for
> some reason doesn't go well with dialyzer.
> The error would probably have shown its ugly head in different
> applications as well. dialyzer was only the unlucky one to reliable
> trigger it.
>
> For all the gory details:
> http://github.com/mxcl/homebrew/issues/closed/#issue/120
>
> Tamas Nagy
> Erlang Training & Consulting
> http://www.erlang-consulting.com
>
> On 9 Nov 2009, at 11:50, Tino Breddin wrote:
>
>> Hi Tamas,
>>
>> Yes I used homebrew as well. That sounds strange, might it be that
>> those optimizations have a negative impact on the Erlang
>> performance on Mac OSX?
>>
>> Cheers,
>> Tino
>>
>> On Nov 9, 2009, at 11:35 AM, Tamas Nagy wrote:
>>
>>> Hi Tino,
>>>
>>> Are you using homebrew?
>>>
>>> So far it seems that homebrew applies some c compiler magic (e.g
>>> optimizations) when building erlang which somehow borks dialyzer
>>> (strange) as compiling from source with the same ./configure
>>> options homebrew uses results in an erlang installation which
>>> works like a charm.
>>>
>>> Regards,
>>> Tamas
>>> ----- "Tino Breddin" <tino.breddin@...> wrote:
>>>
>>>> Hey Tamas,
>>>>
>>>> Here's the output for the command run on my machine (Macbook Pro
>>>> Core
>>>>
>>>> Duo, running OSX 10.6.1 32-bit):
>>>>
>>>> dialyzer --build_plt -r /usr/local/Cellar/erlang/R13B02-1/lib/
>>>> erlang/
>>>>
>>>> lib/erts-5.7.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/lib/erlang/
>>>> lib/
>>>>
>>>> kernel-2.13.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/lib/erlang/
>>>> lib/
>>>>
>>>> stdlib-1.16.3/ebin/
>>>> Creating PLT /Users/tino/.dialyzer_plt ...
>>>> Unknown functions:
>>>> compile:file/2
>>>> compile:forms/2
>>>> compile:noenv_forms/2
>>>> compile:output_generated/1
>>>> crypto:des3_cbc_decrypt/5
>>>> crypto:start/0
>>>> done in 46m22.52s
>>>> done (passed successfully)
>>>>
>>>> Apparently it takes some time to complete but you'll get there.
>>>>
>>>> Cheers,
>>>> Tino
>>>>
>>>> On Nov 8, 2009, at 11:35 PM, Tamas Nagy wrote:
>>>>
>>>>> Hi Kostis,
>>>>>
>>>>> This is what all I get:
>>>>> $ dialyzer --build_plt -r /usr/local/Cellar/erlang/R13B02-1/lib/
>>>>> erlang/lib/erts-5.7.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/lib/
>>>>> erlang/lib/kernel-2.13.3/ebin/
>>>> /usr/local/Cellar/erlang/R13B02-1/lib/
>>>>> erlang/lib/stdlib-1.16.3/ebin/
>>>>> Creating PLT /Users/lestat/.dialyzer_plt ...
>>>>>
>>>>> I will try to rebuild erlang with disabling hipe to see if that
>>>> helps.
>>>>>
>>>>> I am happy to run some tests/traces you would like to see the
>>>>> results of to help.
>>>>>
>>>>> Regards,
>>>>> Tamas
>>>>>
>>>>> Tamas Nagy
>>>>> Erlang Training & Consulting
>>>>> http://www.erlang-consulting.com
>>>>>
>>>>> On 8 Nov 2009, at 19:52, Kostis Sagonas wrote:
>>>>>
>>>>>> Tamas Nagy wrote:
>>>>>>> Hi,
>>>>>>> Am I the only one who cannot build a plt under OSX 10.6 in
>>>>>>> R13B02-1?
>>>>>>> No matter what applications I specify for the initial plt it
>>>>>>> just
>>>>
>>>>>>> hangs eating processing power but not producing anything.
>>>>>>> Erlang was installed with homebrew, but that shouldn't matter.
>>>>>>> Or
>>>>
>>>>>>> maybe it does. (beam is compiled for 64 bit)
>>>>>>> Here is the command I used:
>>>>>>> dialyzer --build_plt -r /usr/local/Cellar/erlang/R13B02-1/lib/
>>>>>>> erlang/lib/erts-5.7.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/
>>>>>>> lib/
>>>>
>>>>>>> erlang/lib/kernel-2.13.3/ebin/ /usr/local/Cellar/erlang/
>>>>>>> R13B02-1/
>>>>
>>>>>>> lib/erlang/lib/stdlib-1.16.3/ebin/
>>>>>>> I gave waiting for it to finish after ~1 hour (on an old type
>>>>>>> white MacBook (MacBook2,1) with 2GB of RAM).
>>>>>>
>>>>>> None of the dialyzer developers has access to such a machine to
>>>>>> see
>>>>
>>>>>> what's happening. Perhaps somebody else in that list might
>>>>>> want to
>>>>
>>>>>> report whether this is reproducible or not.
>>>>>>
>>>>>> However, perhaps the following might help: when building the PLT,
>>>>
>>>>>> do you see a message:
>>>>>>
>>>>>> Compiling some key modules to native code...
>>>>>>
>>>>>> If so, can you disable hipe (64-bit Mac is not a platform we have
>>>>
>>>>>> ever tested due to no access to such a machine) to see whether
>>>>>> this
>>>>
>>>>>> may be the culprit?
>>>>>>
>>>>>> Kostis
>>>>>
>>>>>
>>>>> ________________________________________________________________
>>>>> erlang-questions mailing list. See http://www.erlang.org/faq.html
>>>>> erlang-questions (at) erlang.org
>>>>>
>>>
>>> --
>>> Tamas Nagy
>>> Erlang Training & Consulting
>>> http://www.erlang-consulting.com
>>>
>>> ________________________________________________________________
>>> erlang-questions mailing list. See http://www.erlang.org/faq.html
>>> erlang-questions (at) erlang.org
>>>
>>
>

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

Re: Dialyzer problem on OSX 10.6 (Erlang R13B02-1)

2009-11-09T13:42:29Z

Hi everyone,

The problem has been solved in homebrew. Aggressive optimisation for
some reason doesn't go well with dialyzer.
The error would probably have shown its ugly head in different
applications as well. dialyzer was only the unlucky one to reliable
trigger it.

For all the gory details:
http://github.com/mxcl/homebrew/issues/closed/#issue/120

Tamas Nagy
Erlang Training & Consulting
http://www.erlang-consulting.com

On 9 Nov 2009, at 11:50, Tino Breddin wrote:

> Hi Tamas,
>
> Yes I used homebrew as well. That sounds strange, might it be that
> those optimizations have a negative impact on the Erlang performance
> on Mac OSX?
>
> Cheers,
> Tino
>
> On Nov 9, 2009, at 11:35 AM, Tamas Nagy wrote:
>
>> Hi Tino,
>>
>> Are you using homebrew?
>>
>> So far it seems that homebrew applies some c compiler magic (e.g
>> optimizations) when building erlang which somehow borks dialyzer
>> (strange) as compiling from source with the same ./configure
>> options homebrew uses results in an erlang installation which works
>> like a charm.
>>
>> Regards,
>> Tamas
>> ----- "Tino Breddin" <tino.breddin@...> wrote:
>>
>>> Hey Tamas,
>>>
>>> Here's the output for the command run on my machine (Macbook Pro
>>> Core
>>>
>>> Duo, running OSX 10.6.1 32-bit):
>>>
>>> dialyzer --build_plt -r /usr/local/Cellar/erlang/R13B02-1/lib/
>>> erlang/
>>>
>>> lib/erts-5.7.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/lib/erlang/
>>> lib/
>>>
>>> kernel-2.13.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/lib/erlang/
>>> lib/
>>>
>>> stdlib-1.16.3/ebin/
>>> Creating PLT /Users/tino/.dialyzer_plt ...
>>> Unknown functions:
>>> compile:file/2
>>> compile:forms/2
>>> compile:noenv_forms/2
>>> compile:output_generated/1
>>> crypto:des3_cbc_decrypt/5
>>> crypto:start/0
>>> done in 46m22.52s
>>> done (passed successfully)
>>>
>>> Apparently it takes some time to complete but you'll get there.
>>>
>>> Cheers,
>>> Tino
>>>
>>> On Nov 8, 2009, at 11:35 PM, Tamas Nagy wrote:
>>>
>>>> Hi Kostis,
>>>>
>>>> This is what all I get:
>>>> $ dialyzer --build_plt -r /usr/local/Cellar/erlang/R13B02-1/lib/
>>>> erlang/lib/erts-5.7.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/lib/
>>>> erlang/lib/kernel-2.13.3/ebin/
>>> /usr/local/Cellar/erlang/R13B02-1/lib/
>>>> erlang/lib/stdlib-1.16.3/ebin/
>>>> Creating PLT /Users/lestat/.dialyzer_plt ...
>>>>
>>>> I will try to rebuild erlang with disabling hipe to see if that
>>> helps.
>>>>
>>>> I am happy to run some tests/traces you would like to see the
>>>> results of to help.
>>>>
>>>> Regards,
>>>> Tamas
>>>>
>>>> Tamas Nagy
>>>> Erlang Training & Consulting
>>>> http://www.erlang-consulting.com
>>>>
>>>> On 8 Nov 2009, at 19:52, Kostis Sagonas wrote:
>>>>
>>>>> Tamas Nagy wrote:
>>>>>> Hi,
>>>>>> Am I the only one who cannot build a plt under OSX 10.6 in
>>>>>> R13B02-1?
>>>>>> No matter what applications I specify for the initial plt it just
>>>
>>>>>> hangs eating processing power but not producing anything.
>>>>>> Erlang was installed with homebrew, but that shouldn't matter. Or
>>>
>>>>>> maybe it does. (beam is compiled for 64 bit)
>>>>>> Here is the command I used:
>>>>>> dialyzer --build_plt -r /usr/local/Cellar/erlang/R13B02-1/lib/
>>>>>> erlang/lib/erts-5.7.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/
>>>>>> lib/
>>>
>>>>>> erlang/lib/kernel-2.13.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/
>>>
>>>>>> lib/erlang/lib/stdlib-1.16.3/ebin/
>>>>>> I gave waiting for it to finish after ~1 hour (on an old type
>>>>>> white MacBook (MacBook2,1) with 2GB of RAM).
>>>>>
>>>>> None of the dialyzer developers has access to such a machine to
>>>>> see
>>>
>>>>> what's happening. Perhaps somebody else in that list might want
>>>>> to
>>>
>>>>> report whether this is reproducible or not.
>>>>>
>>>>> However, perhaps the following might help: when building the PLT,
>>>
>>>>> do you see a message:
>>>>>
>>>>> Compiling some key modules to native code...
>>>>>
>>>>> If so, can you disable hipe (64-bit Mac is not a platform we have
>>>
>>>>> ever tested due to no access to such a machine) to see whether
>>>>> this
>>>
>>>>> may be the culprit?
>>>>>
>>>>> Kostis
>>>>
>>>>
>>>> ________________________________________________________________
>>>> erlang-questions mailing list. See http://www.erlang.org/faq.html
>>>> erlang-questions (at) erlang.org
>>>>
>>
>> --
>> Tamas Nagy
>> Erlang Training & Consulting
>> http://www.erlang-consulting.com
>>
>> ________________________________________________________________
>> erlang-questions mailing list. See http://www.erlang.org/faq.html
>> erlang-questions (at) erlang.org
>>
>

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

Re: Re: C Node and {'EXIT', Pid, noconnection} in Erlang

2009-11-09T12:31:14Z

Hi Max,

I am doing some investigations into ports and nodes and ran across your
thread here which nobody seems to have answered, so I will venture a guess.

noconnection means that there is no connection to the node and since you are
linking (in the original code) this will cause an EXIT signal with reason
noconnection.

So your ObjC node is probably not running in the eyes of the Erlang node.
Why? Hard for me to see, but this is how I would interpret the error
messages, perhaps others can shed some more light on what could cause this
problem.

Cheers,
Torben

On Tue, Oct 6, 2009 at 10:43, Max Lapshin <max.lapshin@...> wrote:

> Problem disappeared when I remove link(NetStream) from erlang code.
> It is strange for me, because no LINK messages haven't been sent to ObjC
> node
>
> ________________________________________________________________
> erlang-questions mailing list. See http://www.erlang.org/faq.html
> erlang-questions (at) erlang.org
>
>

--
http://www.linkedin.com/in/torbenhoffmann

Re: Why Beam.smp crashes when memory is over?

2009-11-09T09:42:39Z

The problem is that there is no way for the VM to really know which
process is leaking. The Linux OOM-killer has gone through a bunch of
rewrites and it still doesn't always get it right.

It might be possible to have a threshold trigger that would hit some
sort of global alarm that could be application specific. That's
probably the closest you could get though.

On Nov 8, 2009, at 9:08 PM, Max Lapshin wrote:

> Look, I have a program, that reads MPEG TS stream from network. There
> is some bug in my code
> and my ts_lander crashes beam. How is it possible to store state of
> external HTTP stream??
>
> If Erlang VM wouldn't crash, but only kill the leaking process, I
> could buffer stream in other process and feed
> new mpeg ts lander after restart. But it is impossible =(
>
> ________________________________________________________________
> erlang-questions mailing list. See http://www.erlang.org/faq.html
> erlang-questions (at) erlang.org
>

--
Jayson Vantuyl
kagato@...

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

Re: Why Beam.smp crashes when memory is over?

2009-11-09T09:40:55Z

You could run two Erlang VMs (thus, keeping the HTTP connection and
its state separate). Then the crasher wouldn't take the other one
down. One with the processing and one with the networking, and have
them communicate via distributed Erlang. You should still ulimit beam
to keep the other one from getting OOM-killed though.

Assuming you can get the throughput you need, this also would
decompose some of the components, which might helps scale horizontally
later.

On Nov 8, 2009, at 9:08 PM, Max Lapshin wrote:

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

Re: AW: Why Beam.smp crashes when memory is over?

2009-11-09T09:38:59Z

This would be a very useful tunable.

On Nov 9, 2009, at 2:31 AM, Decker, Nils wrote:

> Hello,
>
> I had a few occasions during developing that a runaway process
> caused my machine to swap and gind to a halt. (i should have used
> ulimit on beam)
>
> I wondered why there is no way to limit the size of a single
> process. It could be a simple option to spawn to limit the heap size
> of a process. (like ulimit)
> The process gets killed if it ever grows beyond the limit. A limit
> for the size of the message queue would be nice too, because a
> process with a few thousand entries is (most of the time) not going
> to cope with it anyway.
> A similar limit for ETS tables would be possible too. A memory limit
> for shared resources (large binaries) would be more difficult.
>
> Nils Decker

--
Jayson Vantuyl
kagato@...

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

Re: Why Beam.smp crashes when memory is over?

2009-11-09T08:21:41Z

El Lunes, 9 de Noviembre de 2009 Tony Rogvall escribió:

> Hi!
>
> On 9 nov 2009, at 14.54, Angel Alvarez wrote:
>
> > Well still there are many issues with this new approach
> >
> Yes! But it does not scare me ;-)
>
> > Where are the maibox of processes located?
> >
> > With a heap pre process...
>
> Depends on the implementation. But in general you could do something
> like, If the the data is shared then
> you split the share (memory_size/ref_count). If the data is copied
> then you must count it in.
>
> >
> > Couldnt you trigger a memory exception on a remote process by just
> > sending one message
> > when the process is almost consuming its reserved memory?
> >
> Yes. But that is the point. If it pass the limit the process will die.
> There are many special cases where you could think of using the memory
> in a better and more optimal way.
> Lets say you are reaching the memory limit you may switch to a
> compression algorithm for heap memory !?
> But lets keep it simple for the prototype, and see if it useful.
>
>
> > Systems other than embedded erlangs deploy ( form de current erlang
> > movement as a server/desktop plataform)
> > will suffer from resource contention beetween erlang VM and other OS
> > processes.
> >
> > Port programs also need system resources...
> >
> For loadable drivers using driver_alloc, one could possibly do
> something, otherwise it will be
> up to the driver designer to handle it. There is a max_ports in the
> prototype that limits number of
> open_ports. If sockets/files are mapped to single ports then it may
> help a bit.
>
>
> > Well in the end your approach is still very interesting as a
> > framework for continous erlang VM innovations...
> >
> Thanks.
>
> > but please correct me if im wrong but I saw that memory carriers
> > allowed to set several options on erlang VM start-up so,
> >
> I am not sure what you mean here?

Well some bits can be controlled as you show on spawn_opt. In the other hand i mean that VM Safe memory
as stated by others (Andrew, Tom ...) should be controlled on memory carriers: ( erts_alloc(3) )

In esence

- A new emergency memory alocator in the alloc_util framework just in case the VM need memory to recover from a OOM.

or perhaps

<WARNING speculative mode ON>

- Using the segment allocator on mmap supported arquitectures to allow fast recovery for the full erlang vm (sort of checkpointing)
using a special BIF you could instruct memory carriers to checkpoint the entire VM usining this allocator just in case the VM crashes.
so on next run the enrire VM can be recovered parsing the last checkpint

</WARNING>

Still i think is better and more clean having two o more instances that bulletprooffing the VM

/Angel

>
> /Tony
>
> > is stil posible to pacth those carriers to allow a safe memory
> > reservation to let de VM manage properly a memory full
> > condition by killing the offending process (sort of a OOM killer for
> > the VM)?
> >
> > Just telling the VM not to "kill system process" and let the
> > supervisors do the work...
> >
>
>
>
> > /Angel
> >
> >
> >
> > El Lunes, 9 de Noviembre de 2009 Tony Rogvall escribió:
> >>
> >> It is still the same "let it crash" concept using the resource limit
> >> system I am designing.
> >> But you can limit the crash in a more controlled way. Also you will
> >> be
> >> able to report
> >> interesting information about what is crashing and when.
> >>
> >> There is sometimes an issue when big systems crash. The restart may
> >> take a lot of time.
> >> Nodes must be synchronised, database tables must be repaired etc etc.
> >> I guess you can design this to be light and easy, but it is not
> >> always
> >> the case.
> >>
> >> /Tony
> >>
> >>
> >>
> >> On 9 nov 2009, at 12.43, Angel Alvarez wrote:
> >>
> >>>
> >>> Well please let me say something
> >>>
> >>> I'm plain new but some things are pretty clear for me.
> >>>
> >>>
> >>> The beauty of the erlang concept is "let it crash" , "don't program
> >>> defensively"
> >>> so the VM and the underlaying hardware are entities that can fail,
> >>> that's it.
> >>>
> >>> What's the problem so?
> >>>
> >>> Joe said...
> >>>
> >>> If you want failure tolerance you need at least two nodes...
> >>>
> >>> From J.A thesis
> >>> " ...Schneider [60, 59] answered this question by giving three
> >>> properties
> >>> that he thought a hardware system should have in order to be
> >>> suitable for
> >>> programming a fault-tolerant system. These properties Schneider
> >>> called:..."
> >>>
> >>> 1. Halt on failure — in the event of an error a processor should
> >>> halt
> >>> instead of performing a possibly erroneous operation.2
> >>>
> >>> So on memory exhaustation the VM has to die and other node (erlang)
> >>> will do the recovery.
> >>>
> >>> that's the distrirbution role, no only to span computations over
> >>> several nodes to enhance performance
> >>> but to provide resilence in the presence of fatal errors (non
> >>> correctable).
> >>>
> >>> As a OS process the VM has to compete with other OS processes so in
> >>> a shared deployment (a VM running
> >>> on a server or a desktop) you cant be safe agaisnt a OOM trigered by
> >>> other entities.
> >>>
> >>> Such resource control thing wil only augment process overhead and
> >>> context switching in the VM.
> >>>
> >>> People new to erlang will be atracted to this hierarquical
> >>> decomposition of tasks as joe stated in his thesis
> >>> "If you cant run certaing task try doing something simpler"
> >>>
> >>> Many languages and VM's are incorporating erlang's good "multicore"
> >>> features but not the erlang powerfull error handling concept
> >>> and you guys want to kill the simpliticy incorporating many
> >>> defensive capabilities to avoid fatality instead of just organize
> >>> code to
> >>> handle such fatality.
> >>>
> >>> ¿whats next?, ¿A mailbox maximum message queue control?
> >>>
> >>>
> >>> Well, that's all i have to say about that, Forrest Gump.
> >>>
> >>>
> >>> El Lunes, 9 de Noviembre de 2009 09:45:10 Tony Rogvall escribió:
> >>>> Interesting discussion!
> >>>>
> >>>> I have been working on a resource system for Erlang for nearly two
> >>>> years now.
> >>>> I have a working (tm) prototype where you can set resource limits
> >>>> like
> >>>> max_processes/max_ports/max_memory/max_time/max_reductions ...
> >>>> The limits are passed with spawn_opt and are inherited by the
> >>>> processes spawned.
> >>>> This means that if you spawn_opt(M,F,A[{max_memory, 1024*1024}])
> >>>> the
> >>>> process
> >>>> will be able to use 1M words for it self and it's "subprocesses".
> >>>> This
> >>>> also means
> >>>> that the spawner will get 1M less to use (as designed right now).
> >>>> If a
> >>>> resource limit
> >>>> is reached the process crash with system_limt reason.
> >>>>
> >>>> There are still some details to work out before a release, but I
> >>>> will
> >>>> try to get it ready before
> >>>> end of this year.
> >>>>
> >>>> /Tony
> >>>>
> >>>>
> >>>>
> >>>> On 9 nov 2009, at 09.16, Robert Virding wrote:
> >>>>
> >>>>> No.
> >>>>>
> >>>>> There is a major difference between handling OOM in an OS and in
> >>>>> the
> >>>>> BEAM.
> >>>>> In an OS it usually at a per process level that memory runs out so
> >>>>> it is
> >>>>> easy to decide which process to kill so that the OS can
> >>>>> continue. In
> >>>>> the
> >>>>> BEAM, however, it is the VM as a whole which has run out of memory
> >>>>> not a
> >>>>> specific, it is. therefore, much more difficult to work out which
> >>>>> process is
> >>>>> the culprit and to decide what to do. For example it might be that
> >>>>> the
> >>>>> process which causes the OOM is not the actual problem process, it
> >>>>> might
> >>>>> just the last straw. Or the actual cause may that the whole app
> >>>>> might be
> >>>>> generating large binaries too quickly. Or it might be that the
> >>>>> whole
> >>>>> app is
> >>>>> spawning to many processes without any one process being the
> >>>>> cause.
> >>>>> Or ...
> >>>>> In all these cases killing the process which triggered the OOM
> >>>>> would
> >>>>> be the
> >>>>> Wrong Thing. We found that it was difficult to work out a
> >>>>> reasonable
> >>>>> strategy to handle the actual cause so we decided not handle it.
> >>>>>
> >>>>> "Don't catch an error which you can't handle" as the bard put it.
> >>>>>
> >>>>> Robert
> >>>>>
> >>>>> 2009/11/9 Max Lapshin <max.lapshin@...>
> >>>>>
> >>>>>> Yes, there are techniques to write watchdogs, but my question
> >>>>>> was: is
> >>>>>> it possible to prevent Erlang VM from crash?
> >>>>>>
> >>>>>> ________________________________________________________________
> >>>>>> erlang-questions mailing list. See http://www.erlang.org/faq.html
> >>>>>> erlang-questions (at) erlang.org
> >>>>>>
> >>>>>>
> >>>>
> >>>>
> >>>> ________________________________________________________________
> >>>> erlang-questions mailing list. See http://www.erlang.org/faq.html
> >>>> erlang-questions (at) erlang.org
> >>>>
> >>>>
> >>>
> >>>
> >>> Este correo no tiene dibujos. Las formas extrañas en la pantalla son
> >>> letras.
> >>> __________________________________________
> >>>
> >>> Clist UAH a.k.a Angel
> >>> __________________________________________
> >>> Warning: Microsoft_bribery.ISO contains OOXML code.
> >>>
> >>> ________________________________________________________________
> >>> erlang-questions mailing list. See http://www.erlang.org/faq.html
> >>> erlang-questions (at) erlang.org
> >>>
> >>
> >>
> >
> >
> >
> > --
> > No imprima este correo si no es necesario. El medio ambiente está en
> > nuestras manos.
> > __________________________________________
> >
> > Clist UAH a.k.a Angel
> > __________________________________________
> > China 'limpia' el Tibet para las Olimpiadas.
>
>

--
Este correo no tiene dibujos. Las formas extrañas en la pantalla son letras.
__________________________________________

Clist UAH a.k.a Angel
__________________________________________
Artista -- (internet) --> Usuario final. Así los artistas cobran más y dicen menos paridas sobre lo que creen que es la piratería.

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

Re: Why Beam.smp crashes when memory is over?

2009-11-09T06:38:02Z

Hi!

On 9 nov 2009, at 14.54, Angel Alvarez wrote:

> Well still there are many issues with this new approach
>
Yes! But it does not scare me ;-)

> Where are the maibox of processes located?
>
> With a heap pre process...

Depends on the implementation. But in general you could do something
like, If the the data is shared then
you split the share (memory_size/ref_count). If the data is copied
then you must count it in.

>
> Couldnt you trigger a memory exception on a remote process by just
> sending one message
> when the process is almost consuming its reserved memory?
>
Yes. But that is the point. If it pass the limit the process will die.
There are many special cases where you could think of using the memory
in a better and more optimal way.
Lets say you are reaching the memory limit you may switch to a
compression algorithm for heap memory !?
But lets keep it simple for the prototype, and see if it useful.

> Systems other than embedded erlangs deploy ( form de current erlang
> movement as a server/desktop plataform)
> will suffer from resource contention beetween erlang VM and other OS
> processes.
>
> Port programs also need system resources...
>
For loadable drivers using driver_alloc, one could possibly do
something, otherwise it will be
up to the driver designer to handle it. There is a max_ports in the
prototype that limits number of
open_ports. If sockets/files are mapped to single ports then it may
help a bit.

> Well in the end your approach is still very interesting as a
> framework for continous erlang VM innovations...
>
Thanks.

> but please correct me if im wrong but I saw that memory carriers
> allowed to set several options on erlang VM start-up so,
>
I am not sure what you mean here?

/Tony

> is stil posible to pacth those carriers to allow a safe memory
> reservation to let de VM manage properly a memory full
> condition by killing the offending process (sort of a OOM killer for
> the VM)?
>
> Just telling the VM not to "kill system process" and let the
> supervisors do the work...
>

> /Angel
>
>
>
> El Lunes, 9 de Noviembre de 2009 Tony Rogvall escribió:
>>
>> It is still the same "let it crash" concept using the resource limit
>> system I am designing.
>> But you can limit the crash in a more controlled way. Also you will
>> be
>> able to report
>> interesting information about what is crashing and when.
>>
>> There is sometimes an issue when big systems crash. The restart may
>> take a lot of time.
>> Nodes must be synchronised, database tables must be repaired etc etc.
>> I guess you can design this to be light and easy, but it is not
>> always
>> the case.
>>
>> /Tony
>>
>>
>>
>> On 9 nov 2009, at 12.43, Angel Alvarez wrote:
>>
>>>
>>> Well please let me say something
>>>
>>> I'm plain new but some things are pretty clear for me.
>>>
>>>
>>> The beauty of the erlang concept is "let it crash" , "don't program
>>> defensively"
>>> so the VM and the underlaying hardware are entities that can fail,
>>> that's it.
>>>
>>> What's the problem so?
>>>
>>> Joe said...
>>>
>>> If you want failure tolerance you need at least two nodes...
>>>
>>> From J.A thesis
>>> " ...Schneider [60, 59] answered this question by giving three
>>> properties
>>> that he thought a hardware system should have in order to be
>>> suitable for
>>> programming a fault-tolerant system. These properties Schneider
>>> called:..."
>>>
>>> 1. Halt on failure — in the event of an error a processor should
>>> halt
>>> instead of performing a possibly erroneous operation.2
>>>
>>> So on memory exhaustation the VM has to die and other node (erlang)
>>> will do the recovery.
>>>
>>> that's the distrirbution role, no only to span computations over
>>> several nodes to enhance performance
>>> but to provide resilence in the presence of fatal errors (non
>>> correctable).
>>>
>>> As a OS process the VM has to compete with other OS processes so in
>>> a shared deployment (a VM running
>>> on a server or a desktop) you cant be safe agaisnt a OOM trigered by
>>> other entities.
>>>
>>> Such resource control thing wil only augment process overhead and
>>> context switching in the VM.
>>>
>>> People new to erlang will be atracted to this hierarquical
>>> decomposition of tasks as joe stated in his thesis
>>> "If you cant run certaing task try doing something simpler"
>>>
>>> Many languages and VM's are incorporating erlang's good "multicore"
>>> features but not the erlang powerfull error handling concept
>>> and you guys want to kill the simpliticy incorporating many
>>> defensive capabilities to avoid fatality instead of just organize
>>> code to
>>> handle such fatality.
>>>
>>> ¿whats next?, ¿A mailbox maximum message queue control?
>>>
>>>
>>> Well, that's all i have to say about that, Forrest Gump.
>>>
>>>
>>> El Lunes, 9 de Noviembre de 2009 09:45:10 Tony Rogvall escribió:
>>>> Interesting discussion!
>>>>
>>>> I have been working on a resource system for Erlang for nearly two
>>>> years now.
>>>> I have a working (tm) prototype where you can set resource limits
>>>> like
>>>> max_processes/max_ports/max_memory/max_time/max_reductions ...
>>>> The limits are passed with spawn_opt and are inherited by the
>>>> processes spawned.
>>>> This means that if you spawn_opt(M,F,A[{max_memory, 1024*1024}])
>>>> the
>>>> process
>>>> will be able to use 1M words for it self and it's "subprocesses".
>>>> This
>>>> also means
>>>> that the spawner will get 1M less to use (as designed right now).
>>>> If a
>>>> resource limit
>>>> is reached the process crash with system_limt reason.
>>>>
>>>> There are still some details to work out before a release, but I
>>>> will
>>>> try to get it ready before
>>>> end of this year.
>>>>
>>>> /Tony
>>>>
>>>>
>>>>
>>>> On 9 nov 2009, at 09.16, Robert Virding wrote:
>>>>
>>>>> No.
>>>>>
>>>>> There is a major difference between handling OOM in an OS and in
>>>>> the
>>>>> BEAM.
>>>>> In an OS it usually at a per process level that memory runs out so
>>>>> it is
>>>>> easy to decide which process to kill so that the OS can
>>>>> continue. In
>>>>> the
>>>>> BEAM, however, it is the VM as a whole which has run out of memory
>>>>> not a
>>>>> specific, it is. therefore, much more difficult to work out which
>>>>> process is
>>>>> the culprit and to decide what to do. For example it might be that
>>>>> the
>>>>> process which causes the OOM is not the actual problem process, it
>>>>> might
>>>>> just the last straw. Or the actual cause may that the whole app
>>>>> might be
>>>>> generating large binaries too quickly. Or it might be that the
>>>>> whole
>>>>> app is
>>>>> spawning to many processes without any one process being the
>>>>> cause.
>>>>> Or ...
>>>>> In all these cases killing the process which triggered the OOM
>>>>> would
>>>>> be the
>>>>> Wrong Thing. We found that it was difficult to work out a
>>>>> reasonable
>>>>> strategy to handle the actual cause so we decided not handle it.
>>>>>
>>>>> "Don't catch an error which you can't handle" as the bard put it.
>>>>>
>>>>> Robert
>>>>>
>>>>> 2009/11/9 Max Lapshin <max.lapshin@...>
>>>>>
>>>>>> Yes, there are techniques to write watchdogs, but my question
>>>>>> was: is
>>>>>> it possible to prevent Erlang VM from crash?
>>>>>>
>>>>>> ________________________________________________________________
>>>>>> erlang-questions mailing list. See http://www.erlang.org/faq.html
>>>>>> erlang-questions (at) erlang.org
>>>>>>
>>>>>>
>>>>
>>>>
>>>> ________________________________________________________________
>>>> erlang-questions mailing list. See http://www.erlang.org/faq.html
>>>> erlang-questions (at) erlang.org
>>>>
>>>>
>>>
>>>
>>> Este correo no tiene dibujos. Las formas extrañas en la pantalla son
>>> letras.
>>> __________________________________________
>>>
>>> Clist UAH a.k.a Angel
>>> __________________________________________
>>> Warning: Microsoft_bribery.ISO contains OOXML code.
>>>
>>> ________________________________________________________________
>>> erlang-questions mailing list. See http://www.erlang.org/faq.html
>>> erlang-questions (at) erlang.org
>>>
>>
>>
>
>
>
> --
> No imprima este correo si no es necesario. El medio ambiente está en
> nuestras manos.
> __________________________________________
>
> Clist UAH a.k.a Angel
> __________________________________________
> China 'limpia' el Tibet para las Olimpiadas.

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

Re: Why Beam.smp crashes when memory is over?

2009-11-09T05:54:10Z

Well still there are many issues with this new approach

Where are the maibox of processes located?

With a heap pre process...

Couldnt you trigger a memory exception on a remote process by just sending one message
when the process is almost consuming its reserved memory?

Systems other than embedded erlangs deploy ( form de current erlang movement as a server/desktop plataform)
will suffer from resource contention beetween erlang VM and other OS processes.

Port programs also need system resources...

Well in the end your approach is still very interesting as a framework for continous erlang VM innovations...

but please correct me if im wrong but I saw that memory carriers allowed to set several options on erlang VM start-up so,

is stil posible to pacth those carriers to allow a safe memory reservation to let de VM manage properly a memory full
condition by killing the offending process (sort of a OOM killer for the VM)?

Just telling the VM not to "kill system process" and let the supervisors do the work...

/Angel

El Lunes, 9 de Noviembre de 2009 Tony Rogvall escribió:

>
> It is still the same "let it crash" concept using the resource limit
> system I am designing.
> But you can limit the crash in a more controlled way. Also you will be
> able to report
> interesting information about what is crashing and when.
>
> There is sometimes an issue when big systems crash. The restart may
> take a lot of time.
> Nodes must be synchronised, database tables must be repaired etc etc.
> I guess you can design this to be light and easy, but it is not always
> the case.
>
> /Tony
>
>
>
> On 9 nov 2009, at 12.43, Angel Alvarez wrote:
>
> >
> > Well please let me say something
> >
> > I'm plain new but some things are pretty clear for me.
> >
> >
> > The beauty of the erlang concept is "let it crash" , "don't program
> > defensively"
> > so the VM and the underlaying hardware are entities that can fail,
> > that's it.
> >
> > What's the problem so?
> >
> > Joe said...
> >
> > If you want failure tolerance you need at least two nodes...
> >
> > From J.A thesis
> > " ...Schneider [60, 59] answered this question by giving three
> > properties
> > that he thought a hardware system should have in order to be
> > suitable for
> > programming a fault-tolerant system. These properties Schneider
> > called:..."
> >
> > 1. Halt on failure — in the event of an error a processor should halt
> > instead of performing a possibly erroneous operation.2
> >
> > So on memory exhaustation the VM has to die and other node (erlang)
> > will do the recovery.
> >
> > that's the distrirbution role, no only to span computations over
> > several nodes to enhance performance
> > but to provide resilence in the presence of fatal errors (non
> > correctable).
> >
> > As a OS process the VM has to compete with other OS processes so in
> > a shared deployment (a VM running
> > on a server or a desktop) you cant be safe agaisnt a OOM trigered by
> > other entities.
> >
> > Such resource control thing wil only augment process overhead and
> > context switching in the VM.
> >
> > People new to erlang will be atracted to this hierarquical
> > decomposition of tasks as joe stated in his thesis
> > "If you cant run certaing task try doing something simpler"
> >
> > Many languages and VM's are incorporating erlang's good "multicore"
> > features but not the erlang powerfull error handling concept
> > and you guys want to kill the simpliticy incorporating many
> > defensive capabilities to avoid fatality instead of just organize
> > code to
> > handle such fatality.
> >
> > ¿whats next?, ¿A mailbox maximum message queue control?
> >
> >
> > Well, that's all i have to say about that, Forrest Gump.
> >
> >
> > El Lunes, 9 de Noviembre de 2009 09:45:10 Tony Rogvall escribió:
> >> Interesting discussion!
> >>
> >> I have been working on a resource system for Erlang for nearly two
> >> years now.
> >> I have a working (tm) prototype where you can set resource limits
> >> like
> >> max_processes/max_ports/max_memory/max_time/max_reductions ...
> >> The limits are passed with spawn_opt and are inherited by the
> >> processes spawned.
> >> This means that if you spawn_opt(M,F,A[{max_memory, 1024*1024}]) the
> >> process
> >> will be able to use 1M words for it self and it's "subprocesses".
> >> This
> >> also means
> >> that the spawner will get 1M less to use (as designed right now).
> >> If a
> >> resource limit
> >> is reached the process crash with system_limt reason.
> >>
> >> There are still some details to work out before a release, but I will
> >> try to get it ready before
> >> end of this year.
> >>
> >> /Tony
> >>
> >>
> >>
> >> On 9 nov 2009, at 09.16, Robert Virding wrote:
> >>
> >>> No.
> >>>
> >>> There is a major difference between handling OOM in an OS and in the
> >>> BEAM.
> >>> In an OS it usually at a per process level that memory runs out so
> >>> it is
> >>> easy to decide which process to kill so that the OS can continue. In
> >>> the
> >>> BEAM, however, it is the VM as a whole which has run out of memory
> >>> not a
> >>> specific, it is. therefore, much more difficult to work out which
> >>> process is
> >>> the culprit and to decide what to do. For example it might be that
> >>> the
> >>> process which causes the OOM is not the actual problem process, it
> >>> might
> >>> just the last straw. Or the actual cause may that the whole app
> >>> might be
> >>> generating large binaries too quickly. Or it might be that the whole
> >>> app is
> >>> spawning to many processes without any one process being the cause.
> >>> Or ...
> >>> In all these cases killing the process which triggered the OOM would
> >>> be the
> >>> Wrong Thing. We found that it was difficult to work out a reasonable
> >>> strategy to handle the actual cause so we decided not handle it.
> >>>
> >>> "Don't catch an error which you can't handle" as the bard put it.
> >>>
> >>> Robert
> >>>
> >>> 2009/11/9 Max Lapshin <max.lapshin@...>
> >>>
> >>>> Yes, there are techniques to write watchdogs, but my question
> >>>> was: is
> >>>> it possible to prevent Erlang VM from crash?
> >>>>
> >>>> ________________________________________________________________
> >>>> erlang-questions mailing list. See http://www.erlang.org/faq.html
> >>>> erlang-questions (at) erlang.org
> >>>>
> >>>>
> >>
> >>
> >> ________________________________________________________________
> >> erlang-questions mailing list. See http://www.erlang.org/faq.html
> >> erlang-questions (at) erlang.org
> >>
> >>
> >
> >
> > Este correo no tiene dibujos. Las formas extrañas en la pantalla son
> > letras.
> > __________________________________________
> >
> > Clist UAH a.k.a Angel
> > __________________________________________
> > Warning: Microsoft_bribery.ISO contains OOXML code.
> >
> > ________________________________________________________________
> > erlang-questions mailing list. See http://www.erlang.org/faq.html
> > erlang-questions (at) erlang.org
> >
>
>

--
No imprima este correo si no es necesario. El medio ambiente está en nuestras manos.
__________________________________________

Clist UAH a.k.a Angel
__________________________________________
China 'limpia' el Tibet para las Olimpiadas.

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

Re: Why Beam.smp crashes when memory is over?

2009-11-09T05:25:09Z

----- Original Message ----

> From: Andrew Thompson <andrew@...>
> ... it'd be nice to have an
> alternative in cases where nuking the VM and having to recover or simply
> discard a lot of state is unpleasant.
>
> In an ideal world I'd like to see an *optional* OOM behaviour similar to
> the following:
>
> On startup, the VM pre-allocates enough memory to be able to attempt OOM
> recovery without additional malloc calls. If and when an OOM condition
> is detected, freeze the VM, iterate the process list and simply
> exit(FatPid, oom) the process with the most memory allocated (obviously
> this strategy is full of holes, epecially with regard to off-heap
> binaries), but *sometimes* it's better than just blowing away the whole
> thing. Then, if the malloc fails again, revert to the current behaviour.

Another way of doing that might be to hand the decision to (a modified version of) the supervisor hierarchy (if any). The supervisor could get info on sibling priority (if any) and/or memory usage as part of the request perhaps. Restart up the supervisor hierarchy to suit. That might give more structure to the restarting.

Or you could do the OOM-killing in distributed erlang + unix instead. Run the buggy, memory-hungry stuff in a separate VM with a memory limit on the process, and manage the work from a "supervisor VM" using monitoring and rpc. Not quite as elegant, of course.

Best,
Thomas

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

Re: Why Beam.smp crashes when memory is over?

2009-11-09T05:11:34Z

It is still the same "let it crash" concept using the resource limit
system I am designing.
But you can limit the crash in a more controlled way. Also you will be
able to report
interesting information about what is crashing and when.

There is sometimes an issue when big systems crash. The restart may
take a lot of time.
Nodes must be synchronised, database tables must be repaired etc etc.
I guess you can design this to be light and easy, but it is not always
the case.

/Tony

On 9 nov 2009, at 12.43, Angel Alvarez wrote:

>
> Well please let me say something
>
> I'm plain new but some things are pretty clear for me.
>
>
> The beauty of the erlang concept is "let it crash" , "don't program
> defensively"
> so the VM and the underlaying hardware are entities that can fail,
> that's it.
>
> What's the problem so?
>
> Joe said...
>
> If you want failure tolerance you need at least two nodes...
>
> From J.A thesis
> " ...Schneider [60, 59] answered this question by giving three
> properties
> that he thought a hardware system should have in order to be
> suitable for
> programming a fault-tolerant system. These properties Schneider
> called:..."
>
> 1. Halt on failure — in the event of an error a processor should halt
> instead of performing a possibly erroneous operation.2
>
> So on memory exhaustation the VM has to die and other node (erlang)
> will do the recovery.
>
> that's the distrirbution role, no only to span computations over
> several nodes to enhance performance
> but to provide resilence in the presence of fatal errors (non
> correctable).
>
> As a OS process the VM has to compete with other OS processes so in
> a shared deployment (a VM running
> on a server or a desktop) you cant be safe agaisnt a OOM trigered by
> other entities.
>
> Such resource control thing wil only augment process overhead and
> context switching in the VM.
>
> People new to erlang will be atracted to this hierarquical
> decomposition of tasks as joe stated in his thesis
> "If you cant run certaing task try doing something simpler"
>
> Many languages and VM's are incorporating erlang's good "multicore"
> features but not the erlang powerfull error handling concept
> and you guys want to kill the simpliticy incorporating many
> defensive capabilities to avoid fatality instead of just organize
> code to
> handle such fatality.
>
> ¿whats next?, ¿A mailbox maximum message queue control?
>
>
> Well, that's all i have to say about that, Forrest Gump.
>
>
> El Lunes, 9 de Noviembre de 2009 09:45:10 Tony Rogvall escribió:
>> Interesting discussion!
>>
>> I have been working on a resource system for Erlang for nearly two
>> years now.
>> I have a working (tm) prototype where you can set resource limits
>> like
>> max_processes/max_ports/max_memory/max_time/max_reductions ...
>> The limits are passed with spawn_opt and are inherited by the
>> processes spawned.
>> This means that if you spawn_opt(M,F,A[{max_memory, 1024*1024}]) the
>> process
>> will be able to use 1M words for it self and it's "subprocesses".
>> This
>> also means
>> that the spawner will get 1M less to use (as designed right now).
>> If a
>> resource limit
>> is reached the process crash with system_limt reason.
>>
>> There are still some details to work out before a release, but I will
>> try to get it ready before
>> end of this year.
>>
>> /Tony
>>
>>
>>
>> On 9 nov 2009, at 09.16, Robert Virding wrote:
>>
>>> No.
>>>
>>> There is a major difference between handling OOM in an OS and in the
>>> BEAM.
>>> In an OS it usually at a per process level that memory runs out so
>>> it is
>>> easy to decide which process to kill so that the OS can continue. In
>>> the
>>> BEAM, however, it is the VM as a whole which has run out of memory
>>> not a
>>> specific, it is. therefore, much more difficult to work out which
>>> process is
>>> the culprit and to decide what to do. For example it might be that
>>> the
>>> process which causes the OOM is not the actual problem process, it
>>> might
>>> just the last straw. Or the actual cause may that the whole app
>>> might be
>>> generating large binaries too quickly. Or it might be that the whole
>>> app is
>>> spawning to many processes without any one process being the cause.
>>> Or ...
>>> In all these cases killing the process which triggered the OOM would
>>> be the
>>> Wrong Thing. We found that it was difficult to work out a reasonable
>>> strategy to handle the actual cause so we decided not handle it.
>>>
>>> "Don't catch an error which you can't handle" as the bard put it.
>>>
>>> Robert
>>>
>>> 2009/11/9 Max Lapshin <max.lapshin@...>
>>>
>>>> Yes, there are techniques to write watchdogs, but my question
>>>> was: is
>>>> it possible to prevent Erlang VM from crash?
>>>>
>>>> ________________________________________________________________
>>>> erlang-questions mailing list. See http://www.erlang.org/faq.html
>>>> erlang-questions (at) erlang.org
>>>>
>>>>
>>
>>
>> ________________________________________________________________
>> erlang-questions mailing list. See http://www.erlang.org/faq.html
>> erlang-questions (at) erlang.org
>>
>>
>
>
> Este correo no tiene dibujos. Las formas extrañas en la pantalla son
> letras.
> __________________________________________
>
> Clist UAH a.k.a Angel
> __________________________________________
> Warning: Microsoft_bribery.ISO contains OOXML code.
>
> ________________________________________________________________
> erlang-questions mailing list. See http://www.erlang.org/faq.html
> erlang-questions (at) erlang.org
>

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

Re: Why Beam.smp crashes when memory is over?

2009-11-09T03:43:02Z

Well please let me say something

I'm plain new but some things are pretty clear for me.

The beauty of the erlang concept is "let it crash" , "don't program defensively"
so the VM and the underlaying hardware are entities that can fail, that's it.

What's the problem so?

Joe said...

If you want failure tolerance you need at least two nodes...

From J.A thesis
" ...Schneider [60, 59] answered this question by giving three properties
that he thought a hardware system should have in order to be suitable for
programming a fault-tolerant system. These properties Schneider called:..."

1. Halt on failure — in the event of an error a processor should halt
instead of performing a possibly erroneous operation.2

So on memory exhaustation the VM has to die and other node (erlang) will do the recovery.

that's the distrirbution role, no only to span computations over several nodes to enhance performance
but to provide resilence in the presence of fatal errors (non correctable).

As a OS process the VM has to compete with other OS processes so in a shared deployment (a VM running
on a server or a desktop) you cant be safe agaisnt a OOM trigered by other entities.

Such resource control thing wil only augment process overhead and context switching in the VM.

People new to erlang will be atracted to this hierarquical decomposition of tasks as joe stated in his thesis
"If you cant run certaing task try doing something simpler"

Many languages and VM's are incorporating erlang's good "multicore" features but not the erlang powerfull error handling concept
and you guys want to kill the simpliticy incorporating many defensive capabilities to avoid fatality instead of just organize code to
handle such fatality.

¿whats next?, ¿A mailbox maximum message queue control?

Well, that's all i have to say about that, Forrest Gump.

El Lunes, 9 de Noviembre de 2009 09:45:10 Tony Rogvall escribió:

> Interesting discussion!
>
> I have been working on a resource system for Erlang for nearly two
> years now.
> I have a working (tm) prototype where you can set resource limits like
> max_processes/max_ports/max_memory/max_time/max_reductions ...
> The limits are passed with spawn_opt and are inherited by the
> processes spawned.
> This means that if you spawn_opt(M,F,A[{max_memory, 1024*1024}]) the
> process
> will be able to use 1M words for it self and it's "subprocesses". This
> also means
> that the spawner will get 1M less to use (as designed right now). If a
> resource limit
> is reached the process crash with system_limt reason.
>
> There are still some details to work out before a release, but I will
> try to get it ready before
> end of this year.
>
> /Tony
>
>
>
> On 9 nov 2009, at 09.16, Robert Virding wrote:
>
> > No.
> >
> > There is a major difference between handling OOM in an OS and in the
> > BEAM.
> > In an OS it usually at a per process level that memory runs out so
> > it is
> > easy to decide which process to kill so that the OS can continue. In
> > the
> > BEAM, however, it is the VM as a whole which has run out of memory
> > not a
> > specific, it is. therefore, much more difficult to work out which
> > process is
> > the culprit and to decide what to do. For example it might be that the
> > process which causes the OOM is not the actual problem process, it
> > might
> > just the last straw. Or the actual cause may that the whole app
> > might be
> > generating large binaries too quickly. Or it might be that the whole
> > app is
> > spawning to many processes without any one process being the cause.
> > Or ...
> > In all these cases killing the process which triggered the OOM would
> > be the
> > Wrong Thing. We found that it was difficult to work out a reasonable
> > strategy to handle the actual cause so we decided not handle it.
> >
> > "Don't catch an error which you can't handle" as the bard put it.
> >
> > Robert
> >
> > 2009/11/9 Max Lapshin <max.lapshin@...>
> >
> >> Yes, there are techniques to write watchdogs, but my question was: is
> >> it possible to prevent Erlang VM from crash?
> >>
> >> ________________________________________________________________
> >> erlang-questions mailing list. See http://www.erlang.org/faq.html
> >> erlang-questions (at) erlang.org
> >>
> >>
>
>
> ________________________________________________________________
> erlang-questions mailing list. See http://www.erlang.org/faq.html
> erlang-questions (at) erlang.org
>
>

Este correo no tiene dibujos. Las formas extrañas en la pantalla son letras.
__________________________________________

Clist UAH a.k.a Angel
__________________________________________
Warning: Microsoft_bribery.ISO contains OOXML code.

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

Re: Dialyzer problem on OSX 10.6 (Erlang R13B02-1)

2009-11-09T03:35:18Z

Hi Tino,

Are you using homebrew?

So far it seems that homebrew applies some c compiler magic (e.g optimizations) when building erlang which somehow borks dialyzer (strange) as compiling from source with the same ./configure options homebrew uses results in an erlang installation which works like a charm.

Regards,
Tamas
----- "Tino Breddin" <tino.breddin@...> wrote:

> Hey Tamas,
>
> Here's the output for the command run on my machine (Macbook Pro Core
>
> Duo, running OSX 10.6.1 32-bit):
>
> dialyzer --build_plt -r /usr/local/Cellar/erlang/R13B02-1/lib/erlang/
>
> lib/erts-5.7.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/lib/erlang/lib/
>
> kernel-2.13.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/lib/erlang/lib/
>
> stdlib-1.16.3/ebin/
> Creating PLT /Users/tino/.dialyzer_plt ...
> Unknown functions:
> compile:file/2
> compile:forms/2
> compile:noenv_forms/2
> compile:output_generated/1
> crypto:des3_cbc_decrypt/5
> crypto:start/0
> done in 46m22.52s
> done (passed successfully)
>
> Apparently it takes some time to complete but you'll get there.
>
> Cheers,
> Tino
>
> On Nov 8, 2009, at 11:35 PM, Tamas Nagy wrote:
>
> > Hi Kostis,
> >
> > This is what all I get:
> > $ dialyzer --build_plt -r /usr/local/Cellar/erlang/R13B02-1/lib/
> > erlang/lib/erts-5.7.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/lib/
> > erlang/lib/kernel-2.13.3/ebin/
> /usr/local/Cellar/erlang/R13B02-1/lib/
> > erlang/lib/stdlib-1.16.3/ebin/
> > Creating PLT /Users/lestat/.dialyzer_plt ...
> >
> > I will try to rebuild erlang with disabling hipe to see if that
> helps.
> >
> > I am happy to run some tests/traces you would like to see the
> > results of to help.
> >
> > Regards,
> > Tamas
> >
> > Tamas Nagy
> > Erlang Training & Consulting
> > http://www.erlang-consulting.com
> >
> > On 8 Nov 2009, at 19:52, Kostis Sagonas wrote:
> >
> >> Tamas Nagy wrote:
> >>> Hi,
> >>> Am I the only one who cannot build a plt under OSX 10.6 in
> >>> R13B02-1?
> >>> No matter what applications I specify for the initial plt it just
>
> >>> hangs eating processing power but not producing anything.
> >>> Erlang was installed with homebrew, but that shouldn't matter. Or
>
> >>> maybe it does. (beam is compiled for 64 bit)
> >>> Here is the command I used:
> >>> dialyzer --build_plt -r /usr/local/Cellar/erlang/R13B02-1/lib/
> >>> erlang/lib/erts-5.7.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/lib/
>
> >>> erlang/lib/kernel-2.13.3/ebin/ /usr/local/Cellar/erlang/R13B02-1/
>
> >>> lib/erlang/lib/stdlib-1.16.3/ebin/
> >>> I gave waiting for it to finish after ~1 hour (on an old type
> >>> white MacBook (MacBook2,1) with 2GB of RAM).
> >>
> >> None of the dialyzer developers has access to such a machine to see
>
> >> what's happening. Perhaps somebody else in that list might want to
>
> >> report whether this is reproducible or not.
> >>
> >> However, perhaps the following might help: when building the PLT,
>
> >> do you see a message:
> >>
> >> Compiling some key modules to native code...
> >>
> >> If so, can you disable hipe (64-bit Mac is not a platform we have
>
> >> ever tested due to no access to such a machine) to see whether this
>
> >> may be the culprit?
> >>
> >> Kostis
> >
> >
> > ________________________________________________________________
> > erlang-questions mailing list. See http://www.erlang.org/faq.html
> > erlang-questions (at) erlang.org
> >

--
Tamas Nagy
Erlang Training & Consulting
http://www.erlang-consulting.com

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org