Review Request: future library (Gaskill version)

[Frank Mori Hess]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Saturday 17 May 2008 07:32 am, Johan Torp wrote:

> Frank Mori Hess wrote:
> > future<void> future_barrier(const future<void> &a1, const future<void>&
> > a2, ... , const future<void> &aN);
> >
> > future<void> future_switch(const future<void> &a1, const future<void>&
> > a2, ... , const future<void> &aN);
> >
> > A naive implementation would probably be pretty inefficient when
> > evaluating a
> > future that has been produced after passing through many
> > future_barrier/future_switch calls.  But I imagine some optimizations
> > could
> > be found under the hood in the implementation to avoid unnecessarily long
> > chains of futures depending on each other.
>
> This would be nice and could potentially solve the inefficiency problem.
> You are allowed to wait for any node in a tree. The main difficulty is
> figuring out which nodes only exists in the tree and which who are visible
> to the outside world. The nodes who only exist in the tree can be
> re-arranged into a compact tree. Maybe we can use unique_future to
> guarantee this. I.e. the combination functions always return unique_futures
> and if we get these as in-parameters, we can safely re-arrenge the tree.

In case you're interested, I've just implemented future_barrier and
future_select (changed name from future_switch) free functions in libpoet
cvs:

http://www.comedi.org/cgi-bin/viewvc.cgi/libpoet/poet/future_waits.hpp?view=markup

I didn't do any clever under-the-hood optimizations, only provided overloads
that accept 2 to 10 future<T> arguments, plus one that accepts first/last
iterators to a container of futures.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFINJB25vihyNWuA4URAso5AKDgrKbSC7xLxMgav+dNB3PQsFSb6ACgmscf
jdiBKjmET32LjvRuAvyUXmw=
=UB2b
-----END PGP SIGNATURE-----
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Johan Torp]

Frank Mori Hess wrote:

In case you're interested, I've just implemented future_barrier and
future_select (changed name from future_switch) free functions in libpoet
cvs:

http://www.comedi.org/cgi-bin/viewvc.cgi/libpoet/poet/future_waits.hpp?view=markup

I didn't do any clever under-the-hood optimizations, only provided overloads
that accept 2 to 10 future<T> arguments, plus one that accepts first/last
iterators to a container of futures.

Interesting! Do you use these within poet anywhere or is it for poet users only?

I think it would be good to add some functionality though. For simplicity, let's only talk about the select case. This is what I propose:

1. Clients will probably be interested in which of the child futures fired. The interface should support this.

2. Upon notification from a child, I'd want to be able to check a condition and then either
   a - become ready
   b - prune the "consumed" child future and go back to sleep
2.1 This condition checking should probably be done by the first thread which is waiting for the future - it belongs to the future-listening threads, not the promise-fulfilling one. If nobody is waiting for the future we should save this evalution and let it be performed lazily as soon as someone queries the future or waits for it.
2.2 This evaluation is not just a predicate which can make the future ready. It can also be actual calculations. These would be performed from within future::wait(), get() and is_ready().
2.3 We probably need to supply a default value if all children have been consumed

3. We probably want to support composite futures from different types. If we want to support 1 or 2 we want to get access to the typed value of the ready future. In practice, this probably means the evaluation function should be added at the same time as the child future to ensure type safety. See the interface example below.


A simplified example of what an interface which supports 1, 2 and 3 might look like is:  

template<class ReturnType, class Arg1Type, class Arg2Type>
future<ReturnType> future_select(future<Arg1Type> f1, function<optional<ReturnType>(Arg1Type)> e1,
                                              future<Arg2Type> f2, function<optional<ReturnType>(Arg2Type)> e2,
                                              ReturnType default_value);

I'm not entirely satisfied with this though.


In the end, I want users to be able to easily "lift" ordinary functions to futures:
int foo_func(int, double, bool) => future<int> foo_func(future<int>, future<double>, future<bool>)


I have also been thinking about we could move part of this composed waiting in to a conditon_variable_node class to separate concerns.


Johan

[Frank Mori Hess]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Friday 23 May 2008 05:45 am, Johan Torp wrote:
> Interesting! Do you use these within poet anywhere or is it for poet users
> only?

It's only for users at the moment, I haven't rewritten the schedulers to use
it.

> I think it would be good to add some functionality though. For simplicity,
> let's only talk about the select case. This is what I propose:

How about an additional future_select overload which accepts two
boost::shared_container_iterators as arguments, and returns a
future<shared_container_iterator>?  The returned iterator would point at the
future which is ready or has an exception.  Then the caller could use the
returned iterator to erase the element from the container before calling
future_select again.  The shared_container_iterator would insure the input
container of futures doesn't die before the return future becomes ready.

> 1. Clients will probably be interested in which of the child futures fired.
> The interface should support this.
>
> 2. Upon notification from a child, I'd want to be able to check a condition
> and then either
>    a - become ready
>    b - prune the "consumed" child future and go back to sleep
> 2.1 This condition checking should probably be done by the first thread
> which is waiting for the future - it belongs to the future-listening
> threads, not the promise-fulfilling one. If nobody is waiting for the
> future we should save this evalution and let it be performed lazily as soon
> as someone queries the future or waits for it.
> 2.2 This evaluation is not just a predicate which can make the future
> ready. It can also be actual calculations. These would be performed from
> within future::wait(), get() and is_ready().
> 2.3 We probably need to supply a default value if all children have been
> consumed

Having a future execute a callback at the end of a wait completing seems only
marginally useful to me.  I'm not sure it's worth the complication.

> 3. We probably want to support composite futures from different types. If
> we want to support 1 or 2 we want to get access to the typed value of the
> ready future. In practice, this probably means the evaluation function
> should be added at the same time as the child future to ensure type safety.
> See the interface example below.
>
>
> A simplified example of what an interface which supports 1, 2 and 3 might
> look like is:
>
> template<class ReturnType, class Arg1Type, class Arg2Type>
> future<ReturnType> future_select(future<Arg1Type> f1,
> function<optional<ReturnType>(Arg1Type)> e1,
>                                               future<Arg2Type> f2,
> function<optional<ReturnType>(Arg2Type)> e2,
>                                               ReturnType default_value);
>
> I'm not entirely satisfied with this though.

Oh, now its clear to me why you were using the name future_switch.

> In the end, I want users to be able to easily "lift" ordinary functions to
> futures:
> int foo_func(int, double, bool) => future<int> foo_func(future<int>,
> future<double>, future<bool>)

That's exactly what poet::active_function does, or did you have something
different in mind?

> I have also been thinking about we could move part of this composed waiting
> in to a conditon_variable_node class to separate concerns.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFINsy+5vihyNWuA4URAj/aAKC5FhgCf3V7oqvAqb3JfsFUPDEbYwCgrJ+P
7J6qL6wPx9y771rTX1e5Q9w=
=ebvj
-----END PGP SIGNATURE-----
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Johan Torp]

Frank Mori Hess wrote:

How about an additional future_select overload which accepts two
boost::shared_container_iterators as arguments, and returns a
future<shared_container_iterator>?  The returned iterator would point at the
future which is ready or has an exception.  Then the caller could use the
returned iterator to erase the element from the container before calling
future_select again.  The shared_container_iterator would insure the input
container of futures doesn't die before the return future becomes ready.

That would restrict users to put their futures in a container created as a shared_ptr, right? Williams' shared_future already has reference semantics and could be copied. unique_futures should somehow be moved in and owned by the composite future as nobody else should look at it's result. Not sure what is best here.

Frank Mori Hess wrote:

Having a future execute a callback at the end of a wait completing seems only
marginally useful to me.  I'm not sure it's worth the complication.

- snip -

That's exactly what poet::active_function does, or did you have something
different in mind?

Yes, I'm thinking of a passive function which doesn't have an internal thread associated with it. One that is evaluated lazily as soon as someone is interested in it.

I think this could be very useful, do you? I haven't worked very much with active objects so your 5 cents are probably worth more than mine :) But yes, it requires some complexity - probably comparable to that of active_function.

Johan

[Frank Mori Hess]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Friday 23 May 2008 10:50 am, Johan Torp wrote:

> Frank Mori Hess wrote:
> > How about an additional future_select overload which accepts two
> > boost::shared_container_iterators as arguments, and returns a
> > future<shared_container_iterator>?  The returned iterator would point at
> > the
> > future which is ready or has an exception.  Then the caller could use the
> > returned iterator to erase the element from the container before calling
> > future_select again.  The shared_container_iterator would insure the
> > input container of futures doesn't die before the return future becomes
> > ready.
>
> That would restrict users to put their futures in a container created as a
> shared_ptr, right?

Yes, but the shared_container_iterator isn't really necessary, I could make
future_select return a future<Iterator> for any type of iterator input.  I
was just trying to make it safer.  Unfortunately, even using
shared_container_iterator still wouldn't prevent the user from modifying the
container so as to invalidate the returned future iterator anyways.  Maybe
I'll just suggest using shared_container_iterator in the docs.

> Yes, I'm thinking of a passive function which doesn't have an internal
> thread associated with it. One that is evaluated lazily as soon as someone
> is interested in it.
>
> I think this could be very useful, do you?

I don't know, it might be.  I've never tried to program something using a lot
of nested lazy evaluation.  Could you use boost::bind to achieve the same
effect?  Doesn't it do something like this by default if you do a recursive
bind of a function to a parameter, and don't wrap the function in
boost::protect()?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFINupX5vihyNWuA4URAt0/AJ9WDjsz8KpTZx3P5xcCpqck3xLapwCdHPrf
YbCeNge/oz3Nbz51kAD40tM=
=S3P6
-----END PGP SIGNATURE-----
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Johan Torp]

Frank Mori Hess wrote:

> Yes, I'm thinking of a passive function which doesn't have an internal
> thread associated with it. One that is evaluated lazily as soon as someone
> is interested in it.
>
> I think this could be very useful, do you?

I don't know, it might be.  I've never tried to program something using a lot
of nested lazy evaluation.  Could you use boost::bind to achieve the same
effect?  Doesn't it do something like this by default if you do a recursive
bind of a function to a parameter, and don't wrap the function in
boost::protect()?

I'm not sure I understand you. boost.bind itself has nothing to do with threads waiting on eachother. You will probably use boost.bind to create the evaluation functions (e1 and e2 in my previous post) though.


Johan

[Frank Mori Hess]

On Friday 23 May 2008 14:32 pm, Johan Torp wrote:
>
> I'm not sure I understand you. boost.bind itself has nothing to do with
> threads waiting on eachother. You will probably use boost.bind to create
> the evaluation functions (e1 and e2 in my previous post) though.
>

I must have read too much into what you said before.  I was thinking you were
entirely in single-threaded mode, and all your futures were just coming from
the same kind of lazy evaluations.

Anyways, I think the main deficiency with the prototype of future_select()
I've coded is it will always be at least O(N) where N is the number of input
futures.  So if you are waiting for N futures one at a time, it will take N
calls to future_select giving O(N^2) complexity.  

If you have some kind of "future combination" class that contains the set of
futures being waited on, and can remove completed futures from it in O(1)
time, you might do better.  It would take O(N) time to create the "future
combination" initially, but then each wait for the next completed future
could be O(1).

Okay, to summarize where I think we are:

1) In order to wait on an arbitrary number of futures, determined at runtime,
we need some kind of container.  

2) Since containers are homogeneous, that means some kind of type erasure
(future<void>).

3) When a future becomes ready, we will often want to dispatch it somehow
which might mean getting its value, or at least identifying the input future
which became ready.  That could mean doing some kind of type unerasure, or
doing an inefficient loop over all the original input futures until you find
the completed one.  Your solution is to bind the dispatcher function to the
future before it is type erased.  For thread-safety, you want to run the
dispatcher function in one of the waiting threads, right after the future
becomes ready.

There will be deadlock hazards running the user's dispatcher function while
holding the future's associated mutex.  Also, it's possible multiple threads
will be waiting on the future, in which case how much thread-safety benefit
is there to running code in one of the waiting threads versus a callback run
in the promise
thread?

What if we allow a key value to be associated with each input future, then
return the key of the completed future?  Then it would be left to the user to
map the key back to whatever future they were originally interested in.
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Johan Torp]

Frank Mori Hess wrote:

1) In order to wait on an arbitrary number of futures, determined at runtime,
we need some kind of container.  

Yes, but it needn't be exposed to users. As you suggested, it can be built up with free functions or expressive templates, similar to Gaskill's comb. This is a detail though, I don't want to focus on it now.

Frank Mori Hess wrote:

2) Since containers are homogeneous, that means some kind of type erasure
(future<void>).

Yes, type erasure will be needed. I hope we can rely on boost.function for that, similar to the predicate version of condition_variable::wait().

Frank Mori Hess wrote:

3) When a future becomes ready, we will often want to dispatch it somehow
which might mean getting its value, or at least identifying the input future
which became ready.  That could mean doing some kind of type unerasure, or
doing an inefficient loop over all the original input futures until you find
the completed one.  Your solution is to bind the dispatcher function to the
future before it is type erased.  For thread-safety, you want to run the
dispatcher function in one of the waiting threads, right after the future
becomes ready.

Thread safety is not my primary concern. The "predicate" code might throw an exception and the promise-fullfilling thread can't be expected to catch it. That would couple the promise-fulfiller and the future-listeners in an awkward way.

An alternative might be to catch the exceptions and set them on the composite future. Thinking of it, this would be the expected behaviour, wouldn't it?

Frank Mori Hess wrote:

There will be deadlock hazards running the user's dispatcher function while
holding the future's associated mutex.

Good point. I hope we don't need to hold any mutex while evaluating the predicate.

Frank Mori Hess wrote:

Also, it's possible multiple threads
will be waiting on the future, in which case how much thread-safety benefit
is there to running code in one of the waiting threads versus a callback run
in the promise
thread?

The callback does not need to be thread safe unless it holds references to some particular thread's data. Only one of the waiting threads will evaluate it. Also, if there is only one "listening" thread it can safely reference it's own data.

Frank Mori Hess wrote:

What if we allow a key value to be associated with each input future, then
return the key of the completed future?  Then it would be left to the user to
map the key back to whatever future they were originally interested in.

This would mean a runtime mapping. This costs some but more importantly could introduce bugs because of the added runtime insecurity.  I'm hoping we can avoid it.

I'm doing a thesis related to C++0x and threading. Peter Dimov suggested I could try and implement an event-like waiting primitive, similar to Windows' WaitForMultipleObjects. If we had such a primitive, we might get better ways to solve this problem. I think part of the reason this is so tricky is because condition_variable's aren't meant to be used for multiple waiting. Maybe Dimov or somebody else has sketches of such an interface. Maybe we should solve this problem first. What do you think?


Johan

[Peter Dimov-5]

Johan Torp:
> Frank Mori Hess wrote:
>>
>> 1) In order to wait on an arbitrary number of futures, determined at
>> runtime,
>> we need some kind of container.
>
> Yes, but it needn't be exposed to users. As you suggested, it can be built
> up with free functions or expressive templates, similar to Gaskill's comb.

Suppose that the programmer wants to spawn n tasks, where n is not a
compile-time constant, and is only interested in whatever task returns a
value first. Something like:

    T f( int x );

    // ...

    vector<future<T>> v;

    for( int i = 0; i < n; ++i )
    {
        v.push_back( async( f, i ) );
    }

    // wait for any of v[i] to complete, get the T

How could this be rephrased with the container being an implementation
detail?

Here's one way:

    future<T> ft;

    for( int i = 0; i < n; ++i )
    {
        ft = ft || async( f, i );
    }

    T t = ft.get();

This however relies on:

    future<T> operator|| ( future<T>, future<T> );

You can't use 'comb' as the return value.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Peter Dimov-5]

Johan Torp:

> I'm doing a thesis related to C++0x and threading. Peter Dimov suggested I
> could try and implement an event-like waiting primitive, similar to
> Windows'
> WaitForMultipleObjects. If we had such a primitive, we might get better
> ways
> to solve this problem. I think part of the reason this is so tricky is
> because condition_variable's aren't meant to be used for multiple waiting.
> Maybe Dimov or somebody else has sketches of such an interface.

I believe that futures only need the basic "manual reset event" in Windows
terms:

class event
{
private:

    bool is_set_;

public:

    event(); // post: !is_set_

    void set(); // post: is_set_
    void reset(); // post: !is_set_

    void wait(); // block until is_set_
    bool try_wait();
    bool try_wait_for(...);
    bool try_wait_until(...);
};

The advantage of this primitive is that it maps to Windows directly. The
disadvantage is that it's not robust when reset() is used because this
introduces a race. This is not a problem for futures since they are one-shot
and their underlying ready_ event is never reset.

A more robust primitive is the so-called "eventcount" which doesn't need to
be reset and avoids the race:

class event
{
private:

    unsigned k_;

public:

    event(); // post: k_ == 0

    void signal(); // effects: ++k_

    unsigned get(); // returns: k_
    void wait( unsigned k ); // block until k_ != k

    // ... try_wait ...
};

Typical use is

// waiter

    for( ;; )
    {
        unsigned k = event_.get();
        if( predicate ) return;
        event_.wait( k );
    }

// setter

    predicate = true;
    event_.signal();

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Johan Torp]

Peter Dimov-5 wrote:

    future<T> operator|| ( future<T>, future<T> );

You can't use 'comb' as the return value.

In the expressive template solution, operator|| would return an unspecified future-expression type and futures can be constructed from it. Just like boost.function and boost.bind interacts. Other overloads would also work with these unnamed future-expression temporaries.

Johan

[Frank Mori Hess-2]

On Sunday 25 May 2008 14:02, Peter Dimov wrote:

> Here's one way:
>
>     future<T> ft;
>
>     for( int i = 0; i < n; ++i )
>     {
>         ft = ft || async( f, i );
>     }
>
>     T t = ft.get();

That reminds me, I've been so focused on trying to make future_select work
with heterogeneous input futures, that I neglected the homogeneous case.  
That is, future_select() always returns a future<void> even when all the
input futures have the same type.  And after all, maybe supporting the
homogeneous case is good enough.  For example, in my use case I've already
taken care to type-erase the method requests in the schedulers down to a
uniform type, because the scheduler needs to store them in a container
anyways.

So the only other thing I'd need is some kind of explicit future container
that can be re-used to "wait for any" repeatedly without rebuilding the
entire set of futures each time.  Maybe something with a queue-like
interface:

template<typename T>
class future_selecter
{
public:
        future<T> front();
        void pop();
        void push(const future<T> &f);
};

where the future returned by front() would have the value from the next
future in the future_selector to become ready.  pop() would discard the
completed future previously being returned by front(), so front() can
return a new future corresponding to the next one to complete.


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Frank Mori Hess-2]

On Friday 23 May 2008 10:50, Johan Torp wrote:
> Yes, I'm thinking of a passive function which doesn't have an internal
> thread associated with it. One that is evaluated lazily as soon as
> someone is interested in it.
>
> I think this could be very useful, do you?

Hmm, so it does seem that I would need something like what you've been
suggesting to convert my scheduler over to waiting on futures instead of
relying directly on signals/slots.  Or, at least I would minimally need to
be able to construct a future<X> from and future<Y> where X and Y are
unrelated types by specifying a passive conversion function.  So for
example, when

future<Y> fut_y;

becomes ready, the future<X> would become ready with the value returned by
conversion_function(fut_y.get()).

The more general solution that accepts multiple arguments would be like
future_barrier, except it would accept an additional first argument that
is the conversion function, and it would return a future<R> where the
value R for the returned future is obtained by calling the conversion
function with all the values from the input futures.  So it might look
something like

R combining_function(const T1 &t1, const T2 &t2, ..., const TN &tn);

future<T1> f1;
future<T1> f2;
//...
future<T1> fN;

future<R> result = future_combining_barrier(&combining_function, f1,
f2, ..., fN);


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Johan Torp]

Frank Mori Hess-2 wrote:

The more general solution that accepts multiple arguments would be like
future_barrier, except it would accept an additional first argument that
is the conversion function, and it would return a future<R> where the
value R for the returned future is obtained by calling the conversion
function with all the values from the input futures.  So it might look
something like

R combining_function(const T1 &t1, const T2 &t2, ..., const TN &tn);

future<T1> f1;
future<T1> f2;
//...
future<T1> fN;

future<R> result = future_combining_barrier(&combining_function, f1,
f2, ..., fN);

This is how I imagine the barrier/wait-for-all case looks too. The select/switch/wait-for-any case is tricker if we wan't to supply the same type safety for heterogenous types.

Johan

[Johan Torp]

Frank Mori Hess-2 wrote:

That reminds me, I've been so focused on trying to make future_select work
with heterogeneous input futures, that I neglected the homogeneous case.  
That is, future_select() always returns a future<void> even when all the
input futures have the same type.  And after all, maybe supporting the
homogeneous case is good enough.  For example, in my use case I've already
taken care to type-erase the method requests in the schedulers down to a
uniform type, because the scheduler needs to store them in a container
anyways.

So the only other thing I'd need is some kind of explicit future container
that can be re-used to "wait for any" repeatedly without rebuilding the
entire set of futures each time.  Maybe something with a queue-like
interface:

template<typename T>
class future_selecter
{
public:
        future<T> front();
        void pop();
        void push(const future<T> &f);
};

where the future returned by front() would have the value from the next
future in the future_selector to become ready.  pop() would discard the
completed future previously being returned by front(), so front() can
return a new future corresponding to the next one to complete.

I still think solving the heterogenous case is worthwhile. The interface isn't much more complicated, it could look like this:

template<typename ResultType>
class future_selecter
{
public:
        template<class T>
        void push(const future<T> &f, function<optional<ResultType>(T)>& eval);
};

template<class ReturnType>
future::future(future_selector<ReturnType>&)

Johan

[Frank Mori Hess-2]

On Monday 26 May 2008 03:53, Johan Torp wrote:

> Frank Mori Hess-2 wrote:
> > So the only other thing I'd need is some kind of explicit future
> > container that can be re-used to "wait for any" repeatedly without
> > rebuilding the entire set of futures each time.  Maybe something with
> > a queue-like interface:
> >
> > template<typename T>
> > class future_selecter
> > {
> > public:
> > future<T> front();
> > void pop();
> > void push(const future<T> &f);
> > };
> >
> > where the future returned by front() would have the value from the
> > next future in the future_selector to become ready.  pop() would
> > discard the completed future previously being returned by front(), so
> > front() can return a new future corresponding to the next one to
> > complete.
>
> I still think solving the heterogenous case is worthwhile. The interface
> isn't much more complicated, it could look like this:
>
> template<typename ResultType>
> class future_selecter
> {
> public:
>         template<class T>
> void push(const future<T> &f, function<optional<ResultType>(T)>& eval);
> };
>
> template<class ReturnType>
> future::future(future_selector<ReturnType>&)

The thing is, this whole exercise started for me to see if it was possible
to get rid of the public signal/slots on futures and replace it with just
various waits on futures.  Once we start adding callback functions to the
future_select and future_barrier, which are executed on completion, it's
really a sign of failure to me.  I'd rather just leave the public slots on
futures as that is simpler, more powerful, and no less dangerous.


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Johan Torp]

Frank Mori Hess-2 wrote:

The thing is, this whole exercise started for me to see if it was possible
to get rid of the public signal/slots on futures and replace it with just
various waits on futures.  Once we start adding callback functions to the
future_select and future_barrier, which are executed on completion, it's
really a sign of failure to me.  I'd rather just leave the public slots on
futures as that is simpler, more powerful, and no less dangerous.

I agree public slots are simpler and more powerful. IMO they are a lot more dangerous though. Some examples:

If I connect a slot that might throw an exception, that exception will be thrown from the promise-fullfilling thread's set()-call. This is totally unexpected.

If a slot acquires a lock, it can lead to unexpected deadlocks if the promise-fullfilling thread is holding an external lock while calling future::set(). Assume you have two locks which you always should acquire in some specified order to avoid dead-locks. Say you should acquire mutex1 before mutex2;

slot:
  scoped_lock l(mutex1);

promise-fulfiller:
  scoped_lock l(mutex2);
  promise.set(100); // It is totally unexpected that this code will acquire mutex1

other thread:
  scoped_lock l(mutex1);
  scoped_lock l(mutex2);

I don't know how common this particular example would be but I'm guessing there are lots of similar problems out there. Basically, you don't expect future::set to run a lot of aribtrary code. And if you do, you have coupled the future-listeners and future-fulfillers in a very implicit way.

If futures are internal to a library such as poet it is fine. But to expose a generic future-object with such an interface is far from optimal, IMHO.

Johan

[Frank Mori Hess-2]

On Monday 26 May 2008 16:13, Johan Torp wrote:
> I agree public slots are simpler and more powerful. IMO they are a lot
> more dangerous though. Some examples:
>
> If I connect a slot that might throw an exception, that exception will
> be thrown from the promise-fullfilling thread's set()-call. This is
> totally unexpected.

That's true.  However, the connect call of the future which connects a slot
to the future-complete signal could return a future<void>.  The returned
future<void> would either become ready if the slot runs successfully, or
transport the exception the slot threw.  I think you could even make it a
template function and have it return a future<R>, and let it accept the
completed future's value as an input parameter.

> If a slot acquires a lock, it can lead to unexpected deadlocks if the
> promise-fullfilling thread is holding an external lock while calling
> future::set(). Assume you have two locks which you always should acquire

That is even more true when the callback code is run in future::get() or
future::ready(), since the callback code will have to be run while holding
the future's mutex (unless perhaps if it is a unique_future).  If the user
callback code is run in the promise-setting thread, it can be done without
holding any locks internal to the library.

> I don't know how common this particular example would be but I'm
> guessing there are lots of similar problems out there. Basically, you
> don't expect future::set to run a lot of aribtrary code. And if you do,

Is future::get or future::ready to running a lot of arbitrary code any less
surprising?


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Johan Torp]

Frank Mori Hess-2 wrote:

> If I connect a slot that might throw an exception, that exception will
> be thrown from the promise-fullfilling thread's set()-call. This is
> totally unexpected.

That's true.  However, the connect call of the future which connects a slot
to the future-complete signal could return a future<void>.  The returned
future<void> would either become ready if the slot runs successfully, or
transport the exception the slot threw.  I think you could even make it a
template function and have it return a future<R>, and let it accept the
completed future's value as an input parameter.

You would need to return a connection too so that you can disconnect. And after a wait-for-any composition has become ready, how would you know which of all the connected slots you need to check for an exception? I think that having a composite future in which the exception appears is a more natural interface.

Frank Mori Hess-2 wrote:

> If a slot acquires a lock, it can lead to unexpected deadlocks if the
> promise-fullfilling thread is holding an external lock while calling
> future::set(). Assume you have two locks which you always should acquire

That is even more true when the callback code is run in future::get() or
future::ready(), since the callback code will have to be run while holding
the future's mutex (unless perhaps if it is a unique_future).  If the user
callback code is run in the promise-setting thread, it can be done without
holding any locks internal to the library.

Do you really need to hold the future's mutex while doing the evaluation? Can't other threads just see it as not_ready until the evaluation is complete and one of the future-listeners call future::set() or future::set_exception() on the composite future?

Frank Mori Hess-2 wrote:

Is future::get or future::ready to running a lot of arbitrary code any less
surprising?

IMHO, yes. The future-listeners are the ones which set up the lazy evaluation so they should be prepared for it. Just like condition_variable's predicate wait. If you set up a composite future and then share it to other future-listening threads, you can state that this future can throw this and that or does xxx. The promise-fulfiller OTOH should be de-coupled from what it's listeners are doing.


Johan

[Frank Mori Hess]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tuesday 27 May 2008 05:46 am, Johan Torp wrote:
> Do you really need to hold the future's mutex while doing the evaluation?
> Can't other threads just see it as not_ready until the evaluation is
> complete and one of the future-listeners call future::set() or
> future::set_exception() on the composite future?

You're probably right, I shouldn't say things can't be done until I've tried
to do them :)

> If you set up a composite future and
> then share it to other future-listening threads, you can state that this
> future can throw this and that or does xxx. The promise-fulfiller OTOH
> should be de-coupled from what it's listeners are doing.

That's a good point, if the only way to add callback code is in the factory
functions for the composite futures, it means the callback code is only run
by the newly created composite future (or possibly a copy of it), but doesn't
effect the input futures or any other copies of the input futures which may
already exist.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFIPA9H5vihyNWuA4URArUEAJwI6tkpgEUVnedavsJ//CyFgvBdZgCeJk9A
bX/3oDCNumLAqJgZSxvcSow=
=Moh4
-----END PGP SIGNATURE-----
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost