module_transparent/1 (was: splitting facts and predicates between files)

[Günter Kniesel]

[Warning: Reading this mail takes at least 5 minutes.]
-----------------------------------------------------------------------

Hi Paulo and everybody,

Paulo Moura wrote:
 > I believe that the legacy (?) directive module_transparent/1 should
 > be avoided. New code should preferably use the standard directive
 > meta_predicate/1.

This is a tricky issue on which Jan and others might also want to comment.

My personal opinion is that module_transparent/1 was and is a great idea
but was not well documented and thus very hard to use right -- definitely
a source of confusion for novices. Therefore, I understand Jan's
recommendation not to use it for things for which there is a simpler
solution, such as meta_predicate/1.

The source of the confusion was (again in my personal hindsight) the fact
that module_transparent was mainly used in conjunction with meta-predicates
and therefore documented as something that was tightly connected to
meta-predicates. However, that is wrong.

In my opinion, module_transparent/1 and context_module/1 are a
simple and effective solution to a much more fundamental problem:
dynamic binding.

Module_transparent/1 enforces that a predicate "knows" its calling
module (= context module) and context_module/1 lets you find out
about the current caller. Thus it gets easy to enforce dynamic binding
whenever one wants it, overcoming the default static binding of Prolog
predicates.

For instance, each of the subgoals in the following predicate

     inside(X, Y) :-
         obj(X),
         obj(Y),
         consists(X, Y).

will by default be bound to (=will invoke) a predicate definition
in the same module in which inside/2 was declared. Let's call it
the textually containing module.

In contrast to the default behaviour

     inside(X, Y) :-
         context_module(Self),
         Self:obj(X),
         Self:obj(Y),
         Self:consists(X, Y).

will bind each of the subgoals dynamically to the context module,
that is the module that called inside/2. This will work even
transitively, as long as all called predicates are module_transparent.
So if outmostModule:outmostPredicate calls inside through a
chain of module_transparent predicates, then Self will be unified
to outmostModule.

Note that it is easy to combine static and dynamic binding. The
following

     inside(X, Y) :-
         context_module(Self),
         obj(X),
         obj(Y),
         Self:consists(X, Y).

fixes the definitions of obj/1 to be statically bound and only
lets consists/2 be bound dynamically. In object oriented terms
this is a "template method".

Note that all of the above is independent of meta_predicates!

So module_transparent/1 has a right to exist even if meta_predicate/2
is available and people who understand module_transparent/1 well enough
should not be discouraged using it. In particular, Jan has used
module_transparent/1 for implementing meta_predicate/2 ;)

Cheers,
Günter
_______________________________________________
SWI-Prolog mailing list
SWI-Prolog@...
https://mailbox.iai.uni-bonn.de/mailman/listinfo.cgi/swi-prolog

[Ulrich Neumerkel]

Günter Kniesel writes:

>My personal opinion is that module_transparent/1 was and is a great idea
>but was not well documented and thus very hard to use right -- definitely
>a source of confusion for novices. Therefore, I understand Jan's
>recommendation not to use it for things for which there is a simpler
>solution, such as meta_predicate/1.
>
>The source of the confusion was (again in my personal hindsight) the fact
>that module_transparent was mainly used in conjunction with meta-predicates
>and therefore documented as something that was tightly connected to
>meta-predicates. However, that is wrong.
>
>In my opinion, module_transparent/1 and context_module/1 are a
>simple and effective solution to a much more fundamental problem:
>dynamic binding.

The more or less original intention of modules in Prolog was
different.  The idea was to have "the same comfort" and even same
speed as no module system plus "modularity".  Of course that cannot
work out in general, since one point of modularity is to hide certain
things - somewhere you will realize the seams.

Further, programmers often do not completely understand what they want
hidden and what not.  Even more so in Prolog, where you might model
some interface, but on the other hand you also would like to allow
meta-interpreters to inspect your program.  Similar problems are less
evident in other languages where reflection is not so easily accessible
as in Prolog.

Just think of a very simple case of a meta program: a
cross-referencer.  Here, you want to see the otherwise hidden
dependencies of a module.  So you breach that information hiding
promise.

You mentioned cases of dynamic binding using module_transparent can
be done equally well in a system with meta_predicate-only
declarations!  Simply add a further meta-argument and call such a
predicate just with an _ in the argument.

:- meta_predicate inside(?,?,:).
inside(X, Y,  ContextModule:_) :-

Now, the new argument will be a colon structure with the calling
module as prefix.  If you want to make this portable between SWI and
YAP (with Quintus style modules), access the module like so:

inside(X, Y,  Context) :-
    strip_module(Context, ContextModule,_),

In this manner you can be quite portable throughout many systems.  In
fact, this can be considered even ISO 13211-2 compliant.

This is useful for lambda expressions run like this on YAP and SWI:

http://www.complang.tuwien.ac.at/ulrich/Prolog-inedit/lambda.pl



But I agree with you that sometimes module_transparent/1 feels better.
However please note that there are many cases where completely
unexpected things happen!  Here is the worst I am aware of:

:- module(m, [p/2]).

:- module_transparent p/2.

p(Goal,R) :-
   Goal,
   f(R).

f(from_m_1).
f(from_m_2).
f(from_m_2).


From the user module, we get redundant solutions for p(true,R).  So we
wrap a setof/3 around the goal setof(t, f(R),_).  Now everything is
broken.


?- p(true,X).
X = from_m_1 ;
X = from_m_2 ;
X = from_m_2.

?- make.
% /tmp/m.pl compiled into m 0.01 sec, 48 bytes
true.

?- p(true,X).
ERROR: fa_loop/5: Undefined procedure: f/1

_______________________________________________
SWI-Prolog mailing list
SWI-Prolog@...
https://mailbox.iai.uni-bonn.de/mailman/listinfo.cgi/swi-prolog

[Paulo Moura]

On 2009/09/22, at 20:32, Ulrich Neumerkel wrote:

> The more or less original intention of modules in Prolog was
> different.  The idea was to have "the same comfort" and even same
> speed as no module system plus "modularity".  Of course that cannot
> work out in general, since one point of modularity is to hide certain
> things - somewhere you will realize the seams.
>
> Further, programmers often do not completely understand what they want
> hidden and what not.

That sounds a bit too much of patronizing Prolog programmers for my  
taste. Deciding on which predicates should be public and which  
predicates should be private sometimes require a bit of  
experimentation but in the end is usually clear which predicates  
should be part of the module/object interface.

>  Even more so in Prolog, where you might model
> some interface, but on the other hand you also would like to allow
> meta-interpreters to inspect your program.  Similar problems are less
> evident in other languages where reflection is not so easily  
> accessible
> as in Prolog.

At least in Logtalk, is easy to write meta-interpreters that access  
private predicates without being necessary to change the predicate  
scope declarations to public.

> Just think of a very simple case of a meta program: a
> cross-referencer.  Here, you want to see the otherwise hidden
> dependencies of a module.  So you breach that information hiding
> promise.


Why should module dependencies be private in the first place? Module  
dependencies are relations between units of encapsulation and thus, by  
definition, public. In fact, comprehensive support for structural  
reflection requires public access to entity relations.

Cheers,

Paulo


-----------------------------------------------------------------
Paulo Jorge Lopes de Moura, PhD
Assistant Professor
Dep. of Computer Science, University of Beira Interior
6201-001 Covilhã, Portugal

Office 3.18  Ext. 3276
Phone: [phone] Fax: [phone]
Email: <mailto:pmoura@...>

Home page: <http://www.di.ubi.pt/~pmoura>
Research:  <http://logtalk.org/> Blog: <http://blog.logtalk.org/>
-----------------------------------------------------------------

_______________________________________________
SWI-Prolog mailing list
SWI-Prolog@...
https://mailbox.iai.uni-bonn.de/mailman/listinfo.cgi/swi-prolog

[Günter Kniesel]

Hi Ulrich,

Ulrich Neumerkel wrote:
 >> In my opinion, module_transparent/1 and context_module/1 are a
 >> simple and effective solution to a much more fundamental problem:
 >> dynamic binding.
 >
 > The more or less original intention of modules in Prolog was
 > different.  The idea was to have "the same comfort" and even same
 > speed as no module system plus "modularity".  Of course that cannot
 > work out in general, since one point of modularity is to hide certain
 > things - somewhere you will realize the seams.

When I wrote that module_transparent is in my opinion a solution to
dynamic binding I didn't mean that it was intended for that, just
that it (maybe accidentally) turns out the be one.

 > Further, programmers often do not completely understand what they want
 > hidden and what not.  Even more so in Prolog, where you might model
 > some interface, but on the other hand you also would like to allow
 > meta-interpreters to inspect your program.  Similar problems are less
 > evident in other languages where reflection is not so easily accessible
 > as in Prolog.
 >
 > Just think of a very simple case of a meta program: a
 > cross-referencer.  Here, you want to see the otherwise hidden
 > dependencies of a module.  So you breach that information hiding
 > promise.

Modules in Prolog are not an encapsulation mechanism, just a
namespace mechanism. They prevent accidental name clashes but
do not prevent deliberate access to the internals of a module
(one can always use the explicit module prefix to access also
predicates that are not exported). There is no information
hiding promise, just the promise not to overwhelm you with
everything that you don't want to see. ;)

 > You mentioned cases of dynamic binding using module_transparent can
 > be done equally well in a system with meta_predicate-only
 > declarations!  Simply add a further meta-argument and call such a
 > predicate just with an _ in the argument.
 >
 > :- meta_predicate inside(?,?,:).
 > inside(X, Y,  ContextModule:_) :-
 >
 > Now, the new argument will be a colon structure with the calling
 > module as prefix.  If you want to make this portable between SWI and
 > YAP (with Quintus style modules), access the module like so:
 >
 > inside(X, Y,  Context) :-
 >     strip_module(Context, ContextModule,_),
 >
 > In this manner you can be quite portable throughout many systems.  In
 > fact, this can be considered even ISO 13211-2 compliant.
 >
 > This is useful for lambda expressions run like this on YAP and SWI:
 >
 > http://www.complang.tuwien.ac.at/ulrich/Prolog-inedit/lambda.pl

Your absolutely right.

However, this approach has the disadvantage of having to change
the signature (number of arguments) of a predicate and hence
adapt all its clients -- which ranges between hard in non-trivial
applications and impossible if the clients are not under your control.
(E.g. if you are writing a library and don't want to break your
user's code).

 > But I agree with you that sometimes module_transparent/1 feels better.
 > However please note that there are many cases where completely
 > unexpected things happen!  Here is the worst I am aware of:
 >
 > :- module(m, [p/2]).
 >
 > :- module_transparent p/2.
 >
 > p(Goal,R) :-
 >    Goal,
 >    f(R).
 >
 > f(from_m_1).
 > f(from_m_2).
 > f(from_m_2).
 >
 >
 > From the user module, we get redundant solutions for p(true,R).  So we
 > wrap a setof/3 around the goal setof(t, f(R),_).  Now everything is
 > broken.
 >
 >
 > ?- p(true,X).
 > X = from_m_1 ;
 > X = from_m_2 ;
 > X = from_m_2.
 >
 > ?- make.
 > % /tmp/m.pl compiled into m 0.01 sec, 48 bytes
 > true.
 >
 > ?- p(true,X).
 > ERROR: fa_loop/5: Undefined procedure: f/1
 >

I'd like to understand better your point about the dangers of
module_transparent raised above. Where did you insert the setof
call? Do you mean

p(Goal,R) :-
    Goal,
    setof(t, f(R),_).

If yes, the problem is that setof is a metapredicate and that
SWI has an implicit rule that makes metapredicates dynamically
bound to the context module (whereas other predicates are not).
So setof and hence f(R) will now be executed in the context module
(leading to the complaint that there is no f/1 defined there),
which obviously wasn't your intention.

I'd like to thank you for this example because it nicely
illustrates the danger of letting the language determine what
is dynamically bound and what isn't binding. The only way to
avoid bad surprises is to state your intention explicitly,
like this:

p(Goal,R) :-
    Goal,
    m:setof(t, f(R),_).

By this simple change you say that you don't want the setof call
to be dynamically bound to the context module. Now everything
should work as intended, since setof should be executed in the
context of m.

?- m:setof(t, f(R),_).
R = from_m_1 ;
R = from_m_2.

Unfortunately, SWI doesn't, like my redefinition of p. No matter whether
I consult the edited file or use make, the m:setof(...) is replaced by
an unqualified setof, which I consider a plain bug:

?- make.

?- listing(p).
m:p(A, B) :-
        call(A),
        setof(t, f(B), _).

Could it be that m:setof(...) is translated to setof(...)
  as an optimization forgetting that this is only correct
  for statically bound non-metapredicates?

Günter




_______________________________________________
SWI-Prolog mailing list
SWI-Prolog@...
https://mailbox.iai.uni-bonn.de/mailman/listinfo.cgi/swi-prolog

[Paulo Moura]

On 2009/09/24, at 19:57, Günter Kniesel wrote:

> Modules in Prolog are not an encapsulation mechanism, just a
> namespace mechanism. They prevent accidental name clashes but
> do not prevent deliberate access to the internals of a module
> (one can always use the explicit module prefix to access also
> predicates that are not exported). There is no information
> hiding promise, just the promise not to overwhelm you with
> everything that you don't want to see. ;)


Modules fail miserably as a namespace mechanism. One of the reasons is  
the unfortunate mixup between loading and importing. Another one is  
the way module predicates are often imported and used. I have blogged  
in the past about these problems so please forgive me to not take the  
time to repeat the arguments here.

Cheers,

Paulo


-----------------------------------------------------------------
Paulo Jorge Lopes de Moura, PhD
Assistant Professor
Dep. of Computer Science, University of Beira Interior
6201-001 Covilhã, Portugal

Office 3.18  Ext. 3276
Phone: [phone] Fax: [phone]
Email: <mailto:pmoura@...>

Home page: <http://www.di.ubi.pt/~pmoura>
Research:  <http://logtalk.org/> Blog: <http://blog.logtalk.org/>
-----------------------------------------------------------------

_______________________________________________
SWI-Prolog mailing list
SWI-Prolog@...
https://mailbox.iai.uni-bonn.de/mailman/listinfo.cgi/swi-prolog