Wrapping external libraries in cint7

[A Navaei]

Hi,

Are there any improvements in cint7 for wrapping external libraries,
or is the old instruction in
http://root.cern.ch/viewcvs/trunk/doc/extlib.txt?root=cint still
valid?


-Ali

[Philippe Canal]

Hi,

Yes the instructions are still the same.

Cheers,
Philippe.

A Navaei wrote:

> Hi,
>
> Are there any improvements in cint7 for wrapping external libraries,
> or is the old instruction in
> http://root.cern.ch/viewcvs/trunk/doc/extlib.txt?root=cint still
> valid?
>
>
> -Ali
>
>  

[A Navaei]

Hi Philippe,

I understand that the instructions in the that link are still valid,
but perhaps there is a better and more updated way of importing
existing external libraries into cint? For example, that instruction
does not mention anything about dictionaries.

I am interested in cint wrapping with these criteria:

(a)  Importing existing libraries without any modifications
(obviously, the wrapped objects are *not* going to be of root object
types).

(b) With the use of stand-alone cint (that is, avoiding tools provided
in root, but the use of root tool kit is welcome as long as the output
does not depend on root).

(c) Performing the wrapping process in an automated way.

Here are some questions based on this:

- Is implementing the above currently feasible?  If not, what are the
main obstacles?

- I am not sure if 'wrapping' is the right term to be used for this,
after all cint is a 'c/c++' interpreter. The same way that a
python/java interpreter does not need to wrap a python/java compiled
code and can call the methods directly, can cint interpreter
*directly* call methods in a compiled shared/static library without
the use of dictionary generation etc?

- Are there any real-world examples of importing existing libraries
into stand-alone cint?


Thanks,

Ali


2009/1/7 Philippe Canal <pcanal@...>:

> Hi,
>
> Yes the instructions are still the same.
>
> Cheers,
> Philippe.
>
> A Navaei wrote:
>>
>> Hi,
>>
>> Are there any improvements in cint7 for wrapping external libraries,
>> or is the old instruction in
>> http://root.cern.ch/viewcvs/trunk/doc/extlib.txt?root=cint still
>> valid?
>>
>>
>> -Ali
>>
>>
>

[Philippe Canal]

Hi Ali,

The dictionary generation in CINT was actually designed/build to meet a)
and c) (and of
course b) :) ).

> - Is implementing the above currently feasible?  If not, what are the
> main obstacles?

So, yes it is possible and even easy.  If you look at the makefile
created by makecint, you
will see a set of makefile rule, to
   1) generate the dictionary file
   2) compile it.
   3) link it in a shared library
   4) link it in an executable.
To automate it in your own (makefile) framework, all you would need is
to use 1)
and make sure 2 and 3 or 4 happens correctly in your case.

 > - Are there any real-world examples of importing existing libraries
 > into stand-alone cint?

Yes.  For example cint/demo/gl (for the opengl library)  and
cint/cint/lib/qt (for th QT library).

> can cint interpreter
> *directly* call methods in a compiled shared/static library without
> the use of dictionary generation etc?

This has been implemented  and is working for C library;  We have also
started work
on doing the same for C++ library but have not completed the work yet.

 > (that is, avoiding tools provided in root, but the use of root tool
kit is
 > welcome as long as the output does not depend on root).

Actually ROOT has a tool that makes the interface with a library extremely
straightforward:
   root [] .L vendorLibrary.so
   root [] .L vendorHeader.h+
   root [] vendor_function_call();
where vendor_fuinction_call is one the function (and/or class) declared
in vendorHeader.h

Cheers,
Philippe.

A Navaei wrote:

> Hi Philippe,
>
> I understand that the instructions in the that link are still valid,
> but perhaps there is a better and more updated way of importing
> existing external libraries into cint? For example, that instruction
> does not mention anything about dictionaries.
>
> I am interested in cint wrapping with these criteria:
>
> (a)  Importing existing libraries without any modifications
> (obviously, the wrapped objects are *not* going to be of root object
> types).
>
> (b) With the use of stand-alone cint (that is, avoiding tools provided
> in root, but the use of root tool kit is welcome as long as the output
> does not depend on root).
>
> (c) Performing the wrapping process in an automated way.
>
> Here are some questions based on this:
>
> - Is implementing the above currently feasible?  If not, what are the
> main obstacles?
>
> - I am not sure if 'wrapping' is the right term to be used for this,
> after all cint is a 'c/c++' interpreter. The same way that a
> python/java interpreter does not need to wrap a python/java compiled
> code and can call the methods directly, can cint interpreter
> *directly* call methods in a compiled shared/static library without
> the use of dictionary generation etc?
>
> - Are there any real-world examples of importing existing libraries
> into stand-alone cint?
>
>
> Thanks,
>
> Ali
>
>
> 2009/1/7 Philippe Canal <pcanal@...>:
>  
>> Hi,
>>
>> Yes the instructions are still the same.
>>
>> Cheers,
>> Philippe.
>>
>> A Navaei wrote:
>>    
>>> Hi,
>>>
>>> Are there any improvements in cint7 for wrapping external libraries,
>>> or is the old instruction in
>>> http://root.cern.ch/viewcvs/trunk/doc/extlib.txt?root=cint still
>>> valid?
>>>
>>>
>>> -Ali
>>>
>>>
>>>      

[Severin Ecker-2]

Hi,

I'm trying to port Cint to some platforms we're using but which I'm
pretty sure don't have a working port lurking around anywhere (xbox
anyone?).
Now i do have some questions about how I'd best go for it (clearly this
is cint only, no root)

1) does cint work correctly with big endian machines or is this my first
show stopper anyway?
2) do i have to use a specific predefine for big endian machines
3) I've seen checks for _WIN32 et. al. throughout the code. i suppose
this means there's not one or two sites where all the platform specific
code is implemented? (e.g.: I've replaced the calls to getenv (there are
still some others to do) with G__getenv and implemented this depending
on whether the getenv function is available on the platform or not, but
this prevents me from simply updated the sources with the ones on SVN
since i have to change quite a large amount of code in many places) Are
there any plans to only use CRT and cint functions throughout the
project and have all platform specific (system calls..) implemented in
one or two dedicated files?
4) how do i best add a new platform? I've seen that for windows the code
is compiled with both the G__WIN32 and G__UNIX predefines. this seems
slightly counter-intuitive to me so I'm wondering if there's any
documentation or info around that briefly describes which predefines are
available and what they're used to.

hmm I think that's it for now.
Cheers and many many thanks in advance,
severin

ps.: I do have some weird performance drawbacks if i link cint
statically instead of dynamically (even worse, the app seems to hang on
shutdown). I'll try to nail the problem down and give you a note if I
find anything helpful and more specific.

[Axel Naumann]

Hi Severin,

On 2009-01-09 10:30, Severin Ecker wrote:
> I'm trying to port Cint to some platforms we're using but which I'm
> pretty sure don't have a working port lurking around anywhere (xbox
> anyone?).

I considered supporting XBox - but MS doesn't want to donate one to me ;-)

> 1) does cint work correctly with big endian machines or is this my first
> show stopper anyway?

We have used CINT on Solaris with PowerPC. So this is expected to work.

> 2) do i have to use a specific predefine for big endian machines

This should not be necessary.

> 3) I've seen checks for _WIN32 et. al. throughout the code. i suppose
> this means there's not one or two sites where all the platform specific
> code is implemented?

This is usually a posix / non-posix issue, as you mention below.

> (e.g.: I've replaced the calls to getenv (there are
> still some others to do) with G__getenv and implemented this depending
> on whether the getenv function is available on the platform or not, but
> this prevents me from simply updated the sources with the ones on SVN
> since i have to change quite a large amount of code in many places)

Of course the easiest way to solve this is by you contributing your patches.
:-) You're very welcome to send them in so we can review them!

> Are
> there any plans to only use CRT and cint functions throughout the
> project and have all platform specific (system calls..) implemented in
> one or two dedicated files?

We have no such plans, mainly because this is not on our list of most pressing
issues with CINT. But we are happy to consider any such patches!

> 4) how do i best add a new platform?

It very much depends on what the platform looks like. I have absolutely no
experience with the XBox environment, I don't even know whether it's posix or
not. I guess it's not, but who knows, maybe MS gives in at some point! :-)

> I've seen that for windows the code
> is compiled with both the G__WIN32 and G__UNIX predefines. this seems
> slightly counter-intuitive to me

Yes...

> so I'm wondering if there's any
> documentation or info around that briefly describes which predefines are
> available and what they're used to.

I'm afraid not. But there is a procedure that - so far - has worked pretty well:

1. You get the sources compiled. You start with the platform flags that are
closest to the one you have. You patch configure and the makefiles in build/
accordingly.
2. You append your platform to init.cxx (e.g. check for BSD). This should get
simple dictionaries building.
3. You append your platform to vararg.h. This should make the simple
dictionaries usable for ellipsis parameters.
4. You patch tools/makecint if needed (see make test).
5. If make test works (good enough for you) we would really appreciate if you
could contribute your changes to we upload them to the repository!

If you want to extract the platform specific parts and change the build system
you might want to contact s.zelzer@.... He suggested to change
the build system to something CMake based - which might help with porting to
new platforms.

> ps.: I do have some weird performance drawbacks if i link cint
> statically instead of dynamically (even worse, the app seems to hang on
> shutdown). I'll try to nail the problem down and give you a note if I
> find anything helpful and more specific.

Interesting! Yes, please let us know! And please shout for help if you get
stuck porting.

Cheers, Axel.

[Severin Ecker-2]

Hi,

right, I'm leaking memory (well cint is. I'm still working with cint7
since porting version 5 is quite a lot more difficult with all those
.dlls it's expecting and needing to work.. weird thing...)

so I suppose (taken from a sample) that this is correct

int main(int argc, char* argv[])
{
  G__init_cint("cint source/testscript.cpp");
  G__scratch_all();
  return 0;
}

and there aren't any other cleaning up functions needed right? well my
mem-tracker tells me

Total Unfreed: 13352 bytes

it seems not much but seeing that the leaks come from all over the place
there's definitely something amiss.

there are quite a few things that don't get cleaned up but here's a
small list that I've compiled together along with the stacktrace:

(function          file            line)

1)
Reflex::Any::Holder<G__RflxProperties>::Clone    any.h     149
Reflex::Any::Any                                                  
any.h        51
Reflex::Any::operator=                                          
any.h        86
Reflex::PropertyListImpl::AddProperty                  
propertylistimpl.h     309
Reflex::PropertyList::AddProperty                        
propertylist.h            345
Cint::I::G__set_properties<Reflex::Type,G__RflxProperties>  
fproto.h       656
Cint::I::G__get_properties                                                      
rflxutil.cxx    557
Cint::I::G__init_globals                                                            
init.cxx         582
G__scratch_upto_work                                                          
scrupto.cxx     644
G__scratch_all                                                                      
scrupto.cxx      1075

2)
Reflex::PropertyListImpl::AddProperty         propertylistimpl.h      308
Reflex::PropertyList::AddProperty                
propertylist.h           345
Cint::I::G__set_properties<Reflex::Type,G__RflxProperties>      
fproto.h    656
Cint::I::G__get_properties          rflxutil.cxx    557
Cint::I::G__init_globals               init.cxx         582
G__scratch_upto_work       scrupto.cxx       644
G__scratch_all                    scrupto.cxx      1075

3)
Cint::I::G__BuilderInfo::Build           rflxutil.cxx     2283
Cint::I::G__make_ifunctable            ifunc.cxx       1211
Cint::I::G__define_var                    decl.cxx        2136
Cint::I::G__exec_statement             parse.cxx      5500
G__loadfile                                    loadfile.cxx     2373
G__main                                       init.cxx           2001
G__init_cint                                  init.cxx          1295

I roughly counted more than 300 distinct leaks which seem not to be
concentrated in a specific function but a quick check tells me that it's
only reflex types that are leaked.

cheers,
severin