Compiling CLISP on ARM Linux - ffi problems

[Jamie Craig]

Hi,

I've been trying to get CLISP up and running on a Nokia N800 internet
tablet, which runs one of the usual little debian-based Linux
distributions. While I've had some success getting CLISP proper running
(configure settings and make check-recompile timings below) I can't get
it to build with FFI or any of the standard base modules. Can anyone
offer any suggestions?

The following might shed some light on the problems:
When I run configure (CFLAGS set to -DSAFETY=3 -mcpu=arm1136j-s, no
other configure options) mostly everything looks OK, but near the end of
the process, vacall-arm.s fails to assemble with:

make[1]: Entering directory `/root/clisp-2.41/src/callback/vacall_r'
/bin/sh ./libtool --mode=compile gcc -x none -c vacall-arm.s
 gcc -x none -c vacall-arm.s -o vacall-arm.o
vacall-arm.s: Assembler messages:
vacall-arm.s:3: Warning: ignoring attempt to redefine built-in register 'sl'
vacall-arm.s:3: Warning: ignoring attempt to redefine built-in register 'SL'
vacall-arm.s:4: Warning: ignoring attempt to redefine built-in register 'fp'
vacall-arm.s:4: Warning: ignoring attempt to redefine built-in register 'FP'
vacall-arm.s:5: Warning: ignoring attempt to redefine built-in register 'ip'
vacall-arm.s:5: Warning: ignoring attempt to redefine built-in register 'IP'
vacall-arm.s:6: Warning: ignoring attempt to redefine built-in register 'sp'
vacall-arm.s:6: Warning: ignoring attempt to redefine built-in register 'SP'
vacall-arm.s:7: Warning: ignoring attempt to redefine built-in register 'lr'
vacall-arm.s:7: Warning: ignoring attempt to redefine built-in register 'LR'
vacall-arm.s:8: Warning: ignoring attempt to redefine built-in register 'pc'
vacall-arm.s:8: Warning: ignoring attempt to redefine built-in register 'PC'
vacall-arm.s:77: Error: selected processor does not support `ldfeqs f0,[sp,#20]'
vacall-arm.s:81: Error: selected processor does not support `ldfeqd f0,[sp,#20]'
make[1]: *** [vacall-arm.lo] Error 1
make[1]: Leaving directory `/root/clisp-2.41/src/callback/vacall_r'
make: *** [all-subdirs] Error 2
Configure findings:
  FFI:        no (user requested: default)
  readline:   yes (user requested: default)
  libsigsegv: yes

This appears to be complaining about a pretty simple float-point load
instruction; later ARMs have different FPUs (or none, in the more common
case) but as far as I knew they all supported at least the instruction
set of the original ARMv2 FPU. Does anyone know of a working vacall-arm?

However, configure completes regardless, and it's happy from that point
to compile CLISP itself. Unfortunately, although it can compile the main
program no problem, and produces a pretty decent working version, it
can't compile any of the modules, all of which fail with syntax errors
in clisp.h. I'm not sure why this breaks; I gather clisp.h is
autogenerated, but presumably it's generated much the same regardless of
the fine details of the platform? In any case, I don't think I'm
expecting this to work too well until vacall works. For now, if I remove
the modules from the base modules list in the Makefile, then make clisp
etc. all work nicely.

For the platforms list, make check-recompile on a Nokia N800 Internet
tablet finishes successfully as:

ARM 1136J-S, 128M RAM, gcc 3.4.4, 339 seconds (306 user, 12 sys), all
running from SD card instead of internal flash.

CFLAGS are as above, no configure options, but anyone else trying it
will want to set the TMP environment variable to point to a space on
SD/disk, not leave it pointing to /tmp in core.

-- Jamie Craig
<jamie@...>

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

[Sam Steingold]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jamie Craig wrote:
>
> The following might shed some light on the problems:
> When I run configure (CFLAGS set to -DSAFETY=3 -mcpu=arm1136j-s, no

"./configure --with-debug" is the more standard way to get -DSAFETY=3
(if you really need it).
what happens when you drop it?

> make[1]: Entering directory `/root/clisp-2.41/src/callback/vacall_r'
> /bin/sh ./libtool --mode=compile gcc -x none -c vacall-arm.s
>  gcc -x none -c vacall-arm.s -o vacall-arm.o
> vacall-arm.s: Assembler messages:
> vacall-arm.s:3: Warning: ignoring attempt to redefine built-in register 'sl'
> vacall-arm.s:3: Warning: ignoring attempt to redefine built-in register 'SL'
> vacall-arm.s:4: Warning: ignoring attempt to redefine built-in register 'fp'
> vacall-arm.s:4: Warning: ignoring attempt to redefine built-in register 'FP'
> vacall-arm.s:5: Warning: ignoring attempt to redefine built-in register 'ip'
> vacall-arm.s:5: Warning: ignoring attempt to redefine built-in register 'IP'
> vacall-arm.s:6: Warning: ignoring attempt to redefine built-in register 'sp'
> vacall-arm.s:6: Warning: ignoring attempt to redefine built-in register 'SP'
> vacall-arm.s:7: Warning: ignoring attempt to redefine built-in register 'lr'
> vacall-arm.s:7: Warning: ignoring attempt to redefine built-in register 'LR'
> vacall-arm.s:8: Warning: ignoring attempt to redefine built-in register 'pc'
> vacall-arm.s:8: Warning: ignoring attempt to redefine built-in register 'PC'
> vacall-arm.s:77: Error: selected processor does not support `ldfeqs f0,[sp,#20]'
> vacall-arm.s:81: Error: selected processor does not support `ldfeqd f0,[sp,#20]'
> make[1]: *** [vacall-arm.lo] Error 1
> make[1]: Leaving directory `/root/clisp-2.41/src/callback/vacall_r'
> make: *** [all-subdirs] Error 2
> Configure findings:
>   FFI:        no (user requested: default)
>   readline:   yes (user requested: default)
>   libsigsegv: yes
>
> This appears to be complaining about a pretty simple float-point load
> instruction; later ARMs have different FPUs (or none, in the more common
> case) but as far as I knew they all supported at least the instruction
> set of the original ARMv2 FPU. Does anyone know of a working vacall-arm?

I know nothing about assembly, FPU &c, but I will happily accept a
working patch :-)

> However, configure completes regardless, and it's happy from that point
> to compile CLISP itself. Unfortunately, although it can compile the main
> program no problem, and produces a pretty decent working version, it
> can't compile any of the modules, all of which fail with syntax errors
> in clisp.h. I'm not sure why this breaks; I gather clisp.h is
> autogenerated, but presumably it's generated much the same regardless of
> the fine details of the platform? In any case, I don't think I'm
> expecting this to work too well until vacall works. For now, if I remove
> the modules from the base modules list in the Makefile, then make clisp
> etc. all work nicely.

actually, modules do not require FFI, so I would like to see the errors
you get from clisp.h (in fact, no base module uses FFI specifically
because CLISP supports more platforms than FFI does).

> ARM 1136J-S, 128M RAM, gcc 3.4.4, 339 seconds (306 user, 12 sys), all
> running from SD card instead of internal flash.

thanks.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGlNkMPp1Qsf2qnMcRAqm6AKCgnpsnyxj8XykTb0GuYemTLf4mfQCfbhRp
Q/7zKc70S6mBpbeLwps4FXk=
=oYs2
-----END PGP SIGNATURE-----

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

[Jamie Craig-2]

On Wed, Jul 11, 2007 at 09:20:12AM -0400, Sam Steingold wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Jamie Craig wrote:
> >
> > The following might shed some light on the problems:
> > When I run configure (CFLAGS set to -DSAFETY=3 -mcpu=arm1136j-s, no
>
> "./configure --with-debug" is the more standard way to get -DSAFETY=3
> (if you really need it).
> what happens when you drop it?

I get a binary that crashes (segfault) when it tries to compile one of
the first few LISP files. (defs? I'll check and get back to you on
that.) With -DSAFETY=3 it works nicely, so I wasn't inclined to blame
that on anything other than a slightly ropey platform generally. The
unix/PLATFORMS file suggests checking out the setjmp/longjmp stuff, but
I didn't see anything blindingly obviously wrong there. It's pretty hard
to read through, that said. I could easily be missing something - is
there anything specific I should be looking for in lispbibl.d?

> > Configure findings:
> >   FFI:        no (user requested: default)
> >   readline:   yes (user requested: default)
> >   libsigsegv: yes
> >
> > This appears to be complaining about a pretty simple float-point load
> > instruction; later ARMs have different FPUs (or none, in the more common
> > case) but as far as I knew they all supported at least the instruction
> > set of the original ARMv2 FPU. Does anyone know of a working vacall-arm?
>
> I know nothing about assembly, FPU &c, but I will happily accept a
> working patch :-)

I do know ARM integer assembly, but strangely have never used the FPU,
so I'm probably no real help there. Might be worth my while just
dropping float from vacall and seeing if it works more generally - even
a slightly broken FFI beats none.

> > expecting this to work too well until vacall works. For now, if I remove
> > the modules from the base modules list in the Makefile, then make clisp
> > etc. all work nicely.
>
> actually, modules do not require FFI, so I would like to see the errors
> you get from clisp.h (in fact, no base module uses FFI specifically
> because CLISP supports more platforms than FFI does).

OK, excellent. That should be something fixable, then.
The output from make when it tries compiling various bits of the modules
(much the same messages for all the modules) is:

make[1]: Entering directory `/root/clisp-2.41/src/syscalls'
gcc  -DSAFETY=3 -mcpu=arm1136j-s -W -Wswitch -Wcomment -Wpointer-arith
-Wimplicit -Wreturn-type -Wmissing-declarations -Wno-sign-compare -O2
-DUNICODE -I. -I.. -c calls.m.c -o calls.o
In file included from calls.c:20:
../clisp.h:255: error: syntax error before "obj"
../clisp.h: In function `check_uint_defaulted':
../clisp.h:255: warning: implicit declaration of function `missingp'
../clisp.h:255: error: `obj' undeclared (first use in this function)
../clisp.h:255: error: (Each undeclared identifier is reported only once
../clisp.h:255: error: for each function it appears in.)
../clisp.h:255: error: `defolt' undeclared (first use in this function)
../clisp.h:255: warning: implicit declaration of function `I_to_uint'
../clisp.h:255: warning: implicit declaration of function `check_uint'
calls.c: At top level:
calls.c:113: error: syntax error before "gcv_object_t"
calls.c:113: warning: no semicolon at end of struct or union
calls.c:119: warning: type defaults to `int' in declaration of
`_object_K2_char_term'
calls.c:119: warning: data definition has no type or storage class
calls.c:122: error: syntax error before "_object_K2_c_bind"
calls.c:122: warning: type defaults to `int' in declaration of
`_object_K2_c_bind'

<repeats with small variations in what's being declared>

clisp.h line 255 looks pretty inoffensive:

static inline unsigned int check_uint_defaulted (object obj, unsigned
int defolt) { return missingp(obj) ? defolt :
I_to_uint(check_uint(obj)); }

and it seems to be the word "object" that it objects to (although
there's an object* type used 4 lines earlier in

#define subr_rest_function_args  (uintC argcount, object*
rest_args_pointer)

that doesn't bother the compiler at all)

Any suggestions will be gratefully received.

-- Jamie Craig
<jamie@...>

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

[Sam Steingold]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jamie Craig wrote:
> On Wed, Jul 11, 2007 at 09:20:12AM -0400, Sam Steingold wrote:
>> Jamie Craig wrote:
>>> The following might shed some light on the problems:
>>> When I run configure (CFLAGS set to -DSAFETY=3 -mcpu=arm1136j-s, no
>> what happens when you drop it?
>
> I get a binary that crashes (segfault) when it tries to compile one of
> the first few LISP files. (defs? I'll check and get back to you on
> that.)

yes, defs - this is the first GC.
please try replacing SAFETY=3 with NO_GENERATIONAL_GC (see the first 50
lines of src/lispbibl.d).

> make[1]: Entering directory `/root/clisp-2.41/src/syscalls'
> gcc  -DSAFETY=3 -mcpu=arm1136j-s -W -Wswitch -Wcomment -Wpointer-arith
> -Wimplicit -Wreturn-type -Wmissing-declarations -Wno-sign-compare -O2
> -DUNICODE -I. -I.. -c calls.m.c -o calls.o
> In file included from calls.c:20:
> ../clisp.h:255: error: syntax error before "obj"
> clisp.h line 255 looks pretty inoffensive:
>
> static inline unsigned int check_uint_defaulted (object obj, unsigned
> int defolt) { return missingp(obj) ? defolt :
> I_to_uint(check_uint(obj)); }
>
> and it seems to be the word "object" that it objects to

indeed.
so, why isn't object defined?
you should have something like

typedef void * gcv_object_t;
typedef gcv_object_t object;

see src/genclisph.d and build/gen.lispbibl.c

> (although there's an object* type used 4 lines earlier in
>
> #define subr_rest_function_args  (uintC argcount, object*
> rest_args_pointer)
>
> that doesn't bother the compiler at all)

this is a CPP macro, the compiles does not see "object" type at this
point yet.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGlOxnPp1Qsf2qnMcRAk++AJ4wFo1ld0PKjObeZoM9Nffu05LOywCcCW+6
ejHKoxPRQD2tNyCACaJf9mk=
=cPe7
-----END PGP SIGNATURE-----

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

[Jamie Craig-2]

Hi,

On Wed, Jul 11, 2007 at 10:42:47AM -0400, Sam Steingold wrote:

> Jamie Craig wrote:
> > On Wed, Jul 11, 2007 at 09:20:12AM -0400, Sam Steingold wrote:
> >> Jamie Craig wrote:
> >>> The following might shed some light on the problems:
> >>> When I run configure (CFLAGS set to -DSAFETY=3 -mcpu=arm1136j-s, no
> >> what happens when you drop it?
> >
> > I get a binary that crashes (segfault) when it tries to compile one of
> > the first few LISP files. (defs? I'll check and get back to you on
> > that.)
>
> yes, defs - this is the first GC.
> please try replacing SAFETY=3 with NO_GENERATIONAL_GC (see the first 50
> lines of src/lispbibl.d).

Thanks, that worked nicely. What needs done to make the generational GC
work on a new platform? (this is a horribly RAM-starved platform,
bearing in mind that it's running full X11 etc. in 128M, so I can see
the GC having a fair bit of work to do. :) )

> > static inline unsigned int check_uint_defaulted (object obj, unsigned
> > int defolt) { return missingp(obj) ? defolt :
> > I_to_uint(check_uint(obj)); }
> >
> > and it seems to be the word "object" that it objects to
>
> indeed.
> so, why isn't object defined?
> you should have something like
>
> typedef void * gcv_object_t;
> typedef gcv_object_t object;
>
> see src/genclisph.d and build/gen.lispbibl.c

Thanks for this one too. Turned out that gen.lispbibl.c was generated
empty, which was caused by busybox's sed not liking the regexps in the
Makefile much. Oddly it ran OK, but deleted the contents of every line
from the output except the header #line tag. I just installed regular
GNU sed and it straightened that out with no further complications.

In fact, despite vacall-arm being broken, it managed to compile at least
enough of the FFI bits and pieces for readline to work (the module seems
to depend on the existence of it - certainly it didn't compile when
set up with configure --without-dynamic-ffi. The defpackage in
readline.lisp says it uses (:use "CL" "EXT" "FFI"))

So, for future reference for anyone else trying to get it going on
small ARM linux platforms:

CFLAGS="-DNO_GENERATIONAL_GC" and install GNU sed is all it takes
to get
it to work on the Nokia N800. There are no particular configure options
needed to make it work.

Oh, and without -DSAFETY we get:

Test passed.

real    5m28.941s
user    4m58.844s
sys     0m13.234s

Not too shabby - I guess debugging adds about 10 seconds to that time
normally.

Thanks again for your help,

-- Jamie Craig
<jamie@...>

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list

[Sam Steingold]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jamie Craig wrote:
>> please try replacing SAFETY=3 with NO_GENERATIONAL_GC (see the first 50
>> lines of src/lispbibl.d).
>
> Thanks, that worked nicely. What needs done to make the generational GC
> work on a new platform? (this is a horribly RAM-starved platform,
> bearing in mind that it's running full X11 etc. in 128M, so I can see
> the GC having a fair bit of work to do. :) )

Only Bruno can help you here...
All I know is what's written at the end of clisp/unix/PLATFORMS in the
"Hints for porting to new platforms" section.

> In fact, despite vacall-arm being broken, it managed to compile at least
> enough of the FFI bits and pieces for readline to work (the module seems
> to depend on the existence of it - certainly it didn't compile when
> set up with configure --without-dynamic-ffi. The defpackage in
> readline.lisp says it uses (:use "CL" "EXT" "FFI"))

apparently only floats do not work, and readline does not need them.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGl3jtPp1Qsf2qnMcRAgFQAJoCAqFPmQK9i1/dMn21r5SbNll+iACeOyP0
lCnrvlbcsn1KZSiiNAlQqZQ=
=wImN
-----END PGP SIGNATURE-----

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
clisp-list mailing list
clisp-list@...
https://lists.sourceforge.net/lists/listinfo/clisp-list