Compilation problems on Ubuntu 64-bit

[Marcus Clyne]

Hi,

(I'm posting this one again, because I think I might have been confusing
when asking people to ignore my issue - which was for a different problem.)


I'm trying to create a shared object file calling GCC using execvp, but
am getting some errors.

I'm forking, setting the env PATH and calling execvp (bin, argv);
(twice) where bin is a char* to "gcc" and argv in each case is a pointer
to a null-terminated array of char*'s - the options - given below.


First - .o creation, using the following options:

---------------------
-fPIC
-o
/tmp/simpl-so/objs/10.o
-I
/simpl/dev/src
-I
/simpl/dev/src/core
-I
/simpl/dev/src/include
-c
/t/bufout.c
-------------------------

This seems to go ok, then calling with the following options:
-------------------------
-v
-shared
-o
/tmp/simpl-so/objs/10.so
/tmp/simpl-so/objs/10.o
-lsimpl
--------------------------
produces:
============
/usr/bin/ld: /tmp/simpl-so/objs/10.o: relocation R_X86_64_32 against `a
local symbol' can not be used when making a shared object; recompile
with -fPIC
/tmp/simpl-so/objs/10.o: could not read symbols: Bad value
collect2: ld returned 1 exit status
============
whilst
---------------------------
-fPIC
-v
-shared
-o
/tmp/simpl-so/objs/10.so
/tmp/simpl-so/objs/10.o
-lsimpl
-----------------------------
produces:
============

Using built-in specs.
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu
4.3.3-5ubuntu4' --with-bugurl=file:///usr/share/doc/gcc-4.3/README.Bugs
--enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr
--enable-shared --with-system-zlib --libexecdir=/usr/lib
--without-included-gettext --enable-threads=posix --enable-nls
--with-gxx-include-dir=/usr/include/c++/4.3 --program-suffix=-4.3
--enable-clocale=gnu --enable-libstdcxx-debug --enable-objc-gc
--enable-mpfr --with-tune=generic --enable-checking=release
--build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.3.3 (Ubuntu 4.3.3-5ubuntu4)
COMPILER_PATH=../lib/gcc/x86_64-linux-gnu/4.3.3/:../lib/gcc/:/usr/lib/gcc/x86_64-linux-gnu/4.3.3/:/usr/lib/gcc/x86_64-linux-gnu/

LIBRARY_PATH=../lib/gcc/x86_64-linux-gnu/4.3.3/:../lib/gcc/:/usr/lib/gcc/x86_64-linux-gnu/4.3.3/:../lib/gcc/x86_64-linux-gnu/4.3.3/../../../../lib/:/lib/../lib/:/usr/lib/../lib/:../lib/gcc/x86_64-linux-gnu/4.3.3/../../../:/lib/:/usr/lib/

COLLECT_GCC_OPTIONS='-v' '-shared' '-o' '/tmp/simpl-so/objs/10.so'
'-mtune=generic'
../lib/gcc/x86_64-linux-gnu/4.3.3/collect2 --eh-frame-hdr -m elf_x86_64
--hash-style=both -shared -o /tmp/simpl-so/objs/10.so -z relro
../lib/gcc/x86_64-linux-gnu/4.3.3/../../../../lib/crti.o
../lib/gcc/x86_64-linux-gnu/4.3.3/crtbeginS.o
-L../lib/gcc/x86_64-linux-gnu/4.3.3 -L../lib/gcc
-L/usr/lib/gcc/x86_64-linux-gnu/4.3.3
-L../lib/gcc/x86_64-linux-gnu/4.3.3/../../../../lib -L/lib/../lib
-L/usr/lib/../lib -L../lib/gcc/x86_64-linux-gnu/4.3.3/../../..
/tmp/simpl-so/objs/10.o -lsimpl -lgcc --as-needed -lgcc_s --no-as-needed
-lc -lgcc --as-needed -lgcc_s --no-as-needed
../lib/gcc/x86_64-linux-gnu/4.3.3/crtendS.o
../lib/gcc/x86_64-linux-gnu/4.3.3/../../../../lib/crtn.o
/usr/bin/ld: /tmp/simpl-so/objs/10.o: relocation R_X86_64_32 against `a
local symbol' can not be used when making a shared object; recompile
with -fPIC
/tmp/simpl-so/objs/10.o: could not read symbols: Bad value
collect2: ld returned 1 exit status
===============

libsimpl is compiled using fPIC too.

I'm using Ubuntu 9.04 64-bit.  It compiled ok on the 32-bit version.

Can anyone give any guidance on how to solve this?

Thanks,

Marcus Clyne.

[Ian Lance Taylor-3]

Marcus Clyne <maccaday@...> writes:

> I'm trying to create a shared object file calling GCC using execvp,
> but am getting some errors.

Do you get the same errors when you just run gcc from the command line?

If not, then you need to figure out what you are doing differently in
your invocation of execvp.

Ian

[Marcus Clyne]

Hi,

Thanks for the previous responses.  I'm now getting some different
errors, which are only displayed when using execv:


../lib/gcc/x86_64-linux-gnu/4.3.3/../../../../lib/crt1.o: In function
`_start':
/build/buildd/glibc-2.9/csu/../sysdeps/x86_64/elf/start.S:109: undefined
reference to `main'
/usr/local/lib/libsimpl.so: undefined reference to `ngx_http_core_module'
/usr/local/lib/libsimpl.so: undefined reference to `ngx_http_script_compile'
/usr/local/lib/libsimpl.so: undefined reference to `pcre_free'
/usr/local/lib/libsimpl.so: undefined reference to `ngx_palloc'
/usr/local/lib/libsimpl.so: undefined reference to `ngx_http_add_variable'
/usr/local/lib/libsimpl.so: undefined reference to `ngx_parse_size'
/usr/local/lib/libsimpl.so: undefined reference to `ngx_pcalloc'
/usr/local/lib/libsimpl.so: undefined reference to `simpl_request_handler'
/usr/local/lib/libsimpl.so: undefined reference to `ngx_array_create'
/usr/local/lib/libsimpl.so: undefined reference to `ngx_http_script_run'
/usr/local/lib/libsimpl.so: undefined reference to `ngx_parse_time'
/usr/local/lib/libsimpl.so: undefined reference to `pcre_malloc'
/usr/local/lib/libsimpl.so: undefined reference to `ngx_parse_offset'
/usr/local/lib/libsimpl.so: undefined reference to `ngx_http_output_filter'
/usr/local/lib/libsimpl.so: undefined reference to
`ngx_http_variable_null_value'
/usr/local/lib/libsimpl.so: undefined reference to `ngx_core_module'
/usr/local/lib/libsimpl.so: undefined reference to `ngx_conf_log_error'
/usr/local/lib/libsimpl.so: undefined reference to `ngx_strcasecmp'
/usr/local/lib/libsimpl.so: undefined reference to
`ngx_http_get_variable_index'
/usr/local/lib/libsimpl.so: undefined reference to `ngx_http_send_header'
/usr/local/lib/libsimpl.so: undefined reference to `ngx_log_error_core'
collect2: ld returned 1 exit status


I have tried compiling on the same system/setup using the shell, and it
works without displaying any errors or undefined references.

I have set the cwd to /usr/bin before calling execvp (I've also tried
with execv with the same results).

There are two things that are puzzling to me.  First the first two
lines, and secondly that undefined symbol references would appear, since
they aren't actually being used and the .so isn't being loaded.

Thanks in advance for any responses that can help me solve these.

Marcus.

[Marcus Clyne]

Hi,


Kai Ruottu wrote:
> Marcus Clyne wrote:
>
>> I'm trying to create a shared object file calling GCC using execvp,
>> but am getting some errors.
>
> Does it work directly from the shell?
Yes

>
>> First - .o creation, using the following options:
>>
>> ---------------------
>> -fPIC -o /tmp/simpl-so/objs/10.o -I/simpl/dev/src
>> -I/simpl/dev/src/core \
>> -I/simpl/dev/src/include -c /t/bufout.c
>> ---------------------
>
> Would the result be identical with these put into the usual command
> line, with the '.o' got via execvp?  Seen with 'size', 'objdump -p',
> 'readelf -h' etc.

No, though I'm not sure why.  The following results for readelf -h are:

(shell created)

ELF Header:
  Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00
  Class:                             ELF64
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              REL (Relocatable file)
  Machine:                           Advanced Micro Devices X86-64
  Version:                           0x1
  Entry point address:               0x0
  Start of program headers:          0 (bytes into file)
  Start of section headers:          1400 (bytes into file)
  Flags:                             0x0
  Size of this header:               64 (bytes)
  Size of program headers:           0 (bytes)
  Number of program headers:         0
  Size of section headers:           64 (bytes)
  Number of section headers:         13
  Section header string table index: 10

(execvp created)

ELF Header:                                                          
  Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00          
  Class:                             ELF64                          
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              REL (Relocatable file)
  Machine:                           Advanced Micro Devices X86-64
  Version:                           0x1
  Entry point address:               0x0
  Start of program headers:          0 (bytes into file)
  Start of section headers:          1384 (bytes into file)
  Flags:                             0x0
  Size of this header:               64 (bytes)
  Size of program headers:           0 (bytes)
  Number of program headers:         0
  Size of section headers:           64 (bytes)
  Number of section headers:         13
  Section header string table index: 10

>
>> This seems to go ok, then calling with the following options:
>> -------------------------
>> -fPIC -shared -o /tmp/simpl-so/objs/10.so \
> > /tmp/simpl-so/objs/10.o -lsimpl
>> -------------------------
>
> And what the 'manual' linking would tell? Does it succeed?
If you mean by calling gcc from the shell, yes, it does work.
>
>> COLLECT_GCC_OPTIONS='-v' '-shared' '-o' '/tmp/simpl-so/objs/10.so'
>> '-mtune=generic'
>>
>> libsimpl is compiled using fPIC too.
>
> And with the same '-mtune=generic' ?
Yes - I hadn't changed it, but I also set it to 'generic' explicitly,
and the new errors (which are different from the original errors) are
still the same - I put these in a separate post.
>
> -shared
>   Produce a shared object which can then be linked with other objects to
>   form an executable. Not all systems support this option. For
>   predictable results, you must also specify the same set of options
>   that were used to generate code (‘-fpic’, ‘-fPIC’, or model
>   suboptions) when you specify this option.
I'm actually creating so's that will be linked using dlopen etc - not
sure if this makes a difference or not.

In any case, it works ok when I use the shell but not when I use
execv(p) to call gcc.

Thanks,

Marcus.

[Marcus Clyne]

Update:

I've now tried removing the -lsimpl from the commands, but am still
getting the following error when calling using execv(p):

(on x86-64)

../lib/gcc/x86_64-linux-gnu/4.3.3/../../../../lib/crt1.o: In function
`_start':
/build/buildd/glibc-2.9/csu/../sysdeps/x86_64/elf/start.S:109: undefined
reference to `main'
collect2: ld returned 1 exit status

(on x86)

../lib/gcc/x86_64-linux-gnu/4.3.3/../../../../lib/crt1.o: In function
`_start':
/build/buildd/glibc-2.9/csu/../sysdeps/i386/elf/start.S:115: undefined
reference to `main'
collect2: ld returned 1 exit status

However, when compiling from the shell, it works no problem.

These were both attempted on Ubuntu 9.04.  My old computer had Ubuntu
8.04 on, and worked, and AFAIK I didn't change any code between
switching over to the new computer.

I've tried looking for whether this is an OS issue, but haven't found a
definitive answer yet.  The above errors are not related to any of my
code, but to that used by gcc/ld.

I think I need to add something to the environment, but am not sure what.

Any ideas would be gratefully received.

Marcus.


Marcus Clyne wrote:

> Hi,
>
>
> Kai Ruottu wrote:
>> Marcus Clyne wrote:
>>
>>> I'm trying to create a shared object file calling GCC using execvp,
>>> but am getting some errors.
>>
>> Does it work directly from the shell?
> Yes
>>
>>> First - .o creation, using the following options:
>>>
>>> ---------------------
>>> -fPIC -o /tmp/simpl-so/objs/10.o -I/simpl/dev/src
>>> -I/simpl/dev/src/core \
>>> -I/simpl/dev/src/include -c /t/bufout.c
>>> ---------------------
>>
>> Would the result be identical with these put into the usual command
>> line, with the '.o' got via execvp?  Seen with 'size', 'objdump -p',
>> 'readelf -h' etc.
> No, though I'm not sure why.  The following results for readelf -h are:
>
> (shell created)
>
> ELF Header:
>  Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00
>  Class:                             ELF64
>  Data:                              2's complement, little endian
>  Version:                           1 (current)
>  OS/ABI:                            UNIX - System V
>  ABI Version:                       0
>  Type:                              REL (Relocatable file)
>  Machine:                           Advanced Micro Devices X86-64
>  Version:                           0x1
>  Entry point address:               0x0
>  Start of program headers:          0 (bytes into file)
>  Start of section headers:          1400 (bytes into file)
>  Flags:                             0x0
>  Size of this header:               64 (bytes)
>  Size of program headers:           0 (bytes)
>  Number of program headers:         0
>  Size of section headers:           64 (bytes)
>  Number of section headers:         13
>  Section header string table index: 10
>
> (execvp created)
>
> ELF Header:                                                          
>  Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00          
>  Class:                             ELF64                          
>  Data:                              2's complement, little endian
>  Version:                           1 (current)
>  OS/ABI:                            UNIX - System V
>  ABI Version:                       0
>  Type:                              REL (Relocatable file)
>  Machine:                           Advanced Micro Devices X86-64
>  Version:                           0x1
>  Entry point address:               0x0
>  Start of program headers:          0 (bytes into file)
>  Start of section headers:          1384 (bytes into file)
>  Flags:                             0x0
>  Size of this header:               64 (bytes)
>  Size of program headers:           0 (bytes)
>  Number of program headers:         0
>  Size of section headers:           64 (bytes)
>  Number of section headers:         13
>  Section header string table index: 10
>
>>
>>> This seems to go ok, then calling with the following options:
>>> -------------------------
>>> -fPIC -shared -o /tmp/simpl-so/objs/10.so \
>> > /tmp/simpl-so/objs/10.o -lsimpl
>>> -------------------------
>>
>> And what the 'manual' linking would tell? Does it succeed?
> If you mean by calling gcc from the shell, yes, it does work.
>>
>>> COLLECT_GCC_OPTIONS='-v' '-shared' '-o' '/tmp/simpl-so/objs/10.so'
>>> '-mtune=generic'
>>>
>>> libsimpl is compiled using fPIC too.
>>
>> And with the same '-mtune=generic' ?
> Yes - I hadn't changed it, but I also set it to 'generic' explicitly,
> and the new errors (which are different from the original errors) are
> still the same - I put these in a separate post.
>>
>> -shared
>>   Produce a shared object which can then be linked with other objects to
>>   form an executable. Not all systems support this option. For
>>   predictable results, you must also specify the same set of options
>>   that were used to generate code (‘-fpic’, ‘-fPIC’, or model
>>   suboptions) when you specify this option.
> I'm actually creating so's that will be linked using dlopen etc - not
> sure if this makes a difference or not.
>
> In any case, it works ok when I use the shell but not when I use
> execv(p) to call gcc.
>
> Thanks,
>
> Marcus.
>

[Ian Lance Taylor-3]

Marcus Clyne <maccaday@...> writes:

> I think I need to add something to the environment, but am not sure what.

We don't know either.  I recommend running printenv to see what you have
in your environment, adding all of it to your execve call, verify that
it works, and then remove items one by one until it fails.

Ian

[Marcus Clyne]

I've tried adding all the elements in the environment, but it still
doesn't work. :-(

Is there any reason why gcc/ld called from exec functions would have
different options set (other than those relating to the environment)
compared to being called from within a shell?

Marcus.


Ian Lance Taylor wrote:

> Marcus Clyne <maccaday@...> writes:
>
>  
>> I think I need to add something to the environment, but am not sure what.
>>    
>
> We don't know either.  I recommend running printenv to see what you have
> in your environment, adding all of it to your execve call, verify that
> it works, and then remove items one by one until it fails.
>
> Ian
>
>  

[Marcus Clyne]

Hi,

I've got it to work, by adding a 'main' function to my shared object code.

I don't understand why I need to do this though.  I didn't think that
'main' functions were required in shared objects, and when I compile
using the shell, it compiles ok without one.  When I used the same code
on my old system, it worked without needing to add one, even in the exec
function executions.

Could this be a bug in GCC?  The version I'm using currently is 4.3.3.

Alternatively, is there another option I should add when using exec
functions to call GCC to prevent the need for adding 'main' functions?

Thanks,

Marcus.


Marcus Clyne wrote:

> I've tried adding all the elements in the environment, but it still
> doesn't work. :-(
>
> Is there any reason why gcc/ld called from exec functions would have
> different options set (other than those relating to the environment)
> compared to being called from within a shell?
>
> Marcus.
>
>
> Ian Lance Taylor wrote:
>> Marcus Clyne <maccaday@...> writes:
>>
>>  
>>> I think I need to add something to the environment, but am not sure
>>> what.
>>>    
>>
>> We don't know either.  I recommend running printenv to see what you have
>> in your environment, adding all of it to your execve call, verify that
>> it works, and then remove items one by one until it fails.
>>
>> Ian
>>
>>  
>
>

[Ian Lance Taylor-3]

Marcus Clyne <maccaday@...> writes:

> Is there any reason why gcc/ld called from exec functions would have
> different options set (other than those relating to the environment)
> compared to being called from within a shell?

The shell just calls exec functions itself, so, no.

Ian

[Ian Lance Taylor-3]

Marcus Clyne <maccaday@...> writes:

> I've got it to work, by adding a 'main' function to my shared object code.
>
> I don't understand why I need to do this though.  I didn't think that
> main' functions were required in shared objects, and when I compile
> using the shell, it compiles ok without one.  When I used the same
> code on my old system, it worked without needing to add one, even in
> the exec function executions.
>
> Could this be a bug in GCC?  The version I'm using currently is 4.3.3.
>
> Alternatively, is there another option I should add when using exec
> functions to call GCC to prevent the need for adding 'main' functions?

If gcc works as you expect when invoked from the shell, then I don't
think it can be a bug in gcc.  I think it must be a bug in the way that
you are invoking it from your program.  Or maybe you have a shell alias
or something, who knows.

There is nothing magic about exec.  The shell uses exec also.  gcc does
not behave differently when invoked via exec or via the shell--it can't.

Ian

[Marcus Clyne]

Ok,

Thanks for your help.

Marcus.


Ian Lance Taylor wrote:

> Marcus Clyne <maccaday@...> writes:
>
>  
>> I've got it to work, by adding a 'main' function to my shared object code.
>>
>> I don't understand why I need to do this though.  I didn't think that
>> main' functions were required in shared objects, and when I compile
>> using the shell, it compiles ok without one.  When I used the same
>> code on my old system, it worked without needing to add one, even in
>> the exec function executions.
>>
>> Could this be a bug in GCC?  The version I'm using currently is 4.3.3.
>>
>> Alternatively, is there another option I should add when using exec
>> functions to call GCC to prevent the need for adding 'main' functions?
>>    
>
> If gcc works as you expect when invoked from the shell, then I don't
> think it can be a bug in gcc.  I think it must be a bug in the way that
> you are invoking it from your program.  Or maybe you have a shell alias
> or something, who knows.
>
> There is nothing magic about exec.  The shell uses exec also.  gcc does
> not behave differently when invoked via exec or via the shell--it can't.
>
> Ian
>
>  

[Marcus Clyne]

Hi,

Problem solved : I needed to add a '-v' option to the options when
invoking GCC using exec functions, but not when using the shell.

Does anyone know why?

Thanks,

Marcus.