Bug Report: "make install" on Ubuntu should run "ldconfig" as final step

[dennisf486]

This build problem first occurred a couple years ago for me, and I just noticed the latest code I pulled from github still has the same issue.

Steps to reproduce the problem:
- git-clone latest Io code from github
- make all
- sudo make install
- Try to execute "io" (either the newly installed version or the one from _build/binaries, doesn't matter)
- Gives error saying it can't load the IoVM library.
- Try running "io_static" instead (io_static works)

Temporary workaround:
- after installing, run "sudo ldconfig"
- Now run "io", and see that it now works

I spent considerable time tracking this problem down last year.  I'm still not 100% clear what ldconfig actually *does* but it has something to do with linking version-dependent installed library file names with version-independent symbolic aliases for the libraries.  My clue was that when you install a normal .deb package, at the end the package says "ldconfig deferred processing now taking place".  It turns out you can just run ldconfig yourself at any time (you have to be root (sudo)).  AFAIK it's fairly innocuous and running when it isn't needed doesn't cause problems; it just doesn't do anything if it isn't needed.  Since you have to sudo to make install anyway, I would assume just adding "ldconfig" in the end of the makefile for make install will work perfectly.

Note that after the first time you run ldconfig after building Io from source, you can no longer reproduce the bug because the Linux system is already properly set up when you build from source a 2nd time.

[Jeremy Tregunna-2]

In my opinion this isn't the responsibility of io, nor any other  
library to do. It's not third party developers fault that GNU ld is  
"dumb" (I don't use that as derogatory, I use that as you have to tell  
it everything to do and when it do it).

Different platforms solve the problem differently, for instance, NeXT  
(and now Apple) dyld requires that when you link a library, you tell  
the linker where you're going to install it, and that information is  
stored in the resulting object file. This is a fragile solution, but  
there is "relative path" support, which handles probably 99% of the  
cases you'd want to handle, when static paths are of little use.

BSD ld uses a different approach still (though closer related to how  
GNU ld does it, but more how GNU ld should do it). It is also  
partially dumb, but what happens is, it'll scan its cache first, load  
paths into memory, then the runtime linker will check to find any  
library in those paths if it's not in its cache already.

Better than 99% of the time, BSD ld finds the appropriate library.

Now, all the systems are a little more complicated than I make it out  
to be, but the details are less important than the summary.

On 6-Jun-09, at 11:06 PM, dennisf486 wrote:

> This build problem first occurred a couple years ago for me, and I  
> just noticed the latest code I pulled from github still has the same  
> issue.
>
> Steps to reproduce the problem:
> - git-clone latest Io code from github
> - make all
> - sudo make install
> - Try to execute "io" (either the newly installed version or the one  
> from _build/binaries, doesn't matter)
> - Gives error saying it can't load the IoVM library.
> - Try running "io_static" instead (io_static works)
>
> Temporary workaround:
> - after installing, run "sudo ldconfig"
> - Now run "io", and see that it now works
>
> I spent considerable time tracking this problem down last year.  I'm  
> still not 100% clear what ldconfig actually *does* but it has  
> something to do with linking version-dependent installed library  
> file names with version-independent symbolic aliases for the  
> libraries.  My clue was that when you install a normal .deb package,  
> at the end the package says "ldconfig deferred processing now taking  
> place".  It turns out you can just run ldconfig yourself at any time  
> (you have to be root (sudo)).  AFAIK it's fairly innocuous and  
> running when it isn't needed doesn't cause problems; it just doesn't  
> do anything if it isn't needed.  Since you have to sudo to make  
> install anyway, I would assume just adding "ldconfig" in the end of  
> the makefile for make install will work perfectly.
>
> Note that after the first time you run ldconfig after building Io  
> from source, you can no longer reproduce the bug because the Linux  
> system is already properly set up when you build from source a 2nd  
> time.

Regards,

Jeremy Tregunna
jeremy.tregunna@...

[Steve Dekorte]

On 2009-06-06, at 8:06 PM, dennisf486 wrote:

> This build problem first occurred a couple years ago for me, and I  
> just noticed the latest code I pulled from github still has the same  
> issue.
>
> Steps to reproduce the problem:
> - git-clone latest Io code from github
> - make all
> - sudo make install
> - Try to execute "io" (either the newly installed version or the one  
> from _build/binaries, doesn't matter)
> - Gives error saying it can't load the IoVM library.
> - Try running "io_static" instead (io_static works)
>
> Temporary workaround:
> - after installing, run "sudo ldconfig"
> - Now run "io", and see that it now works

Hi Dennis,

Thanks for the issue report. Could you add something to the makefile  
to fix Ubuntu that doesn't break other systems? I'd be happy to accept  
a patch for that.

Steve

[Jeremy Tregunna-2]

On 7-Jun-09, at 6:38 PM, Steve Dekorte wrote:

>
> On 2009-06-06, at 8:06 PM, dennisf486 wrote:
>
>> This build problem first occurred a couple years ago for me, and I
>> just noticed the latest code I pulled from github still has the same
>> issue.
>>
>> Steps to reproduce the problem:
>> - git-clone latest Io code from github
>> - make all
>> - sudo make install
>> - Try to execute "io" (either the newly installed version or the one
>> from _build/binaries, doesn't matter)
>> - Gives error saying it can't load the IoVM library.
>> - Try running "io_static" instead (io_static works)
>>
>> Temporary workaround:
>> - after installing, run "sudo ldconfig"
>> - Now run "io", and see that it now works
>
>
> Hi Dennis,
>
> Thanks for the issue report. Could you add something to the makefile
> to fix Ubuntu that doesn't break other systems? I'd be happy to accept
> a patch for that.

That's a bad idea Steve, as it depends on "sudo" being installed, AND  
the user actually having privileges to run it. It would break  
installation on systems that don't meet those dependencies that run  
even Ubuntu.

My suggestion would be a notice in the build process on Linux/Hurd  
targets that says "Your platform may require you to run ldconfig after  
installation before the io binary will work as expected. Consult the  
ldconfig manual page for any information if you are unfamiliar with  
the standard procedure." or something like that.

Regards,

Jeremy Tregunna
jeremy.tregunna@...

[Guy Hulbert]

On Sun, 2009-07-06 at 18:51 -0400, Jeremy Tregunna wrote:
> That's a bad idea Steve, as it depends on "sudo" being installed, AND
> the user actually having privileges to run it.

iirc, "it" *already* depends on "sudo" ... I was going to "fix" it but
still have no round tuits ...

--
--gh

[Jeremy Tregunna-2]

On 7-Jun-09, at 7:04 PM, Guy Hulbert wrote:

> On Sun, 2009-07-06 at 18:51 -0400, Jeremy Tregunna wrote:
>> That's a bad idea Steve, as it depends on "sudo" being installed, AND
>> the user actually having privileges to run it.
>
> iirc, "it" *already* depends on "sudo" ... I was going to "fix" it but
> still have no round tuits ...

Actually, io does not depend on sudo. It recommends using it to  
install, as most users build io as an unprivileged user.

Regards,

Jeremy Tregunna
jeremy.tregunna@...

[Steve Dekorte]

On 2009-06-07, at 4:58 PM, Jeremy Tregunna wrote:
>> iirc, "it" *already* depends on "sudo" ... I was going to "fix" it  
>> but
>> still have no round tuits ...
>
> Actually, io does not depend on sudo. It recommends using it to
> install, as most users build io as an unprivileged user.

My main concern is with making Io easy to set up. If there is a  
security concern with sudo, perhaps we can prompt the user to ask if  
it is ok to use it first?

[Jeremy Tregunna-2]

On 7-Jun-09, at 7:58 PM, Jeremy Tregunna wrote:

>
> On 7-Jun-09, at 7:04 PM, Guy Hulbert wrote:
>
>> On Sun, 2009-07-06 at 18:51 -0400, Jeremy Tregunna wrote:
>>> That's a bad idea Steve, as it depends on "sudo" being installed,  
>>> AND
>>> the user actually having privileges to run it.
>>
>> iirc, "it" *already* depends on "sudo" ... I was going to "fix" it  
>> but
>> still have no round tuits ...
>
> Actually, io does not depend on sudo. It recommends using it to
> install, as most users build io as an unprivileged user.

There's not... It's just something a user has to actually have  
installed to work. I'd say conservatively, only about 40% of users  
actually have it installed, liberally, maybe 70%. While relatively  
common, it's still absent on a lot of machines.

Regards,

Jeremy Tregunna
jeremy.tregunna@...

[Guy Hulbert]

On Sun, 2009-07-06 at 19:58 -0400, Jeremy Tregunna wrote:

>
>
>
> On 7-Jun-09, at 7:04 PM, Guy Hulbert wrote:
>
> > On Sun, 2009-07-06 at 18:51 -0400, Jeremy Tregunna wrote:
> >> That's a bad idea Steve, as it depends on "sudo" being installed,
> AND
> >> the user actually having privileges to run it.
> >
> > iirc, "it" *already* depends on "sudo" ... I was going to "fix" it
> but
> > still have no round tuits ...
>
> Actually, io does not depend on sudo. It recommends using it to
> install, as most users build io as an unprivileged user.

Try: git clone <io-url>
cd <top-dir>
find -type f -exec grep 'sudo' '{}' \;

I don't believe you will find the output empty (it's in one of the
add-ons but a plain 'make' hits it.  I think I mentioned this on the
list and Steve said it was an external library which is a dependency
(but is not required on every platform -- i.e. if it's already
intstalled).

Did you notice that I quoted 'it' ?

>
> Regards,
>
> Jeremy Tregunna
> jeremy.tregunna@...
>
>
>
>
>

--
Guy Hulbert
gwhulbert@... (preferred)
work: [phone] (no voicemail)
cell: [phone] (voicemail)

[Jeremy Tregunna-2]

On 7-Jun-09, at 8:41 PM, Guy Hulbert wrote:

> On Sun, 2009-07-06 at 19:58 -0400, Jeremy Tregunna wrote:
>>
>>
>>
>> On 7-Jun-09, at 7:04 PM, Guy Hulbert wrote:
>>
>>> On Sun, 2009-07-06 at 18:51 -0400, Jeremy Tregunna wrote:
>>>> That's a bad idea Steve, as it depends on "sudo" being installed,
>> AND
>>>> the user actually having privileges to run it.
>>>
>>> iirc, "it" *already* depends on "sudo" ... I was going to "fix" it
>> but
>>> still have no round tuits ...
>>
>> Actually, io does not depend on sudo. It recommends using it to
>> install, as most users build io as an unprivileged user.
>
> Try: git clone <io-url>
> cd <top-dir>
> find -type f -exec grep 'sudo' '{}' \;

grep -ri "sudo" * shows:

addons/SGML/build.io:                   cmd := "cd addons/SGML/
source/" .. libSgmlDirectory .. ";" .. configure .. " make && echo  
\"sudo make install\" && sudo make install"

> I don't believe you will find the output empty (it's in one of the
> add-ons but a plain 'make' hits it.  I think I mentioned this on the
> list and Steve said it was an external library which is a dependency
> (but is not required on every platform -- i.e. if it's already
> intstalled).

Output isn't empty, but if you look, there's an "echo" before it,  
meaning, that it gets printed to the screen, rather than executed.

Regards,

Jeremy Tregunna
jeremy.tregunna@...

[Guy Hulbert]

On Sun, 2009-07-06 at 17:05 -0700, Steve Dekorte wrote:
> My main concern is with making Io easy to set up. If there is a
> security concern with sudo, perhaps we can prompt the user to ask if
> it is ok to use it first?

The (naive) GNU standard used to be:
        $ su - root
        # ./configure
        # make
        # make install
However you only need to be root for the final step and normally that's
only if you don't have write privilege.  Sun has a 'bin' user, which
owns executables and shared libraries (or they did on SunOS - haven't
used Solaris for a while).

Of course the "proper" way to build things is:
        $ ./configure
        $ make
        $ su - root
        # make install
if you try building RCS as root it won't let you.  It's the only package
I know of that is that careful.  

On debian /usr/local/ is group-writable by 'staff' so you never need
root -- but the Io build doesn't know that :-(  Unfortunately 'root'
has /usr/local/sbin first in it's path -- Debian has changed their
documentation (recently, at my request) to make this more clear.  So you
can't put anyone you don't trust in the 'staff' group.  It's still
better than running installs as 'root'.

--
--gh

[Guy Hulbert]

On Sun, 2009-07-06 at 20:46 -0400, Jeremy Tregunna wrote:
> Output isn't empty, but if you look, there's an "echo" before it,
> meaning, that it gets printed to the screen, rather than executed.

The 'echo' is new since I did this (last year).  When I ran the 'make',
I got a password prompt.  Perhaps Steve added the 'echo' since then.

--
--gh

[Kevin Edwards-2]

On 6/7/2009 7:51 PM, Guy Hulbert wrote:

> On Sun, 2009-07-06 at 20:46 -0400, Jeremy Tregunna wrote:
>> grep -ri "sudo" * shows:
>>
>> addons/SGML/build.io:                   cmd := "cd addons/SGML/
>> source/" .. libSgmlDirectory .. ";" .. configure .. " make && echo  
>> \"sudo make install\" && sudo make install"
>> [...]
>> Output isn't empty, but if you look, there's an "echo" before it,
>> meaning, that it gets printed to the screen, rather than executed.
>
> The 'echo' is new since I did this (last year).  When I ran the 'make',
> I got a password prompt.  Perhaps Steve added the 'echo' since then.
>

It looks like it echoes it and then executes it, no?

[Jeremy Tregunna-2]

On 7-Jun-09, at 10:10 PM, Kevin Edwards wrote:

> On 6/7/2009 7:51 PM, Guy Hulbert wrote:
>> On Sun, 2009-07-06 at 20:46 -0400, Jeremy Tregunna wrote:
>>> grep -ri "sudo" * shows:
>>>
>>> addons/SGML/build.io:                   cmd := "cd addons/SGML/
>>> source/" .. libSgmlDirectory .. ";" .. configure .. " make && echo
>>> \"sudo make install\" && sudo make install"
>>> [...]
>>> Output isn't empty, but if you look, there's an "echo" before it,
>>> meaning, that it gets printed to the screen, rather than executed.
>>
>> The 'echo' is new since I did this (last year).  When I ran the  
>> 'make',
>> I got a password prompt.  Perhaps Steve added the 'echo' since then.
>>
>
> It looks like it echoes it and then executes it, no?

The System system(cmd) is what actually executes the string stored in  
the cmd slot which contains the echo. So no, it does not appear to  
actually execute sudo, just echo sudo usage to the screen.

Regards,

Jeremy Tregunna
jeremy.tregunna@...

[Kevin Edwards-2]

On 6/7/2009 9:18 PM, Jeremy Tregunna wrote:

> On 7-Jun-09, at 10:10 PM, Kevin Edwards wrote:
>> On 6/7/2009 7:51 PM, Guy Hulbert wrote:
>>> On Sun, 2009-07-06 at 20:46 -0400, Jeremy Tregunna wrote:
>>>> grep -ri "sudo" * shows:
>>>>
>>>> addons/SGML/build.io:                   cmd := "cd addons/SGML/
>>>> source/" .. libSgmlDirectory .. ";" .. configure .. " make && echo
>>>> \"sudo make install\" && sudo make install"
>>>> [...]
>>>> Output isn't empty, but if you look, there's an "echo" before it,
>>>> meaning, that it gets printed to the screen, rather than executed.
>>>
>>> The 'echo' is new since I did this (last year).  When I ran the  
>>> 'make',
>>> I got a password prompt.  Perhaps Steve added the 'echo' since then.
>>>
>>
>> It looks like it echoes it and then executes it, no?
>
> The System system(cmd) is what actually executes the string stored in  
> the cmd slot which contains the echo. So no, it does not appear to  
> actually execute sudo, just echo sudo usage to the screen.
>

We're talking about the SGML/build.io cmd string above, right?  It
ends like this, which is sent to the system sh:

echo "sudo make install" && sudo make install

That doesn't execute sudo?

Kevin

[Jeremy Tregunna-2]

Sorry, missed that.

On 7-Jun-09, at 11:14 PM, Kevin Edwards wrote:

> On 6/7/2009 9:18 PM, Jeremy Tregunna wrote:
>> On 7-Jun-09, at 10:10 PM, Kevin Edwards wrote:
>>> On 6/7/2009 7:51 PM, Guy Hulbert wrote:
>>>> On Sun, 2009-07-06 at 20:46 -0400, Jeremy Tregunna wrote:
>>>>> grep -ri "sudo" * shows:
>>>>>
>>>>> addons/SGML/build.io:                   cmd := "cd addons/SGML/
>>>>> source/" .. libSgmlDirectory .. ";" .. configure .. " make && echo
>>>>> \"sudo make install\" && sudo make install"
>>>>> [...]
>>>>> Output isn't empty, but if you look, there's an "echo" before it,
>>>>> meaning, that it gets printed to the screen, rather than executed.
>>>>
>>>> The 'echo' is new since I did this (last year).  When I ran the
>>>> 'make',
>>>> I got a password prompt.  Perhaps Steve added the 'echo' since  
>>>> then.
>>>>
>>>
>>> It looks like it echoes it and then executes it, no?
>>
>> The System system(cmd) is what actually executes the string stored in
>> the cmd slot which contains the echo. So no, it does not appear to
>> actually execute sudo, just echo sudo usage to the screen.
>>
>
> We're talking about the SGML/build.io cmd string above, right?  It
> ends like this, which is sent to the system sh:
>
> echo "sudo make install" && sudo make install
>
> That doesn't execute sudo?
>
> Kevin
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>

Regards,

Jeremy Tregunna
jeremy.tregunna@...

[dennisf486]

--- In iolanguage@..., Jeremy Tregunna <jeremy.tregunna@...> wrote:
>
>
> That's a bad idea Steve, as it depends on "sudo" being installed, AND  
> the user actually having privileges to run it. It would break  
> installation on systems that don't meet those dependencies that run  
> even Ubuntu.

Sorry for the confusion what I meant is that (on Ubuntu) the user has already sudoed the whole make install command in the first place, so you wouldn't have to put "sudo ldconfig" in the makefile, just ldconfig.  We would of course have to check for what kind of OS the user is running to determine whether or not to run the ldconfig.  I guess Jeremy's objection is the possibility of the build logic running into the ldconfig code on a system which BOTH shows up as GNU/ld based and doesn't require super user privileges for the make install.

Couldn't we just also check that we have the root user id before attempting to run the ldconfig command?

The details aren't important; I'd be perfectly happy just to see it, after make install, echo "It looks like you're running operating system 'foo'.  On this type of system, you may need to run the ldconfig command (probably as root) in order for Io to find its dynamic libraries."  The issue is that a user might well be knowledgeable enough to download the source, build, and install it, yet at the end have no trail of clues to connect the fact that Io doesn't work on his system to an ldconfig command he's never heard of before.  If you don't want to risk breaking your build on other systems, a simple message should be safe enough.

I don't think it's productive to have the attitude "it's the system's (or the linker tool's) fault; it should be smarter."  That may well be true, but unfortunately, the user is not going to think their system is broken, they're going to think Io's broken.

[Guy Hulbert]

On Mon, 2009-08-06 at 04:04 +0000, dennisf486 wrote:
> Couldn't we just also check that we have the root user id before
> attempting to run the ldconfig command?

I would rather that the install software not be so helpful.  As I
pointed out, using sudo is, in principal, not required at all.
Moreover, anything done as root should be a separate step.

--
--gh

[Jeremy Tregunna-2]

On 8-Jun-09, at 12:04 AM, dennisf486 wrote:

> I don't think it's productive to have the attitude "it's the  
> system's (or the linker tool's) fault; it should be smarter."  That  
> may well be true, but unfortunately, the user is not going to think  
> their system is broken, they're going to think Io's broken.

Perhaps not, but over the last 16 years, I have answered this question  
to Linux users many times... so many, I lost count probably around  
1994... Simple fact of the matter is, software evolves -- it's time  
GNU ld did as well. If I didn't have to sign a copyright waiver, I'd  
submit a patch for more modern behaviour.

Regards,

Jeremy Tregunna
jeremy.tregunna@...