Default install location and site-packages

[Michael Foord-5]

Hello guys,

The msi installer installs by default into "C:\Program Files\IronPython
2.6". It also creates a "Lib\site-packages" folder.

Presumably the intention is that site-packages is for installed modules
/ packages, however "Program Files" is a special location and normal
users (Vista / Windows 7) *can't* create files there. This means that
distutils *must* be run with elevated permissions to install into this
location.

It doesn't work anyway because distutils attempts to bytecode-compile,
which unsurprisingly fails with IronPython - but that bug in distutils
will be fixed shortly.

I don't have an obvious solution (per user site-packages perhaps?) but
present the problem. Python circumvents this problem by *not* installing
into "Program Files".

All the best,

Michael

--
http://www.ironpythoninaction.com/

_______________________________________________
Users mailing list
Users@...
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[briancurtin]

On Tue, Oct 6, 2009 at 06:45, Michael Foord <fuzzyman@...> wrote:

Hello guys,

The msi installer installs by default into "C:\Program Files\IronPython 2.6". It also creates a "Lib\site-packages" folder.

Presumably the intention is that site-packages is for installed modules / packages, however "Program Files" is a special location and normal users (Vista / Windows 7) *can't* create files there. This means that distutils *must* be run with elevated permissions to install into this location.

It doesn't work anyway because distutils attempts to bytecode-compile, which unsurprisingly fails with IronPython - but that bug in distutils will be fixed shortly.

I don't have an obvious solution (per user site-packages perhaps?) but present the problem. Python circumvents this problem by *not* installing into "Program Files".

All the best,

Michael

FWIW, the first time I installed IronPython via MSI I just clicked through the installer without looking and was surprised to see that it went to Program Files (user error, I know). I actually had to do a file search to find it because, out of habit, I was expecting it at C:\IronPython.

_______________________________________________
Users mailing list
Users@...
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[Curt Hagenlocher]

In principle, allowing unprivileged users to install code into a location where it can unknowingly be accessed by privileged users is a security problem. A "per-user" approach is the right one.

On Tue, Oct 6, 2009 at 4:45 AM, Michael Foord <fuzzyman@...> wrote:

Hello guys,

The msi installer installs by default into "C:\Program Files\IronPython 2.6". It also creates a "Lib\site-packages" folder.

Presumably the intention is that site-packages is for installed modules / packages, however "Program Files" is a special location and normal users (Vista / Windows 7) *can't* create files there. This means that distutils *must* be run with elevated permissions to install into this location.

It doesn't work anyway because distutils attempts to bytecode-compile, which unsurprisingly fails with IronPython - but that bug in distutils will be fixed shortly.

I don't have an obvious solution (per user site-packages perhaps?) but present the problem. Python circumvents this problem by *not* installing into "Program Files".

All the best,

Michael

--
http://www.ironpythoninaction.com/

_______________________________________________
Users mailing list
Users@...
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

_______________________________________________
Users mailing list
Users@...
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[Michael Foord-5]

Curt Hagenlocher wrote:
> In principle, allowing unprivileged users to install code into a
> location where it can unknowingly be accessed by privileged users is a
> security problem. A "per-user" approach is the right one.

Unknowingly?

Michael

>
> On Tue, Oct 6, 2009 at 4:45 AM, Michael Foord
> <fuzzyman@... <mailto:fuzzyman@...>> wrote:
>
>     Hello guys,
>
>     The msi installer installs by default into "C:\Program
>     Files\IronPython 2.6". It also creates a "Lib\site-packages" folder.
>
>     Presumably the intention is that site-packages is for installed
>     modules / packages, however "Program Files" is a special location
>     and normal users (Vista / Windows 7) *can't* create files there.
>     This means that distutils *must* be run with elevated permissions
>     to install into this location.
>
>     It doesn't work anyway because distutils attempts to
>     bytecode-compile, which unsurprisingly fails with IronPython - but
>     that bug in distutils will be fixed shortly.
>
>     I don't have an obvious solution (per user site-packages perhaps?)
>     but present the problem. Python circumvents this problem by *not*
>     installing into "Program Files".
>
>     All the best,
>
>     Michael
>
>     --
>     http://www.ironpythoninaction.com/
>
>     _______________________________________________
>     Users mailing list
>     Users@... <mailto:Users@...>
>     http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Users mailing list
> Users@...
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>  

--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


_______________________________________________
Users mailing list
Users@...
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[Dino Viehland]

Michael wrote:
> I don't have an obvious solution (per user site-packages perhaps?) but
> present the problem. Python circumvents this problem by *not* installing
> into "Program Files".

I would actually say that CPython seems to circumvent this by allowing
users to write to its installation directory.  Interestingly it does not
allow modifying the existing files (e.g. I can't modify site.py w/ out
elevating to admin, just as I can't create files at C:\ w/o elevating
to admin).


_______________________________________________
Users mailing list
Users@...
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[Dino Viehland]

Michael wrote:
> Curt Hagenlocher wrote:
> > In principle, allowing unprivileged users to install code into a
> > location where it can unknowingly be accessed by privileged users is a
> > security problem. A "per-user" approach is the right one.
>
> Unknowingly?

I've just installed some software.  Installing that software required that
I elevate to admin and left that software in a typically global location
on my machine (either C:\... or C:\Program Files\...) where my normal user
account does not have writes to access.  

What's the least surprising - that the global location is now suddenly
writable or that the global location remains writable only be
administrators?



_______________________________________________
Users mailing list
Users@...
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[Michael Foord-5]

Dino Viehland wrote:

> Michael wrote:
>  
>> Curt Hagenlocher wrote:
>>    
>>> In principle, allowing unprivileged users to install code into a
>>> location where it can unknowingly be accessed by privileged users is a
>>> security problem. A "per-user" approach is the right one.
>>>      
>> Unknowingly?
>>    
>
> I've just installed some software.  Installing that software required that
> I elevate to admin and left that software in a typically global location
> on my machine (either C:\... or C:\Program Files\...) where my normal user
> account does not have writes to access.  
>
> What's the least surprising - that the global location is now suddenly
> writable or that the global location remains writable only be
> administrators?
>  

Your answer seems orthogonal to the question I asked.

As you answered my question with a question perhaps I can do the same:

A user has an installed version of Python and an installed version of
IronPython. He wishes to install a library for both IronPython and
Python so he runs:

    python setup.py install
    ipy.exe setup.py install

The first succeeds, naturally. Are you saying that it would be *more*
surprising if the second succeeded?

Unfortunately at the moment it fails silently, but an "access denied"
error would not be much more helpful.

All the best,

Michael
>
>
> _______________________________________________
> Users mailing list
> Users@...
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>  


--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


_______________________________________________
Users mailing list
Users@...
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[Michael Foord-5]

Dino Viehland wrote:

> Michael wrote:
>  
>> I don't have an obvious solution (per user site-packages perhaps?) but
>> present the problem. Python circumvents this problem by *not* installing
>> into "Program Files".
>>    
>
> I would actually say that CPython seems to circumvent this by allowing
> users to write to its installation directory.  Interestingly it does not
> allow modifying the existing files (e.g. I can't modify site.py w/ out
> elevating to admin, just as I can't create files at C:\ w/o elevating
> to admin).
>  

Ignoring for the moment that distutils doesn't work yet, it seems to me
to be a "bug" (or at the very least a problem) that a normal user can't
install packages into their standard installation.

If Python libraries write or modify files in their package directory
then elevating on install wouldn't be enough - elevation would be
required to use them. I don't have any metrics on how common that is,
but my guess would be "not uncommon".

Does IronPython implement PEP 370?

    http://www.python.org/dev/peps/pep-0370/


Michael

>
> _______________________________________________
> Users mailing list
> Users@...
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>  


--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


_______________________________________________
Users mailing list
Users@...
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[Michael Foord-5]

Dino Viehland wrote:

> Michael wrote:
>  
>> Curt Hagenlocher wrote:
>>    
>>> In principle, allowing unprivileged users to install code into a
>>> location where it can unknowingly be accessed by privileged users is a
>>> security problem. A "per-user" approach is the right one.
>>>      
>> Unknowingly?
>>    
>
> I've just installed some software.  Installing that software required that
> I elevate to admin and left that software in a typically global location
> on my machine (either C:\... or C:\Program Files\...) where my normal user
> account does not have writes to access.  
>
> What's the least surprising - that the global location is now suddenly
> writable or that the global location remains writable only be
> administrators?
>
>  

As a user I probably don't care (and won't even check) whether a
sub-folder in the install location is now writable. What I *do* care
about is whether that software *works* - and an access denied error on
using aspects of that software *will* 'surprise' me, yes. :-)

Another way of phrasing the question - does writability of a sub-folder
in the IronPython install folder *trump* compatibility with CPython
behaviour?

Michael

>
> _______________________________________________
> Users mailing list
> Users@...
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>  


--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


_______________________________________________
Users mailing list
Users@...
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[Jeff Hardy-4]

On Tue, Oct 6, 2009 at 10:53 AM, Michael Foord
<fuzzyman@...> wrote:
> The first succeeds, naturally. Are you saying that it would be *more*
> surprising if the second succeeded?

It should be surprising - a limited user should *never* be able to
install software into a shared location. The fact that it works for
CPython is a bug, not a feature. I would totally expect to have to
elevate to install a package; IIRC, it's the norm on *nix machines to
need to sudo to write to site-packages.

Limited users should only be able to write to a per-user site-packages
(I think this is what virtualenv is supposed to accomplish, but
virtualenv has always been a mystery to me). It's no different that a
limited user being able to install into the GAC.

- Jeff
_______________________________________________
Users mailing list
Users@...
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[Michael Foord-5]

Jeff Hardy wrote:

> On Tue, Oct 6, 2009 at 10:53 AM, Michael Foord
> <fuzzyman@...> wrote:
>  
>> The first succeeds, naturally. Are you saying that it would be *more*
>> surprising if the second succeeded?
>>    
>
> It should be surprising - a limited user should *never* be able to
> install software into a shared location. The fact that it works for
> CPython is a bug, not a feature. I would totally expect to have to
> elevate to install a package; IIRC, it's the norm on *nix machines to
> need to sudo to write to site-packages.
>  

You can try raising a bug against CPython for this, but I *imagine* you
will get short-shrift (although in the presence of PEP 370 I am willing
to be surprised).

So the question is, how important is CPython compatibility in regard to
this behaviour?

Perhaps IronPython could make PEP 370 site-packages the *default*?

Michael

> Limited users should only be able to write to a per-user site-packages
> (I think this is what virtualenv is supposed to accomplish, but
> virtualenv has always been a mystery to me). It's no different that a
> limited user being able to install into the GAC.
>
> - Jeff
> _______________________________________________
> Users mailing list
> Users@...
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>  

--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


_______________________________________________
Users mailing list
Users@...
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[Dino Viehland]

Michael wrote:

> Dino Viehland wrote:
> > Michael wrote:
> >
> >> Curt Hagenlocher wrote:
> >>
> >>> In principle, allowing unprivileged users to install code into a
> >>> location where it can unknowingly be accessed by privileged users is a
> >>> security problem. A "per-user" approach is the right one.
> >>>
> >> Unknowingly?
> >>
> >
> > I've just installed some software.  Installing that software required that
> > I elevate to admin and left that software in a typically global location
> > on my machine (either C:\... or C:\Program Files\...) where my normal user
> > account does not have writes to access.
> >
> > What's the least surprising - that the global location is now suddenly
> > writable or that the global location remains writable only be
> > administrators?
> >
>
> Your answer seems orthogonal to the question I asked.
>
> As you answered my question with a question perhaps I can do the same:
>
> A user has an installed version of Python and an installed version of
> IronPython. He wishes to install a library for both IronPython and
> Python so he runs:
>
>     python setup.py install
>     ipy.exe setup.py install
>
> The first succeeds, naturally. Are you saying that it would be *more*
> surprising if the second succeeded?
>
> Unfortunately at the moment it fails silently, but an "access denied"
> error would not be much more helpful.

My point is simply that if a user is surprised by the fact that we've created
a globally writable place that effects the code they're running then they have
unknowingly done this.  Or another way to put this is any decision which leads
to less security shouldn't ever be a surprise to the user.

I'll agree that the difference between CPython and IronPython may be
surprising to someone who is used to CPython.  But it seems like CPython is
the one who's doing something wrong here.  I just checked on a Linux machine
and there CPython is behaving like we are:

dinov@sh0:/usr/lib/python2.5/site-packages$ ls
apt          aptsources                  python-support.pth
apt_inst.so  debconf.py                  README
apt_pkg.so   python_apt-0.6.17.egg-info  unattended_upgrades-0.1.egg-info
dinov@sh0:/usr/lib/python2.5/site-packages$ cp apt_inst.so xx
cp: cannot create regular file `xx': Permission denied
dinov@sh0:/usr/lib/python2.5/site-packages$


_______________________________________________
Users mailing list
Users@...
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[Giles Thomas]

Dino Viehland wrote:
> But it seems like CPython is the one who's doing something wrong here.
>  
Another data point; easy_install under CPython using Vista with UAC
switched on tries to escalate permissions as you would expect -- the
normal grey screen, "please enter an administrator's details" thing --
and then happily installs the egg under C:\PythonXX\lib\site-packages,
owned by Administrator, with no read permissions for anyone else.  So
you then have to go and find it and manually change the permissions if
you want to use it.

Somewhat orthogonal, but it does make it sound rather like the setup
with CPython is accidental rather than by deliberate choice.


Cheers,

Giles

--
Giles Thomas
giles.thomas@...
[phone]

17a Clerkenwell Road, London EC1M 5RD, UK
VAT No.: GB 893 5643 79
Registered in England and Wales as company number 5467329.
Registered address: 843 Finchley Road, London NW11 8NA, UK


_______________________________________________
Users mailing list
Users@...
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[Michael Foord-5]

Giles Thomas wrote:

> Dino Viehland wrote:
>> But it seems like CPython is the one who's doing something wrong here.
>>  
> Another data point; easy_install under CPython using Vista with UAC
> switched on tries to escalate permissions as you would expect -- the
> normal grey screen, "please enter an administrator's details" thing --
> and then happily installs the egg under C:\PythonXX\lib\site-packages,
> owned by Administrator, with no read permissions for anyone else.  So
> you then have to go and find it and manually change the permissions if
> you want to use it.
>
> Somewhat orthogonal, but it does make it sound rather like the setup
> with CPython is accidental rather than by deliberate choice.
>

I doubt it can be accidental - I think the changed permissions would
have to be done deliberately in the installer?

Historically Windows has made much less of a distinction between user
and system installed programs. On the Mac I need to sudo to install into
my *system* site-packages but not into the site-packages of a user
installed version of Python.

I still see it as a question of usability rather than security. (I'm
honestly not sure how creating a writable directory is a security
issue?) If the default install location of IronPython makes installing
and using Python packages with IronPython impossible for non-elevated
users then that is an extreme misfeature.

Michael

>
> Cheers,
>
> Giles
>


--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


_______________________________________________
Users mailing list
Users@...
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[Giles Thomas]

Michael Foord wrote:
> (I'm honestly not sure how creating a writable directory is a security
> issue?)
I suspect people are thinking of an attack where an untrusted user
installs a package that looks like a normal one, but actually does
something nefarious like install a rootkit (and perhaps does what the
package is meant to do as well).  If the administrator then uses the
package, the machine is compromised.


Cheers,

Giles

--
Giles Thomas
giles.thomas@...
[phone]

17a Clerkenwell Road, London EC1M 5RD, UK
VAT No.: GB 893 5643 79
Registered in England and Wales as company number 5467329.
Registered address: 843 Finchley Road, London NW11 8NA, UK


_______________________________________________
Users mailing list
Users@...
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[Dino Viehland]

Michael wrote:
> I still see it as a question of usability rather than security. (I'm
> honestly not sure how creating a writable directory is a security
> issue?) If the default install location of IronPython makes installing
> and using Python packages with IronPython impossible for non-elevated
> users then that is an extreme misfeature.

This is the security problem.  Let's say I, a normal user, goes into
C:\Python26\Lib\site-packages and creates or modifies sitecustomize.py.  
In sitecustomize.py I add some code like:

import os
if os.environ['USERNAME'] == 'Administrator':
    # install malware here, set myself as an administrator, format C,
    # etc...
    pass

Now I just sit back and wait for an administrator to start some program
which relies on Python.  I now have full control of a machine which I was
originally only granted normal user access on.





_______________________________________________
Users mailing list
Users@...
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[Michael Foord-5]

Dino Viehland wrote:

> Michael wrote:
>  
>> I still see it as a question of usability rather than security. (I'm
>> honestly not sure how creating a writable directory is a security
>> issue?) If the default install location of IronPython makes installing
>> and using Python packages with IronPython impossible for non-elevated
>> users then that is an extreme misfeature.
>>    
>
> This is the security problem.  Let's say I, a normal user, goes into
> C:\Python26\Lib\site-packages and creates or modifies sitecustomize.py.  
> In sitecustomize.py I add some code like:
>
> import os
> if os.environ['USERNAME'] == 'Administrator':
>     # install malware here, set myself as an administrator, format C,
>     # etc...
>     pass
>
> Now I just sit back and wait for an administrator to start some program
> which relies on Python.  I now have full control of a machine which I was
> originally only granted normal user access on.
>
>
>  

Well, fair enough [1]. :-)

Except it may *still* leave distutils / package management basically
unusable for many people. That would still seem to be bad. I'd like to
work on making Distribute (the successor to setuptools) compatible with
IronPython but it is going to require a working distutils system.

Can PEP 370 style site-packages be made the default for IronPython?

Michael

[1] I don't have this problem on the Mac. I have a system installed
Python that I must sudo to modify and a user installed one that I don't.
Even a user installed IronPython wouldn't have write permissions in the
normal site-packages folder on Windows, right?

>
>
> _______________________________________________
> Users mailing list
> Users@...
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>  


--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


_______________________________________________
Users mailing list
Users@...
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[Dino Viehland]

Michael wrote:

> Well, fair enough [1]. :-)
>
> Except it may *still* leave distutils / package management basically
> unusable for many people. That would still seem to be bad. I'd like to
> work on making Distribute (the successor to setuptools) compatible with
> IronPython but it is going to require a working distutils system.
>
> Can PEP 370 style site-packages be made the default for IronPython?
>
> Michael
>
> [1] I don't have this problem on the Mac. I have a system installed
> Python that I must sudo to modify and a user installed one that I don't.
> Even a user installed IronPython wouldn't have write permissions in the
> normal site-packages folder on Windows, right?

>From the IronPython side of things I think we should make our installer
have an option to install into a per-user directory.  That is certainly
missing today and would allow us to even have an elevation free installer
(although we couldn't ngen in that case but we might be able to offer the
user an opportunity to ngen even if they do a per-user install if they're
willing to elevate).

Do you have an idea of how to go about making it the default?  It looks
like PEP 370 style site-packages work today - although there might be
some tweaking we want to do.  It looks like if you create
%APPDATA%\Python\Python26\site-packages that site.py will pick it up even
on IronPython.  The only downside to this seems to  be that we share the
same directory as CPython per-user site packages.  But I'm not sure how
we would make that the default location to install to.

We should probably make the "Python" part of the directory be Python,
IronPython, Jython, etc... depending upon what implementation you're
running on.

_______________________________________________
Users mailing list
Users@...
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[Michael Foord-5]

Dino Viehland wrote:

> Michael wrote:
>  
>> Well, fair enough [1]. :-)
>>
>> Except it may *still* leave distutils / package management basically
>> unusable for many people. That would still seem to be bad. I'd like to
>> work on making Distribute (the successor to setuptools) compatible with
>> IronPython but it is going to require a working distutils system.
>>
>> Can PEP 370 style site-packages be made the default for IronPython?
>>
>> Michael
>>
>> [1] I don't have this problem on the Mac. I have a system installed
>> Python that I must sudo to modify and a user installed one that I don't.
>> Even a user installed IronPython wouldn't have write permissions in the
>> normal site-packages folder on Windows, right?
>>    
>
> >From the IronPython side of things I think we should make our installer
> have an option to install into a per-user directory.  That is certainly
> missing today and would allow us to even have an elevation free installer
> (although we couldn't ngen in that case but we might be able to offer the
> user an opportunity to ngen even if they do a per-user install if they're
> willing to elevate).
>
> Do you have an idea of how to go about making it the default?  It looks
> like PEP 370 style site-packages work today - although there might be
> some tweaking we want to do.  It looks like if you create
> %APPDATA%\Python\Python26\site-packages that site.py will pick it up even
> on IronPython.  The only downside to this seems to  be that we share the
> same directory as CPython per-user site packages.  But I'm not sure how
> we would make that the default location to install to.
>
>  

I'll look into this with the distutils guys.

> We should probably make the "Python" part of the directory be Python,
> IronPython, Jython, etc... depending upon what implementation you're
> running on.
>  

Yes, agreed - IronPython should have a separate user site-packages from
CPython.

Michael

> _______________________________________________
> Users mailing list
> Users@...
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
>  


--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


_______________________________________________
Users mailing list
Users@...
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com

[Jeff Hardy-4]

On Tue, Oct 6, 2009 at 11:53 AM, Giles Thomas
<giles.thomas@...> wrote:

> Michael Foord wrote:
>>
>> (I'm honestly not sure how creating a writable directory is a security
>> issue?)
>
> I suspect people are thinking of an attack where an untrusted user installs
> a package that looks like a normal one, but actually does something
> nefarious like install a rootkit (and perhaps does what the package is meant
> to do as well).  If the administrator then uses the package, the machine is
> compromised.

Exactly. And Python doesn't have codesigning or such to prevent such an attack.

For desktops it might not seem like a big deal, but for servers it's
an absolute disaster. It's better if it's not even possible.

- Jeff
_______________________________________________
Users mailing list
Users@...
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com