Freezing python files into executables

[Girish Venkatasubramanian]

Hello,
I have been using freeze.py on 32 bit linux distributions without a
problem. But recently I tried to do the same on RHEL5 x86_64 and ran
into some issues.

1) When I ran the script, I got
Error: needed directory /usr/lib/python2.4/config not found

2) Then I "yum install python-devel" which installed python-devel.i386
and python-devel.x86_64. Then when I tried freezing, it worked but
make barfed up a bunch of errors during ld like

/usr/bin/ld: warning: i386 architecture of input file
`/usr/lib/python2.4/config/libpython2.4.a(rangeobject.o)' is
incompatible with i386:x86-64 output

and

/usr/lib/python2.4/config/libpython2.4.a(posixmodule.o): In function
`posix_getcwd':
(.text+0x53a1): undefined reference to `PyEval_RestoreThread'

3) I tried uninstalling python-devel.i386 (retaining only the x86_64)
and freeze fails with the same error message as in 1

Any help with this is greatly appreciated.
Thanks
--
http://mail.python.org/mailman/listinfo/python-list

[Rami Chowdhury]

On Tue, 03 Nov 2009 11:15:39 -0800, Girish Venkatasubramanian  
<girishvs@...> wrote:

> Hello,
> I have been using freeze.py on 32 bit linux distributions without a
> problem. But recently I tried to do the same on RHEL5 x86_64 and ran
> into some issues.
>
> 1) When I ran the script, I got
> Error: needed directory /usr/lib/python2.4/config not found
>

I don't know anything about freeze.py but on 64-bit Red Hat distros (RHEL,  
Fedora, etc) it should be /usr/lib64/python2.4/config :-)


--
Rami Chowdhury
"Never attribute to malice that which can be attributed to stupidity" --  
Hanlon's Razor
408-597-7068 (US) / 07875-841-046 (UK) / 0189-245544 (BD)
--
http://mail.python.org/mailman/listinfo/python-list

[Girish Venkatasubramanian]

Hi Rami,
Thanks for pointing this out. I did see that point - but apart from
installing python-devel (which has created and populated
/usr/lib64/python2.4/...) I am not sure what I should do - is there
some setting in python where I can ask it to look at lib64 instead of
lib?
Thanks.

On Tue, Nov 3, 2009 at 11:21 AM, Rami Chowdhury
<rami.chowdhury@...> wrote:

> On Tue, 03 Nov 2009 11:15:39 -0800, Girish Venkatasubramanian
> <girishvs@...> wrote:
>
>> Hello,
>> I have been using freeze.py on 32 bit linux distributions without a
>> problem. But recently I tried to do the same on RHEL5 x86_64 and ran
>> into some issues.
>>
>> 1) When I ran the script, I got
>> Error: needed directory /usr/lib/python2.4/config not found
>>
>
> I don't know anything about freeze.py but on 64-bit Red Hat distros (RHEL,
> Fedora, etc) it should be /usr/lib64/python2.4/config :-)
>
>
> --
> Rami Chowdhury
> "Never attribute to malice that which can be attributed to stupidity" --
> Hanlon's Razor
> 408-597-7068 (US) / 07875-841-046 (UK) / 0189-245544 (BD)
>

--
http://mail.python.org/mailman/listinfo/python-list

[M.-A. Lemburg]

Rami Chowdhury wrote:

> On Tue, 03 Nov 2009 11:15:39 -0800, Girish Venkatasubramanian
> <girishvs@...> wrote:
>
>> Hello,
>> I have been using freeze.py on 32 bit linux distributions without a
>> problem. But recently I tried to do the same on RHEL5 x86_64 and ran
>> into some issues.
>>
>> 1) When I ran the script, I got
>> Error: needed directory /usr/lib/python2.4/config not found
>>
>
> I don't know anything about freeze.py but on 64-bit Red Hat distros
> (RHEL, Fedora, etc) it should be /usr/lib64/python2.4/config :-)

This sounds a lot like a missing Python devel RPM.

--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 03 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
--
http://mail.python.org/mailman/listinfo/python-list

[Girish Venkatasubramanian]

Hey Marc-Andre,
Ummm - I have installed python-devel.x86_64 and checked that the
/usr/lib64/python2.4/ is populated - anything else I can/shuld do to
check/ensure the the devel rpm is installed?
Thanks.

On Tue, Nov 3, 2009 at 11:32 AM, M.-A. Lemburg <mal@...> wrote:

> Rami Chowdhury wrote:
>> On Tue, 03 Nov 2009 11:15:39 -0800, Girish Venkatasubramanian
>> <girishvs@...> wrote:
>>
>>> Hello,
>>> I have been using freeze.py on 32 bit linux distributions without a
>>> problem. But recently I tried to do the same on RHEL5 x86_64 and ran
>>> into some issues.
>>>
>>> 1) When I ran the script, I got
>>> Error: needed directory /usr/lib/python2.4/config not found
>>>
>>
>> I don't know anything about freeze.py but on 64-bit Red Hat distros
>> (RHEL, Fedora, etc) it should be /usr/lib64/python2.4/config :-)
>
> This sounds a lot like a missing Python devel RPM.
>
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source  (#1, Nov 03 2009)
>>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
> ________________________________________________________________________
>
> ::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
>
>
>   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>           Registered at Amtsgericht Duesseldorf: HRB 46611
>               http://www.egenix.com/company/contact/
>

--
http://mail.python.org/mailman/listinfo/python-list

[M.-A. Lemburg]

Girish Venkatasubramanian wrote:
> Hey Marc-Andre,
> Ummm - I have installed python-devel.x86_64 and checked that the
> /usr/lib64/python2.4/ is populated - anything else I can/shuld do to
> check/ensure the the devel rpm is installed?

If you have the config/ sub-dir in there, things should be
fine.

However, it's possible that you need to tweek the freeze.py
script a little, since RedHat chose to split the Python
installation on x64 in two parts and they may have missed
patching freeze.py as well:

The platform independent parts are in /usr/lib, whereas the
platform dependent parts are in /usr/lib64.

Python normally doesn't support this. It only has a
prefix and an exec_prefix, but those only allow to do
things like prefix=/usr and exec_prefix=/usr64, not
changing the lib/ part in /usr/lib/.

> Thanks.
>
> On Tue, Nov 3, 2009 at 11:32 AM, M.-A. Lemburg <mal@...> wrote:
>> Rami Chowdhury wrote:
>>> On Tue, 03 Nov 2009 11:15:39 -0800, Girish Venkatasubramanian
>>> <girishvs@...> wrote:
>>>
>>>> Hello,
>>>> I have been using freeze.py on 32 bit linux distributions without a
>>>> problem. But recently I tried to do the same on RHEL5 x86_64 and ran
>>>> into some issues.
>>>>
>>>> 1) When I ran the script, I got
>>>> Error: needed directory /usr/lib/python2.4/config not found
>>>>
>>>
>>> I don't know anything about freeze.py but on 64-bit Red Hat distros
>>> (RHEL, Fedora, etc) it should be /usr/lib64/python2.4/config :-)
>>
>> This sounds a lot like a missing Python devel RPM.
>>
>> --
>> Marc-Andre Lemburg
>> eGenix.com
>>
>> Professional Python Services directly from the Source  (#1, Nov 03 2009)
>>>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
>> ________________________________________________________________________
>>
>> ::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
>>
>>
>>   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>>    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>>           Registered at Amtsgericht Duesseldorf: HRB 46611
>>               http://www.egenix.com/company/contact/
>>

--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 03 2009)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
--
http://mail.python.org/mailman/listinfo/python-list

[Girish Venkatasubramanian]

I checked and ls /usr/lib64/python2.4/config/ returns
config.c config.c.in install-sh libpython2.4.a Makefile makesetup
python.o Setup Setup.config Setup.local

so I am guessing the python-devel installation went off OK, from what you say.

I looked at the freeze.py code and I see your point. But for tweaking,
I would need to know what modules should get included from lib64
instead of lib and hardcode them ....
Maybe tinkering around with the makefile which is produced by freeze
is a better way to go?

But since I need to freeze this python code by today, can you suggest
any other tool to do this?

Thanks.

On Tue, Nov 3, 2009 at 11:47 AM, M.-A. Lemburg <mal@...> wrote:

> Girish Venkatasubramanian wrote:
>> Hey Marc-Andre,
>> Ummm - I have installed python-devel.x86_64 and checked that the
>> /usr/lib64/python2.4/ is populated - anything else I can/shuld do to
>> check/ensure the the devel rpm is installed?
>
> If you have the config/ sub-dir in there, things should be
> fine.
>
> However, it's possible that you need to tweek the freeze.py
> script a little, since RedHat chose to split the Python
> installation on x64 in two parts and they may have missed
> patching freeze.py as well:
>
> The platform independent parts are in /usr/lib, whereas the
> platform dependent parts are in /usr/lib64.
>
> Python normally doesn't support this. It only has a
> prefix and an exec_prefix, but those only allow to do
> things like prefix=/usr and exec_prefix=/usr64, not
> changing the lib/ part in /usr/lib/.
>
>> Thanks.
>>
>> On Tue, Nov 3, 2009 at 11:32 AM, M.-A. Lemburg <mal@...> wrote:
>>> Rami Chowdhury wrote:
>>>> On Tue, 03 Nov 2009 11:15:39 -0800, Girish Venkatasubramanian
>>>> <girishvs@...> wrote:
>>>>
>>>>> Hello,
>>>>> I have been using freeze.py on 32 bit linux distributions without a
>>>>> problem. But recently I tried to do the same on RHEL5 x86_64 and ran
>>>>> into some issues.
>>>>>
>>>>> 1) When I ran the script, I got
>>>>> Error: needed directory /usr/lib/python2.4/config not found
>>>>>
>>>>
>>>> I don't know anything about freeze.py but on 64-bit Red Hat distros
>>>> (RHEL, Fedora, etc) it should be /usr/lib64/python2.4/config :-)
>>>
>>> This sounds a lot like a missing Python devel RPM.
>>>
>>> --
>>> Marc-Andre Lemburg
>>> eGenix.com
>>>
>>> Professional Python Services directly from the Source  (#1, Nov 03 2009)
>>>>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>>>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>>>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
>>> ________________________________________________________________________
>>>
>>> ::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
>>>
>>>
>>>   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>>>    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>>>           Registered at Amtsgericht Duesseldorf: HRB 46611
>>>               http://www.egenix.com/company/contact/
>>>
>
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source  (#1, Nov 03 2009)
>>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
> ________________________________________________________________________
>
> ::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
>
>
>   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>           Registered at Amtsgericht Duesseldorf: HRB 46611
>               http://www.egenix.com/company/contact/
>

--
http://mail.python.org/mailman/listinfo/python-list

[Rami Chowdhury]

On Tue, 03 Nov 2009 11:57:17 -0800, Girish Venkatasubramanian  
<girishvs@...> wrote:

> I checked and ls /usr/lib64/python2.4/config/ returns
> config.c config.c.in install-sh libpython2.4.a Makefile makesetup
> python.o Setup Setup.config Setup.local
>
> so I am guessing the python-devel installation went off OK, from what  
> you say.
>
> I looked at the freeze.py code and I see your point. But for tweaking,
> I would need to know what modules should get included from lib64
> instead of lib and hardcode them ....
> Maybe tinkering around with the makefile which is produced by freeze
> is a better way to go?
>
> But since I need to freeze this python code by today, can you suggest
> any other tool to do this?
>

I believe /usr/lib64 on a 64-bit RHEL will contain everything you need ;  
can you try just changing the directory freeze.py looks at to 'lib64', and  
see if the freeze works?

>
> On Tue, Nov 3, 2009 at 11:47 AM, M.-A. Lemburg <mal@...> wrote:
>> Girish Venkatasubramanian wrote:
>>> Hey Marc-Andre,
>>> Ummm - I have installed python-devel.x86_64 and checked that the
>>> /usr/lib64/python2.4/ is populated - anything else I can/shuld do to
>>> check/ensure the the devel rpm is installed?
>>
>> If you have the config/ sub-dir in there, things should be
>> fine.
>>
>> However, it's possible that you need to tweek the freeze.py
>> script a little, since RedHat chose to split the Python
>> installation on x64 in two parts and they may have missed
>> patching freeze.py as well:
>>
>> The platform independent parts are in /usr/lib, whereas the
>> platform dependent parts are in /usr/lib64.
>>
>> Python normally doesn't support this. It only has a
>> prefix and an exec_prefix, but those only allow to do
>> things like prefix=/usr and exec_prefix=/usr64, not
>> changing the lib/ part in /usr/lib/.
>>
>>> Thanks.
>>>
>>> On Tue, Nov 3, 2009 at 11:32 AM, M.-A. Lemburg <mal@...> wrote:
>>>> Rami Chowdhury wrote:
>>>>> On Tue, 03 Nov 2009 11:15:39 -0800, Girish Venkatasubramanian
>>>>> <girishvs@...> wrote:
>>>>>
>>>>>> Hello,
>>>>>> I have been using freeze.py on 32 bit linux distributions without a
>>>>>> problem. But recently I tried to do the same on RHEL5 x86_64 and ran
>>>>>> into some issues.
>>>>>>
>>>>>> 1) When I ran the script, I got
>>>>>> Error: needed directory /usr/lib/python2.4/config not found
>>>>>>
>>>>>
>>>>> I don't know anything about freeze.py but on 64-bit Red Hat distros
>>>>> (RHEL, Fedora, etc) it should be /usr/lib64/python2.4/config :-)
>>>>
>>>> This sounds a lot like a missing Python devel RPM.
>>>>
>>>> --
>>>> Marc-Andre Lemburg
>>>> eGenix.com
>>>>
>>>> Professional Python Services directly from the Source  (#1, Nov 03  
>>>> 2009)
>>>>>>> Python/Zope Consulting and Support ...        
>>>>>>>  http://www.egenix.com/
>>>>>>> mxODBC.Zope.Database.Adapter ...              
>>>>>>> http://zope.egenix.com/
>>>>>>> mxODBC, mxDateTime, mxTextTools ...        
>>>>>>>  http://python.egenix.com/
>>>> ________________________________________________________________________
>>>>
>>>> ::: Try our new mxODBC.Connect Python Database Interface for free !  
>>>> ::::
>>>>
>>>>
>>>>   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>>>>    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>>>>           Registered at Amtsgericht Duesseldorf: HRB 46611
>>>>               http://www.egenix.com/company/contact/
>>>>
>>
>> --
>> Marc-Andre Lemburg
>> eGenix.com
>>
>> Professional Python Services directly from the Source  (#1, Nov 03 2009)
>>>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
>> ________________________________________________________________________
>>
>> ::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
>>
>>
>>   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>>    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>>           Registered at Amtsgericht Duesseldorf: HRB 46611
>>               http://www.egenix.com/company/contact/
>>

--
Rami Chowdhury
"Never attribute to malice that which can be attributed to stupidity" --  
Hanlon's Razor
408-597-7068 (US) / 07875-841-046 (UK) / 0189-245544 (BD)
--
http://mail.python.org/mailman/listinfo/python-list

[Girish Venkatasubramanian]

Will try that.

Meanwhile I went ahead and used cx_freeze and that seems to work OK.

Thanks for your help Rami and Marc-Andre.

On Tue, Nov 3, 2009 at 12:31 PM, Rami Chowdhury
<rami.chowdhury@...> wrote:

> On Tue, 03 Nov 2009 11:57:17 -0800, Girish Venkatasubramanian
> <girishvs@...> wrote:
>
>> I checked and ls /usr/lib64/python2.4/config/ returns
>> config.c config.c.in install-sh libpython2.4.a Makefile makesetup
>> python.o Setup Setup.config Setup.local
>>
>> so I am guessing the python-devel installation went off OK, from what you
>> say.
>>
>> I looked at the freeze.py code and I see your point. But for tweaking,
>> I would need to know what modules should get included from lib64
>> instead of lib and hardcode them ....
>> Maybe tinkering around with the makefile which is produced by freeze
>> is a better way to go?
>>
>> But since I need to freeze this python code by today, can you suggest
>> any other tool to do this?
>>
>
> I believe /usr/lib64 on a 64-bit RHEL will contain everything you need ; can
> you try just changing the directory freeze.py looks at to 'lib64', and see
> if the freeze works?
>
>>
>> On Tue, Nov 3, 2009 at 11:47 AM, M.-A. Lemburg <mal@...> wrote:
>>>
>>> Girish Venkatasubramanian wrote:
>>>>
>>>> Hey Marc-Andre,
>>>> Ummm - I have installed python-devel.x86_64 and checked that the
>>>> /usr/lib64/python2.4/ is populated - anything else I can/shuld do to
>>>> check/ensure the the devel rpm is installed?
>>>
>>> If you have the config/ sub-dir in there, things should be
>>> fine.
>>>
>>> However, it's possible that you need to tweek the freeze.py
>>> script a little, since RedHat chose to split the Python
>>> installation on x64 in two parts and they may have missed
>>> patching freeze.py as well:
>>>
>>> The platform independent parts are in /usr/lib, whereas the
>>> platform dependent parts are in /usr/lib64.
>>>
>>> Python normally doesn't support this. It only has a
>>> prefix and an exec_prefix, but those only allow to do
>>> things like prefix=/usr and exec_prefix=/usr64, not
>>> changing the lib/ part in /usr/lib/.
>>>
>>>> Thanks.
>>>>
>>>> On Tue, Nov 3, 2009 at 11:32 AM, M.-A. Lemburg <mal@...> wrote:
>>>>>
>>>>> Rami Chowdhury wrote:
>>>>>>
>>>>>> On Tue, 03 Nov 2009 11:15:39 -0800, Girish Venkatasubramanian
>>>>>> <girishvs@...> wrote:
>>>>>>
>>>>>>> Hello,
>>>>>>> I have been using freeze.py on 32 bit linux distributions without a
>>>>>>> problem. But recently I tried to do the same on RHEL5 x86_64 and ran
>>>>>>> into some issues.
>>>>>>>
>>>>>>> 1) When I ran the script, I got
>>>>>>> Error: needed directory /usr/lib/python2.4/config not found
>>>>>>>
>>>>>>
>>>>>> I don't know anything about freeze.py but on 64-bit Red Hat distros
>>>>>> (RHEL, Fedora, etc) it should be /usr/lib64/python2.4/config :-)
>>>>>
>>>>> This sounds a lot like a missing Python devel RPM.
>>>>>
>>>>> --
>>>>> Marc-Andre Lemburg
>>>>> eGenix.com
>>>>>
>>>>> Professional Python Services directly from the Source  (#1, Nov 03
>>>>> 2009)
>>>>>>>>
>>>>>>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>>>>>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>>>>>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
>>>>>
>>>>>
>>>>> ________________________________________________________________________
>>>>>
>>>>> ::: Try our new mxODBC.Connect Python Database Interface for free !
>>>>> ::::
>>>>>
>>>>>
>>>>>   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>>>>>    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>>>>>           Registered at Amtsgericht Duesseldorf: HRB 46611
>>>>>               http://www.egenix.com/company/contact/
>>>>>
>>>
>>> --
>>> Marc-Andre Lemburg
>>> eGenix.com
>>>
>>> Professional Python Services directly from the Source  (#1, Nov 03 2009)
>>>>>>
>>>>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>>>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>>>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
>>>
>>> ________________________________________________________________________
>>>
>>> ::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
>>>
>>>
>>>   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>>>    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>>>           Registered at Amtsgericht Duesseldorf: HRB 46611
>>>               http://www.egenix.com/company/contact/
>>>
>
>
>
> --
> Rami Chowdhury
> "Never attribute to malice that which can be attributed to stupidity" --
> Hanlon's Razor
> 408-597-7068 (US) / 07875-841-046 (UK) / 0189-245544 (BD)
>

--
http://mail.python.org/mailman/listinfo/python-list

[Mike Driscoll]

On Nov 3, 3:23 pm, Girish Venkatasubramanian <giris...@...>
wrote:
> Will try that.
>
> Meanwhile I went ahead and used cx_freeze and that seems to work OK.
>
> Thanks for your help Rami and Marc-Andre.


Something that you might want to try in the future is GUI2Exe, which
allows you to play with a whole slew of freezing modules:

http://code.google.com/p/gui2exe/

I've been using it to make executables on Windows through it's py2exe
implementation.

-------------------
Mike Driscoll

Blog:   http://blog.pythonlibrary.org
--
http://mail.python.org/mailman/listinfo/python-list

[kee nethery]

Having an issue with elementtree XML() in python 2.6.4.

This code works fine:

      from xml.etree import ElementTree as et
      getResponse = u'''<?xml version="1.0" encoding="UTF-8"?>  
<customer><shipping><state>bobble</state><city>head</
city><street>city</street></shipping></customer>'''
      theResponseXml = et.XML(getResponse)

This code errors out when it tries to do the et.XML()

      from xml.etree import ElementTree as et
      getResponse = u'''<?xml version="1.0" encoding="UTF-8"?>  
<customer><shipping><state>\ue58d83\ue89189\ue79c8C</state><city>
\ue69f8f\ue5b882</city><street>\ue9ab98\ue58d97\ue58fb03</street></
shipping></customer>'''
      theResponseXml = et.XML(getResponse)

In my real code, I'm pulling the getResponse data from a web page that  
returns as XML and when I display it in the browser you can see the  
Japanese characters in the data. I've removed all the stuff in my code  
and tried to distill it down to just what is failing. Hopefully I have  
not removed something essential.

Why is this not working and what do I need to do to use Elementtree  
with unicode?

Thanks, Kee Nethery
--
http://mail.python.org/mailman/listinfo/python-list

[Gabriel Genellina-7]

En Tue, 03 Nov 2009 21:01:46 -0300, Kee Nethery <kee@...> escribió:

> Having an issue with elementtree XML() in python 2.6.4.
>
> This code works fine:
>
>       from xml.etree import ElementTree as et
>       getResponse = u'''<?xml version="1.0" encoding="UTF-8"?>  
> <customer><shipping><state>bobble</state><city>head</
> city><street>city</street></shipping></customer>'''
>       theResponseXml = et.XML(getResponse)
>
> This code errors out when it tries to do the et.XML()
>
>       from xml.etree import ElementTree as et
>       getResponse = u'''<?xml version="1.0" encoding="UTF-8"?>  
> <customer><shipping><state>\ue58d83\ue89189\ue79c8C</state><city>
> \ue69f8f\ue5b882</city><street>\ue9ab98\ue58d97\ue58fb03</street></
> shipping></customer>'''
>       theResponseXml = et.XML(getResponse)
>
> In my real code, I'm pulling the getResponse data from a web page that  
> returns as XML and when I display it in the browser you can see the  
> Japanese characters in the data. I've removed all the stuff in my code  
> and tried to distill it down to just what is failing. Hopefully I have  
> not removed something essential.
>
> Why is this not working and what do I need to do to use Elementtree with  
> unicode?

et expects bytes as input, not unicode. You're decoding too early  
(decoding early is good, but not in this case, because et does the work  
for you). Either feed et.XML with the bytes before decoding, or reencode  
the received xml text in UTF-8 (since this is the declared encoding).

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

[kee nethery]

On Nov 3, 2009, at 4:44 PM, Gabriel Genellina wrote:

> En Tue, 03 Nov 2009 21:01:46 -0300, Kee Nethery <kee@...>  
> escribió:
>
>> I've removed all the stuff in my code and tried to distill it down  
>> to just what is failing. Hopefully I have not removed something  
>> essential.

Sounds like I did remove something essential.

>
> et expects bytes as input, not unicode. You're decoding too early  
> (decoding early is good, but not in this case, because et does the  
> work for you). Either feed et.XML with the bytes before decoding, or  
> reencode the received xml text in UTF-8 (since this is the declared  
> encoding).

Here is the code that hits the URL:
         getResponse1 = urllib2.urlopen(theUrl)
         getResponse2 = getResponse1.read()
         getResponse3 = unicode(getResponse2,'UTF-8')
        theResponseXml = et.XML(getResponse3)

So are you saying I want to do:
         getResponse1 = urllib2.urlopen(theUrl)
         getResponse4 = getResponse1.read()
        theResponseXml = et.XML(getResponse4)

The reason I am confused is that getResponse2 is classified as an  
"str" in the Komodo IDE. I want to make sure I don't lose the non-
ASCII characters coming from the URL. If I do the second set of code,  
does elementtree auto convert the str into unicode? How do I deal with  
the XML as unicode when I put it into elementtree as a string?

Very confusing. Thanks for the help.

Kee
--
http://mail.python.org/mailman/listinfo/python-list

[John Machin]

On Nov 4, 11:01 am, Kee Nethery <k...@...> wrote:

> Having an issue with elementtree XML() in python 2.6.4.
>
> This code works fine:
>
>       from xml.etree import ElementTree as et
>       getResponse = u'''<?xml version="1.0" encoding="UTF-8"?>  
> <customer><shipping><state>bobble</state><city>head</
> city><street>city</street></shipping></customer>'''
>       theResponseXml = et.XML(getResponse)
>
> This code errors out when it tries to do the et.XML()
>
>       from xml.etree import ElementTree as et
>       getResponse = u'''<?xml version="1.0" encoding="UTF-8"?>  
> <customer><shipping><state>\ue58d83\ue89189\ue79c8C</state><city>
> \ue69f8f\ue5b882</city><street>\ue9ab98\ue58d97\ue58fb03</street></
> shipping></customer>'''
>       theResponseXml = et.XML(getResponse)
>
> In my real code, I'm pulling the getResponse data from a web page that  
> returns as XML and when I display it in the browser you can see the  
> Japanese characters in the data. I've removed all the stuff in my code  
> and tried to distill it down to just what is failing. Hopefully I have  
> not removed something essential.
>
> Why is this not working and what do I need to do to use Elementtree  
> with unicode?

On Nov 4, 11:01 am, Kee Nethery <k...@...> wrote:

> Having an issue with elementtree XML() in python 2.6.4.
>
> This code works fine:
>
>       from xml.etree import ElementTree as et
>       getResponse = u'''<?xml version="1.0" encoding="UTF-8"?>
> <customer><shipping><state>bobble</state><city>head</
> city><street>city</street></shipping></customer>'''
>       theResponseXml = et.XML(getResponse)
>
> This code errors out when it tries to do the et.XML()
>
>       from xml.etree import ElementTree as et
>       getResponse = u'''<?xml version="1.0" encoding="UTF-8"?>
> <customer><shipping><state>\ue58d83\ue89189\ue79c8C</state><city>
> \ue69f8f\ue5b882</city><street>\ue9ab98\ue58d97\ue58fb03</street></
> shipping></customer>'''
>       theResponseXml = et.XML(getResponse)
>
> In my real code, I'm pulling the getResponse data from a web page that
> returns as XML and when I display it in the browser you can see the
> Japanese characters in the data. I've removed all the stuff in my code
> and tried to distill it down to just what is failing. Hopefully I have
> not removed something essential.
>
> Why is this not working and what do I need to do to use Elementtree
> with unicode?

What you need to do is NOT feed it unicode. You feed it a str object
and it gets decoded according to the encoding declaration found in the
first line. So take the str object that you get from the web (should
be UTF8-encoded already unless the header is lying), and throw that at
ET ... like this:

| Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit
(Intel)] on win32
| Type "help", "copyright", "credits" or "license" for more
information.
| >>> from xml.etree import ElementTree as et
| >>> ucode = u'''<?xml version="1.0" encoding="UTF-8"?>
| ... <customer><shipping>
| ... <state>\ue58d83\ue89189\ue79c8C</state>
| ... <city>\ue69f8f\ue5b882</city>
| ... <street>\ue9ab98\ue58d97\ue58fb03</street>
| ... </shipping></customer>'''
| >>> xml= et.XML(ucode)
| Traceback (most recent call last):
|   File "<stdin>", line 1, in <module>
|   File "C:\python26\lib\xml\etree\ElementTree.py", line 963, in XML
|     parser.feed(text)
|   File "C:\python26\lib\xml\etree\ElementTree.py", line 1245, in
feed
|     self._parser.Parse(data, 0)
| UnicodeEncodeError: 'ascii' codec can't encode character u'\ue58d'
in position 69: ordinal not in range(128)
| # as expected
| >>> strg = ucode.encode('utf8')
| # encoding as utf8 is for DEMO purposes.
| # i.e. use the original web str object, don't convert it to unicode
| # and back to utf8.
| >>> xml2 = et.XML(strg)
| >>> xml2.tag
| 'customer'
| >>> for c in xml2.getchildren():
| ...    print c.tag, repr(c.text)
| ...
| shipping '\n'
| >>> for c in xml2[0].getchildren():
| ...    print c.tag, repr(c.text)
| ...
| state u'\ue58d83\ue89189\ue79c8C'
| city u'\ue69f8f\ue5b882'
| street u'\ue9ab98\ue58d97\ue58fb03'
| >>>

By the way: (1) it usually helps to be more explicit than "errors
out", preferably the exact copied/pasted output as shown above; this
is one of the rare cases where the error message is predictable (2)
PLEASE don't start a new topic in a reply in somebody else's thread.

--
http://mail.python.org/mailman/listinfo/python-list

[John Machin]

On Nov 4, 12:14 pm, Kee Nethery <k...@...> wrote:

> On Nov 3, 2009, at 4:44 PM, Gabriel Genellina wrote:
>
> > En Tue, 03 Nov 2009 21:01:46 -0300, Kee Nethery <k...@...>  
> > escribió:
>
> >> I've removed all the stuff in my code and tried to distill it down  
> >> to just what is failing. Hopefully I have not removed something  
> >> essential.
>
> Sounds like I did remove something essential.

No, you added something that was not only inessential but caused
trouble.

> > et expects bytes as input, not unicode. You're decoding too early  
> > (decoding early is good, but not in this case, because et does the  
> > work for you). Either feed et.XML with the bytes before decoding, or  
> > reencode the received xml text in UTF-8 (since this is the declared  
> > encoding).
>
> Here is the code that hits the URL:
>          getResponse1 = urllib2.urlopen(theUrl)
>          getResponse2 = getResponse1.read()
>          getResponse3 = unicode(getResponse2,'UTF-8')
>         theResponseXml = et.XML(getResponse3)
>
> So are you saying I want to do:
>          getResponse1 = urllib2.urlopen(theUrl)
>          getResponse4 = getResponse1.read()
>         theResponseXml = et.XML(getResponse4)

You got the essence. Note: that in no way implies any approval of your
naming convention :-)

> The reason I am confused is that getResponse2 is classified as an  
> "str" in the Komodo IDE. I want to make sure I don't lose the non-
> ASCII characters coming from the URL.

str is all about 8-bit bytes. Your data comes from the web in 8-bit
bytes. No problem. Just don't palpate it unnecessarily.

> If I do the second set of code,  
> does elementtree auto convert the str into unicode?

Yes. See the example I gave in my earlier posting:

| ...    print c.tag, repr(c.text)
| state u'\ue58d83\ue89189\ue79c8C'

That first u means the type is unicode.

> How do I deal with  
> the XML as unicode when I put it into elementtree as a string?

That's unfortunately rather ambiguous: (1) put past/present? (2)
string unicode/str? (3) what is referent of "it"?

All text in what et returns is unicode [*] so you read it out as
unicode (see above example) or written as unicode if you want to
change it:

    your_element.text = u'a unicode object'

[*] As an "optimisation", et stores strings as str objects if they
contain only ASCII bytes (and are thus losslessly convertible to
unicode). In preparation for running your code under Python 3.X, it's
best to ignore this and use unicode constants u'foo' (if you need text
constants at all) even if et would let you get away with 'foo'.

HTH,
John
--
http://mail.python.org/mailman/listinfo/python-list

[kee nethery]

On Nov 3, 2009, at 5:27 PM, John Machin wrote:

> On Nov 4, 11:01 am, Kee Nethery <k...@...> wrote:
>> Having an issue with elementtree XML() in python 2.6.4.
>>
>> This code works fine:
>>
>>       from xml.etree import ElementTree as et
>>       getResponse = u'''<?xml version="1.0" encoding="UTF-8"?>
>> <customer><shipping><state>bobble</state><city>head</
>> city><street>city</street></shipping></customer>'''
>>       theResponseXml = et.XML(getResponse)
>>
>> This code errors out when it tries to do the et.XML()
>>
>>       from xml.etree import ElementTree as et
>>       getResponse = u'''<?xml version="1.0" encoding="UTF-8"?>
>> <customer><shipping><state>\ue58d83\ue89189\ue79c8C</state><city>
>> \ue69f8f\ue5b882</city><street>\ue9ab98\ue58d97\ue58fb03</street></
>> shipping></customer>'''
>>       theResponseXml = et.XML(getResponse)
>>
>> In my real code, I'm pulling the getResponse data from a web page  
>> that
>> returns as XML and when I display it in the browser you can see the
>> Japanese characters in the data. I've removed all the stuff in my  
>> code
>> and tried to distill it down to just what is failing. Hopefully I  
>> have
>> not removed something essential.
>>
>> Why is this not working and what do I need to do to use Elementtree
>> with unicode?
>
> On Nov 4, 11:01 am, Kee Nethery <k...@...> wrote:
>> Having an issue with elementtree XML() in python 2.6.4.
>>
>> This code works fine:
>>
>>      from xml.etree import ElementTree as et
>>      getResponse = u'''<?xml version="1.0" encoding="UTF-8"?>
>> <customer><shipping><state>bobble</state><city>head</
>> city><street>city</street></shipping></customer>'''
>>      theResponseXml = et.XML(getResponse)
>>
>> This code errors out when it tries to do the et.XML()
>>
>>      from xml.etree import ElementTree as et
>>      getResponse = u'''<?xml version="1.0" encoding="UTF-8"?>
>> <customer><shipping><state>\ue58d83\ue89189\ue79c8C</state><city>
>> \ue69f8f\ue5b882</city><street>\ue9ab98\ue58d97\ue58fb03</street></
>> shipping></customer>'''
>>      theResponseXml = et.XML(getResponse)
>>
>> In my real code, I'm pulling the getResponse data from a web page  
>> that
>> returns as XML and when I display it in the browser you can see the
>> Japanese characters in the data. I've removed all the stuff in my  
>> code
>> and tried to distill it down to just what is failing. Hopefully I  
>> have
>> not removed something essential.
>>
>> Why is this not working and what do I need to do to use Elementtree
>> with unicode?
>
> What you need to do is NOT feed it unicode. You feed it a str object
> and it gets decoded according to the encoding declaration found in the
> first line.

That it uses "the encoding declaration found in the first line" is the  
nugget of data that is not in the documentation that has stymied me  
for days. Thank you!

The other thing that has been confusing is that I've been using "dump"  
to view what is in the elementtree instance and the non-ASCII  
characters have been displayed as "numbered  
entities" (<city>柏市</city>) and I know that is not the  
representation I want the data to be in. A co-worker suggested that  
instead of "dump" that I use "et.tostring(theResponseXml,  
encoding='utf-8')" and then print that to see the characters. That  
process causes the non-ASCII characters to display as the glyphs I  
know them to be.

If there was a place in the official docs for me to append these  
nuggets of information to the sections for  
"xml.etree.ElementTree.XML(text)" and  
"xml.etree.ElementTree.dump(elem)" I would absolutely do so.

Thank you!
Kee Nethery

> So take the str object that you get from the web (should
> be UTF8-encoded already unless the header is lying), and throw that at
> ET ... like this:
>
> | Python 2.6.4 (r264:75708, Oct 26 2009, 08:23:19) [MSC v.1500 32 bit
> (Intel)] on win32
> | Type "help", "copyright", "credits" or "license" for more
> information.
> | >>> from xml.etree import ElementTree as et
> | >>> ucode = u'''<?xml version="1.0" encoding="UTF-8"?>
> | ... <customer><shipping>
> | ... <state>\ue58d83\ue89189\ue79c8C</state>
> | ... <city>\ue69f8f\ue5b882</city>
> | ... <street>\ue9ab98\ue58d97\ue58fb03</street>
> | ... </shipping></customer>'''
> | >>> xml= et.XML(ucode)
> | Traceback (most recent call last):
> |   File "<stdin>", line 1, in <module>
> |   File "C:\python26\lib\xml\etree\ElementTree.py", line 963, in XML
> |     parser.feed(text)
> |   File "C:\python26\lib\xml\etree\ElementTree.py", line 1245, in
> feed
> |     self._parser.Parse(data, 0)
> | UnicodeEncodeError: 'ascii' codec can't encode character u'\ue58d'
> in position 69: ordinal not in range(128)
> | # as expected
> | >>> strg = ucode.encode('utf8')
> | # encoding as utf8 is for DEMO purposes.
> | # i.e. use the original web str object, don't convert it to unicode
> | # and back to utf8.
> | >>> xml2 = et.XML(strg)
> | >>> xml2.tag
> | 'customer'
> | >>> for c in xml2.getchildren():
> | ...    print c.tag, repr(c.text)
> | ...
> | shipping '\n'
> | >>> for c in xml2[0].getchildren():
> | ...    print c.tag, repr(c.text)
> | ...
> | state u'\ue58d83\ue89189\ue79c8C'
> | city u'\ue69f8f\ue5b882'
> | street u'\ue9ab98\ue58d97\ue58fb03'
> | >>>
>
> By the way: (1) it usually helps to be more explicit than "errors
> out", preferably the exact copied/pasted output as shown above; this
> is one of the rare cases where the error message is predictable (2)
> PLEASE don't start a new topic in a reply in somebody else's thread.
>
> --
> http://mail.python.org/mailman/listinfo/python-list

-------------------------------------------------
I check email roughly 2 to 3 times per business day.
Kagi main office: [phone]


--
http://mail.python.org/mailman/listinfo/python-list

[John Machin]

On Nov 4, 1:06 pm, Kee Nethery <k...@...> wrote:
> On Nov 3, 2009, at 5:27 PM, John Machin wrote:
>
>
>
> > On Nov 4, 11:01 am, Kee Nethery <k...@...> wrote:

> >> Why is this not working and what do I need to do to use Elementtree
> >> with unicode?
>
> > What you need to do is NOT feed it unicode. You feed it a str object
> > and it gets decoded according to the encoding declaration found in the
> > first line.
>
> That it uses "the encoding declaration found in the first line" is the  
> nugget of data that is not in the documentation that has stymied me  
> for days. Thank you!

And under the "don't repeat" principle, it shouldn't be in the
Elementtree docs; it's nothing special about ET -- it's part of the
definition of an XML document (which for universal loss-free
transportability naturally must be encoded somehow, and the document
must state what its own encoding is (if it's not the default
(UTF-8))).

> The other thing that has been confusing is that I've been using "dump"  
> to view what is in the elementtree instance and the non-ASCII  
> characters have been displayed as "numbered  
> entities" (<city>柏市</city>) and I know that is not the  
> representation I want the data to be in. A co-worker suggested that  
> instead of "dump" that I use "et.tostring(theResponseXml,  
> encoding='utf-8')" and then print that to see the characters. That  
> process causes the non-ASCII characters to display as the glyphs I  
> know them to be.
>
> If there was a place in the official docs for me to append these  
> nuggets of information to the sections for  
> "xml.etree.ElementTree.XML(text)" and  
> "xml.etree.ElementTree.dump(elem)" I would absolutely do so.

I don't understand ... tostring() is in the same section as dump(),
about two screen-heights away. You want to include the tostring() docs
in the dump() docs? The usual idea is not to get bogged down in the
first function that looks at first glance like it might do what you
want ("look at the glyphs") but doesn't (it writes a (transportable)
XML stream) but press on to the next plausible candidate.
--
http://mail.python.org/mailman/listinfo/python-list

[Gabriel Genellina-7]

En Tue, 03 Nov 2009 23:06:58 -0300, Kee Nethery <kee@...> escribió:

> If there was a place in the official docs for me to append these nuggets  
> of information to the sections for "xml.etree.ElementTree.XML(text)" and  
> "xml.etree.ElementTree.dump(elem)" I would absolutely do so.

http://bugs.python.org/ applies to documentation too.

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

[kee nethery]

On Nov 3, 2009, at 7:06 PM, Gabriel Genellina wrote:

> En Tue, 03 Nov 2009 23:06:58 -0300, Kee Nethery <kee@...>  
> escribió:
>
>> If there was a place in the official docs for me to append these  
>> nuggets of information to the sections for  
>> "xml.etree.ElementTree.XML(text)" and  
>> "xml.etree.ElementTree.dump(elem)" I would absolutely do so.
>
> http://bugs.python.org/ applies to documentation too.

I've submitted documentation bugs in the past and no action was taken  
on them, the bugs were closed. I'm guessing that information "that  
everyone knows" not being in the documentation is not a bug. It's my  
fault I'm a newbie and I accept that. Thanks to you two for helping me  
get past this block.

Kee
--
http://mail.python.org/mailman/listinfo/python-list

[Stefan Behnel-3]

John Machin, 04.11.2009 02:56:
> On Nov 4, 12:14 pm, Kee Nethery wrote:
>> The reason I am confused is that getResponse2 is classified as an  
>> "str" in the Komodo IDE. I want to make sure I don't lose the non-
>> ASCII characters coming from the URL.
>
> str is all about 8-bit bytes.

True in Py2.x, false in Py3.

What you mean is the "bytes" type, which, sadly, was named "str" in Python 2.x.

The problem the OP ran into was due to the fact that Python 2.x handled
"ASCII characters in a unicode string" <-> "ASCII encoded byte string"
conversion behind the scenes, which lead to all sorts of trouble to loads
of people, and was finally discarded in Python 3.0.

Stefan
--
http://mail.python.org/mailman/listinfo/python-list