imputil.py, is this a bug ?

[stef mientki-4]

hello,

I get an error compiling with pyjamas, in the standard module imputil,
_import_top_module

Traceback (most recent call last):
  File
"D:\Data_Python_25\PyLab_Works\pylab_works_programs\PyJamas_Test\Vraag_Test1.py",
line 22, in <module>
    from Vragenlijsten import Get_Vragenlijst
  File "P:\Python\Lib\site-packages\pyjd\imputil.py", line 109, in
_import_hook
    top_module = self._import_top_module(parts[0])
  File "P:\Python\Lib\site-packages\pyjd\imputil.py", line 217, in
_import_top_module
    module = item.import_top(name)
AttributeError: 'unicode' object has no attribute 'import_top'

It seems that elements of sys.path can be of the type unicode

    def _import_top_module(self, name):
        # scan sys.path looking for a location in the filesystem that
contains
        # the module, or an Importer object that can import the module.
        for item in sys.path:
            if isinstance(item, _StringType):
                module = self.fs_imp.import_from_dir(item, name)
            else:
                module = item.import_top(name)
            if module:
                return module
        return None

so by adding the next 2 lines, everything works ok.
            elif isinstance ( item, basestring ) :
                module = self.fs_imp.import_from_dir ( str(item), name)

is this a bug ?
(I'm using Python 2.5.2 on Windows )

thanks,
Stef Mientki
--
http://mail.python.org/mailman/listinfo/python-list

[Gabriel Genellina-7]

En Fri, 06 Nov 2009 18:33:37 -0300, Stef Mientki <stef.mientki@...>  
escribió:

> I get an error compiling with pyjamas, in the standard module imputil,  
> _import_top_module

Note that imputil is undocumented in 2.5, deprecated in 2.6 and  
definitively gone in 3.0

> AttributeError: 'unicode' object has no attribute 'import_top'
>
>     def _import_top_module(self, name):
>         # scan sys.path looking for a location in the filesystem that  
> contains
>         # the module, or an Importer object that can import the module.
>         for item in sys.path:
>             if isinstance(item, _StringType):
>                 module = self.fs_imp.import_from_dir(item, name)
>             else:
>                 module = item.import_top(name)
>             if module:
>                 return module
>         return None
>
> It seems that elements of sys.path can be of the type unicode
> so by adding the next 2 lines, everything works ok.
>             elif isinstance ( item, basestring ) :
>                 module = self.fs_imp.import_from_dir ( str(item), name)
>
> is this a bug ?
> (I'm using Python 2.5.2 on Windows )

Yes, seems to be a bug. But given the current status of imputil, it's not  
likely to be fixed; certainly not in 2.5 which only gets security fixes  
now.

I cannot test it at this moment, but I'd use the unicode item directly  
(that is, self.fs_imp.import_from_dir(item, name)). Or perhaps  
item.encode(sys.getdefaultfilesystemencoding()). str(item) definitively  
won't work with directory names containing non-ascii characters.

Why are you using imputil in the first place?

--
Gabriel Genellina

--
http://mail.python.org/mailman/listinfo/python-list

[stef mientki-4]

Gabriel Genellina wrote:

> En Fri, 06 Nov 2009 18:33:37 -0300, Stef Mientki
> <stef.mientki@...> escribió:
>
>> I get an error compiling with pyjamas, in the standard module
>> imputil, _import_top_module
>
> Note that imputil is undocumented in 2.5, deprecated in 2.6 and
> definitively gone in 3.0
>
>> AttributeError: 'unicode' object has no attribute 'import_top'
>>
>>     def _import_top_module(self, name):
>>         # scan sys.path looking for a location in the filesystem that
>> contains
>>         # the module, or an Importer object that can import the module.
>>         for item in sys.path:
>>             if isinstance(item, _StringType):
>>                 module = self.fs_imp.import_from_dir(item, name)
>>             else:
>>                 module = item.import_top(name)
>>             if module:
>>                 return module
>>         return None
>>
>> It seems that elements of sys.path can be of the type unicode
>> so by adding the next 2 lines, everything works ok.
>>             elif isinstance ( item, basestring ) :
>>                 module = self.fs_imp.import_from_dir ( str(item), name)
>>
>> is this a bug ?
>> (I'm using Python 2.5.2 on Windows )
>
> Yes, seems to be a bug. But given the current status of imputil, it's
> not likely to be fixed; certainly not in 2.5 which only gets security
> fixes now.
>
> I cannot test it at this moment, but I'd use the unicode item directly
> (that is, self.fs_imp.import_from_dir(item, name)). Or perhaps
> item.encode(sys.getdefaultfilesystemencoding()). str(item)
> definitively won't work with directory names containing non-ascii
> characters.
>
> Why are you using imputil in the first place?
>

thanks Gabriel,
well PyJamas is using (a copy) of it and I bumped into problems using
PyJamas.
I'll send this message to the PyJamas developers,
because this stuff is far beyond my knowledge.

cheers,
Stef
--
http://mail.python.org/mailman/listinfo/python-list

[lkcl-2]

On Nov 7, 2:20 am, "Gabriel Genellina" <gagsl-...@...> wrote:
> Yes, seems to be a bug. But given the current status of imputil, it's not
> likely to be fixed; certainly not in 2.5 which only gets security fixes
> now.

 well, that bug's not the only one.  the other one that i found, which
i have been specifically ordered not to report (that or _any_ python
bugs, of which there have been several discovered in the past eight
months), will have to wait until the python developers rescind that
order.

if the python community is lucky, by the time that decision is made, i
will not have forgotten what those bugs are.


> (that is, self.fs_imp.import_from_dir(item, name)). Or perhaps
> item.encode(sys.getdefaultfilesystemencoding()). str(item) definitively
> won't work with directory names containing non-ascii characters.
>
> Why are you using imputil in the first place?

 it's an absolutely necessary and integral part of pyjamas-desktop
"platform overrides".

 it's absolutely essential to track, in exactly the same manner in
which python "normally" performs importing, and to give the platform-
specific "overrides" a chance to get in there, first.

 so, it is absolutely essential to have a correct working version of
imputil.py - and due to the bugs present, and the unwillingness of the
python team to fix those bugs, pyjamas-desktop is forced to maintain a
copy of imputil.py

 the "platform" is set to e.g. hulahop, pywebkitgtk or mshtml,
depending on the decision made by the user or the developer to use a
particular browser engine.  the platform name is stored in
pyjd.platform in exactly the same way that the system name is stored
in sys.platform.

 the way that the platform-specific overrides works is to perform AST
translation of the module, and then to look for the exact same module
but in platform/{modulename}{platformname}.py and perform AST
translation of _that_ module as well.

 then, at the top level, any global functions in the platform-specific
AST tree *replace* those in the "main" AST.  likewise, a node-walk
along all methods in all classes of the platform-specific AST tree.

 in this way, god-awful messy code like this:

 Widget.py
 class Widget:
     def do_something(self):
        if platform == 'mshtml':
            # do something terrible and ugly
        elif platform == 'pywebkitgtk':
            # do something only marginally better
        else:
            # do the default W3C standards-compliant thing

     def a_standard_function(self):
        # do something normal that all the browser engines get right

 can be replaced by three files, each of which encodes *only* the
logic associated with the god-awful ugliness of each browser:

 Widget.py
 class Widget:
     def do_something(self):
        # do the default W3C standards-compliant thing

     def a_standard_function(self):
        # do something normal that all the browser engines get right

 platform/Widgetpywebkitgtk.py
 class Widget:
     def do_something(self):
        # do something only marginally better

 platform/Widgetmshtml.py
 class Widget:
     def do_something(self):
       # do something terrible and ugly


 a similar trick could in fact be deployed to drastically simplify the
layout of e.g. distutils, _espeeecially_ the compiler and linker
modules, by using sys.platform as the "override", or, given that
that's not entirely the whole decision-making process, as i noted when
doing the mingw32 port of python, it would be better to set something
like distutils.platform and to use that.

 however, although the technique could be used in the distutils/
ccompiler.py case, the level of complexity of the code (the size of
each "override"), and the small number of actual modules, means that
there isn't actually that much benefit in deploying this AST-
overriding technique.

 in the pyjamas API, however, with over 75 user-interface modules, and
over 1200 functions, any one of which could be over-ridden by _eight_
separate platforms, each with their own quirks that can usually be
handled with one or two lines of code, the AST-merging idea that james
tauber came up with is an absolute god-send.

l.
--
http://mail.python.org/mailman/listinfo/python-list

[Terry Reedy]

Stef Mientki wrote:
> Gabriel Genellina wrote:
>> En Fri, 06 Nov 2009 18:33:37 -0300, Stef Mientki
>> <stef.mientki@...> escribió:
>>
>>> I get an error compiling with pyjamas, in the standard module
>>> imputil, _import_top_module
>>
>> Note that imputil is undocumented in 2.5, deprecated in 2.6 and
>> definitively gone in 3.0

It was deprecated because it is buggy and unmaintainable.

In 3.1 it has been replaced with the new written-in-Python importlib.
That has also been added to the upcoming 2.7.

Anyone using imputil and having problems with it should try the 2.7
version of importlib and see if it runs well-enough on earlier versions.

Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list

[Terry Reedy]

lkcl wrote:
> On Nov 7, 2:20 am, "Gabriel Genellina" <gagsl-...@...> wrote:
>> Yes, seems to be a bug. But given the current status of imputil, it's not
>> likely to be fixed; certainly not in 2.5 which only gets security fixes
>> now.
>
>  well, that bug's not the only one.  the other one that i found, which
> i have been specifically ordered not to report

Imputil has been deprecated and has in 3.1 and will-be in 2.7 replaced
by importlib, written in Python. So reporting more bugs in imputil is
rather useless noise. So either use your patched version or try shifting
to the 2.7 importlib and see if it runs well enough on earlier versions.

 >(that or _any_ python bugs, of which there have been several
discovered in the past eight
> months), will have to wait until the python developers rescind that
> order.

Without seeing documentary proof, I am dubious that you have been
ordered to not properly report bugs in supported, not deprecated,
modules. If really so, post reports here and interested parties can try
to verify and possibly report them themselves. I, for instance, would be
interested in 3.1 bugs that do not require outside resources to verify.

Terry Jan Reedy

--
http://mail.python.org/mailman/listinfo/python-list