Strange lookup error...

[B. B.]

Hello,

I have just subscribed this list, to describe a strange error that occured for me. In the bugtrack people seem to be very precise in describing where the error is, I have no idea about where the error is, so I will not polute the bugtrack :-)
I am sorry if writing to this list is not the right thing to do, feel free to guide me then :-).


Background:
-----------------------
After update of
 - an application, with some modules being renamed
 - python from 2.5 to 2.6
 - zodb
an updatescript is called to refresh the broken objects.
The objects are stored in dict of type <class 'zope.container.contained.ContainedProxy'> with a simple unicode key to the object mapping.
The refresh of a broken object precedure is :
- identify broken object
- identify correct type
- create an instance of that type
- transfer state from the broken object to the new instance
- reassign the new object in the dict. ( myDict[ id ] = newInstance )
-----------------------


Observation:
-----------------------
Within the same transaction as the broken objects are refreshed, I can verify that I can lookup all objects..
After a commit, and close / open of the connection, I can not lookup some objects.
But if I make a list of all items like :

allItems = list( myDict.items() )

all ids and objects are represented in the allItems, and NO objects are broken, they look all fine.
But for some ids amongst the allItems, "myDict[ id ]" throws a KeyError ( and "id in myDict" returns False )...

It is about 25 objects or so out of a few tusind that fails.... I can not see anything special with those objects
The FileStorage is last updated with a zodb3 version 3.8.0
-----------------------


The precedure is used to work fine, I have tried with different version of zodb3 from pypi,

#bad zodb3 = 3.9.2
#bad zodb3 = 3.9.1
#bad zodb3 = 3.9.0
#bad zodb3 = 3.9.0c2
#ok zodb3 = 3.9.0c1
#ok zodb3 = 3.9.0b5
#ok zodb3 = 3.9.0a12

and it says, until version 3.9.0c1, it worked, but with 3.9.0c2 it started to fail.


System  Linux :
 - Python 2.6.2
 - uname -iorm -> 2.6.27-gentoo-r8 x86_64 AuthenticAMD GNU/Linux


Am I violating some rules of migration, or is it a bug introduced by version 3.9.0c2?? 

If it is an interesting bug to catch ( for me it is very interesting :-) ), I can test versions or patches.



Best Regards

Brian

_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@...
https://mail.zope.org/mailman/listinfo/zodb-dev

[Jim Fulton]

On Sat, Oct 17, 2009 at 12:49 PM, B. B. <thebbzoo@...> wrote:
...

> Background:
> -----------------------
> After update of
>  - an application, with some modules being renamed
>  - python from 2.5 to 2.6
>  - zodb
> an updatescript is called to refresh the broken objects.
> The objects are stored in dict of type <class
> 'zope.container.contained.ContainedProxy'> with a simple unicode key to the
> object mapping.

ContainedProxy isn't a container type.  It is a generic proxy that ads
__parent__ and __named__ attributes.  Does it actually proxy a Python
dict?  What does foo.__class__ return where foo is one of these
containers?

Jim

--
Jim Fulton
_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@...
https://mail.zope.org/mailman/listinfo/zodb-dev

[B. B.]

 

On Sun, Oct 18, 2009 at 9:55 AM, Jim Fulton <jim@...> wrote:

On Sat, Oct 17, 2009 at 12:49 PM, B. B. <thebbzoo@...> wrote:
...

> Background:
> -----------------------
> After update of
>  - an application, with some modules being renamed
>  - python from 2.5 to 2.6
>  - zodb
> an updatescript is called to refresh the broken objects.
> The objects are stored in dict of type <class
> 'zope.container.contained.ContainedProxy'> with a simple unicode key to the
> object mapping.

ContainedProxy isn't a container type.  It is a generic proxy that ads
__parent__ and __named__ attributes.  Does it actually proxy a Python
dict?  What does foo.__class__ return where foo is one of these
containers?

No, it is a persistent container. The prompt say it s OOBTree...

In [12]: sc
Out[12]: <BTrees.OOBTree.OOBTree object at 0x12bd250>

In [13]: type( sc )
Out[13]: <class 'zope.container.contained.ContainedProxy'>

In [14]: sc.__class__
Out[14]: <type 'BTrees.OOBTree.OOBTree'>


/Brian

_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@...
https://mail.zope.org/mailman/listinfo/zodb-dev

[Jim Fulton]

On Sat, Oct 17, 2009 at 12:49 PM, B. B. <thebbzoo@...> wrote:
...

> The precedure is used to work fine, I have tried with different version of
> zodb3 from pypi,
>
> #bad zodb3 = 3.9.2
> #bad zodb3 = 3.9.1
> #bad zodb3 = 3.9.0
> #bad zodb3 = 3.9.0c2
> #ok zodb3 = 3.9.0c1
> #ok zodb3 = 3.9.0b5
> #ok zodb3 = 3.9.0a12

...

> If it is an interesting bug to catch ( for me it is very interesting :-) ),
> I can test versions or patches.

I found and fixed 2 BTree bugs introduced in 3.9.0c2 that probably
account for the problem you're seeing. Could you test against the 3.9
branch:

  svn://svn.zope.org/repos/main/ZODB/branches/3.9

and let me know if the fix corrects the problem you're seeing?

Jim

--
Jim Fulton
_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@...
https://mail.zope.org/mailman/listinfo/zodb-dev

[B. B.]

On Thu, Oct 22, 2009 at 2:33 PM, Jim Fulton <jim@...> wrote:

On Sat, Oct 17, 2009 at 12:49 PM, B. B. <thebbzoo@...> wrote:
...

> The precedure is used to work fine, I have tried with different version of
> zodb3 from pypi,
>
> #bad zodb3 = 3.9.2
> #bad zodb3 = 3.9.1
> #bad zodb3 = 3.9.0
> #bad zodb3 = 3.9.0c2
> #ok zodb3 = 3.9.0c1
> #ok zodb3 = 3.9.0b5
> #ok zodb3 = 3.9.0a12

...

> If it is an interesting bug to catch ( for me it is very interesting :-) ),
> I can test versions or patches.

I found and fixed 2 BTree bugs introduced in 3.9.0c2 that probably
account for the problem you're seeing. Could you test against the 3.9
branch:

 svn://svn.zope.org/repos/main/ZODB/branches/3.9

and let me know if the fix corrects the problem you're seeing?

Jim

--
Jim Fulton

 

#ok zodb3 = 3.9.3
#bad zodb3 = 3.9.2
#bad zodb3 = 3.9.1
#bad zodb3 = 3.9.0
#bad zodb3 = 3.9.0c2
#ok zodb3 = 3.9.0c1
#ok zodb3 = 3.9.0b5
#ok zodb3 = 3.9.0a12

Checked against the 3.9.3 version from PyPi.
 
The problem is not to be observed anymore, it works fine!!!


Thanks

\Brian

_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@...
https://mail.zope.org/mailman/listinfo/zodb-dev