zodb design document

[Jürgen Herrmann]

hi there!

is there a zodb design document?

what i'm interested in are the following things:
 - some basic description how the zodb works (f.ex. i don't understand
   what the difference between a serial and a transaction id is)?
 - looking at basestorage and the methods that have to be implemented,
   is there documentation what the reimplementation in a concrete
   subclass has to do exactly? (ordering of things, desired side effects
   etc.)?

why do i ask? i'd like to create a RadosStorage zodb backend ceph's
underlying object storage "RADOS" (see
http://ceph.newdream.net/blog/category/rados/ )

thanks in advance for your replies!

best regards,
jürgen herrmann
--
>> XLhost.de - eXperts in Linux hosting ® <<

XLhost.de GmbH
Jürgen Herrmann, Geschäftsführer
Boelckestrasse 21, 93051 Regensburg, Germany

Geschäftsführer: Volker Geith, Jürgen Herrmann
Registriert unter: HRB9918
Umsatzsteuer-Identifikationsnummer: DE245931218

Fon:  +49 (0)800 XLHOSTDE [0800 95467833]
Fax:  [phone]

WEB:  http://www.XLhost.de
IRC:  #XLhost@...

_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@...
http://mail.zope.org/mailman/listinfo/zodb-dev

[Jim Fulton]

On Wed, Aug 12, 2009 at 10:33 AM, Jürgen
Herrmann<Juergen.Herrmann@...> wrote:
> hi there!
>
> is there a zodb design document?

There's: http://www.zope.org/Documentation/Developer/Models/ZODB

But it is pretty old and out of date.

> what i'm interested in are the following things:
>  - some basic description how the zodb works (f.ex. i don't understand
>   what the difference between a serial and a transaction id is)?

There is no difference. I thought there could be when I originally
designed ZODB, but it turns out to be good enough to use the tid as
the serial.

>  - looking at basestorage and the methods that have to be implemented,
>   is there documentation what the reimplementation in a concrete
>   subclass has to do exactly? (ordering of things, desired side effects
>   etc.)?

You should look at the storage interfaces in ZODB.interfaces and in
ZEO.interfaces. You should have a firm understanding of how
transactions work in general.  Our transaction system is based on
time-stamps.

You should also ask questions here.

> why do i ask? i'd like to create a RadosStorage zodb backend ceph's
> underlying object storage "RADOS" (see
> http://ceph.newdream.net/blog/category/rados/ )

That's interesting.  You might want to look at DirectoryStorage,
http://dirstorage.sourceforge.net/, which implements the storage APIs
(at least as they existed a few years ago) on top if a file system.

Jim


--
Jim Fulton
_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@...
http://mail.zope.org/mailman/listinfo/zodb-dev

[Jürgen Herrmann]

On Wed, August 12, 2009 17:58, Jim Fulton wrote:
> On Wed, Aug 12, 2009 at 10:33 AM, Jürgen
> Herrmann<Juergen.Herrmann@...> wrote:
>> hi there!
>>
>> is there a zodb design document?
>
> There's: http://www.zope.org/Documentation/Developer/Models/ZODB
>
> But it is pretty old and out of date.

thanks, that was valueable information, especially the collab diagrams
for transaction commit/abort.
>
>> what i'm interested in are the following things:
>>  - some basic description how the zodb works (f.ex. i don't understand
>>   what the difference between a serial and a transaction id is)?
>
> There is no difference. I thought there could be when I originally
> designed ZODB, but it turns out to be good enough to use the tid as
> the serial.
thought so, but i wasn't sure enough :)

>
>>  - looking at basestorage and the methods that have to be implemented,
>>   is there documentation what the reimplementation in a concrete
>>   subclass has to do exactly? (ordering of things, desired side effects
>>   etc.)?
>
> You should look at the storage interfaces in ZODB.interfaces and in
> ZEO.interfaces. You should have a firm understanding of how
> transactions work in general.  Our transaction system is based on
> time-stamps.
>
> You should also ask questions here.
>
>> why do i ask? i'd like to create a RadosStorage zodb backend ceph's
>> underlying object storage "RADOS" (see
>> http://ceph.newdream.net/blog/category/rados/ )
>
> That's interesting.  You might want to look at DirectoryStorage,
> http://dirstorage.sourceforge.net/, which implements the storage APIs
> (at least as they existed a few years ago) on top if a file system.
>

ok, looking at the interfaces and some more implementation code from
FileStoreage, RelStorage, MappingStorage, MVCCMappingStorage etc.
helped a lot. For now, one more questions though:

i want RadosStorage to implement IMVCCStorage. so my understanding
is that i remeber last_polled_tid (from invalidation polling and
instead of loading the latest revision of an object, the storage
always returns state as of the transaction with last_polled_tid,
is that correct? if RadosStorage implements IMVCCStorage i guess i
can skip the zeo interfaces, right?

thanks for your help!
jürgen
--
>> XLhost.de - eXperts in Linux hosting ® <<

XLhost.de GmbH
Jürgen Herrmann, Geschäftsführer
Boelckestrasse 21, 93051 Regensburg, Germany

Geschäftsführer: Volker Geith, Jürgen Herrmann
Registriert unter: HRB9918
Umsatzsteuer-Identifikationsnummer: DE245931218

Fon:  +49 (0)800 XLHOSTDE [0800 95467833]
Fax:  [phone]

WEB:  http://www.XLhost.de
IRC:  #XLhost@...

_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@...
http://mail.zope.org/mailman/listinfo/zodb-dev

[Shane Hathaway]

Jürgen Herrmann wrote:
> i want RadosStorage to implement IMVCCStorage. so my understanding
> is that i remeber last_polled_tid (from invalidation polling and
> instead of loading the latest revision of an object, the storage
> always returns state as of the transaction with last_polled_tid,
> is that correct? if RadosStorage implements IMVCCStorage i guess i
> can skip the zeo interfaces, right?

Yes and yes.  Also, just to make sure you noticed, if you implement
IMVCCStorage then you'll have to require ZODB 3.9+.

Shane

_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@...
http://mail.zope.org/mailman/listinfo/zodb-dev