How do I find out the size of a subtree of the object graph?

View: New views
10 Messages — Rating Filter:   Alert me  

How do I find out the size of a subtree of the object graph?

by Chris Withers :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi All,

As the subject line says with the following definitions:

- size = size on disk, in Mb

- subtee = Zope folderish type things, in this case...

cheers,

Chris

--
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk
_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@...
https://mail.zope.org/mailman/listinfo/zodb-dev

Re: How do I find out the size of a subtree of the object graph?

by Jim Fulton :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Mon, Sep 14, 2009 at 5:55 AM, Chris Withers <chris@...> wrote:
> Hi All,
>
> As the subject line says with the following definitions:
>
> - size = size on disk, in Mb
>
> - subtee = Zope folderish type things, in this case...

ZODB doesn't provide any direct support because it has no notion of tree.

In Zope 2, you could export the folder and look at the size of the export.

Jim

--
Jim Fulton
_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@...
https://mail.zope.org/mailman/listinfo/zodb-dev

Re: How do I find out the size of a subtree of the object graph?

by Chris Withers :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Jim Fulton wrote:
> ZODB doesn't provide any direct support because it has no notion of tree.

Does it have any notion of object size on disk for a given object?

> In Zope 2, you could export the folder and look at the size of the export.

Sadly, when you're trying to figure out which folder is using all the
space on a disk that's running out of space, that's not practical...

Chris

--
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk
_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@...
https://mail.zope.org/mailman/listinfo/zodb-dev

Re: How do I find out the size of a subtree of the object graph?

by Jens Vagelpohl :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message


On Sep 14, 2009, at 13:10 , Chris Withers wrote:

> Jim Fulton wrote:
>> ZODB doesn't provide any direct support because it has no notion of  
>> tree.
>
> Does it have any notion of object size on disk for a given object?
>
>> In Zope 2, you could export the folder and look at the size of the  
>> export.
>
> Sadly, when you're trying to figure out which folder is using all the
> space on a disk that's running out of space, that's not practical...
How about simply writing a script that crawls the ZODB and gives you  
paths and sizes?

jens




_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@...
https://mail.zope.org/mailman/listinfo/zodb-dev

smime.p7s (6K) Download Attachment

Re: How do I find out the size of a subtree of the object graph?

by Chris Withers :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Jens Vagelpohl wrote:
>
>> Sadly, when you're trying to figure out which folder is using all the
>> space on a disk that's running out of space, that's not practical...
>
> How about simply writing a script that crawls the ZODB and gives you
> paths and sizes?

Indeed, and what's the magic incantation to get the size of the object
as it is on disk?

cheers,

Chris

--
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk
_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@...
https://mail.zope.org/mailman/listinfo/zodb-dev

Re: How do I find out the size of a subtree of the object graph?

by Andreas Jung-5 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On 14.09.09 13:27, Chris Withers wrote:

> Jens Vagelpohl wrote:
>  
>>    
>>> Sadly, when you're trying to figure out which folder is using all the
>>> space on a disk that's running out of space, that's not practical...
>>>      
>> How about simply writing a script that crawls the ZODB and gives you
>> paths and sizes?
>>    
> Indeed, and what's the magic incantation to get the size of the object
> as it is on disk?
Perhaps useful:

https://bugs.launchpad.net/zodb/+bug/424172

(but pretty low-level).

Andreas

[lists.vcf]

begin:vcard
fn:Andreas Jung
n:Jung;Andreas
org:ZOPYX Ltd. & Co. KG
adr;quoted-printable:;;Charlottenstr. 37/1;T=C3=BCbingen;;72070;Germany
email;internet:info@...
title:CEO
tel;work:+49-7071-793376
tel;fax:+49-7071-7936840
tel;home:+49-7071-793257
x-mozilla-html:FALSE
url:www.zopyx.com
version:2.1
end:vcard



_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@...
https://mail.zope.org/mailman/listinfo/zodb-dev

Re: How do I find out the size of a subtree of the object graph?

by Jens Vagelpohl :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message


On Sep 14, 2009, at 13:27 , Chris Withers wrote:

> Jens Vagelpohl wrote:
>>> Sadly, when you're trying to figure out which folder is using all  
>>> the
>>> space on a disk that's running out of space, that's not practical...
>> How about simply writing a script that crawls the ZODB and gives  
>> you paths and sizes?
>
> Indeed, and what's the magic incantation to get the size of the  
> object as it is on disk?

There is none that I know. I would simply look at the data items I  
know from the object in the ZODB. Like, I know how OFS.Image and  
OFS.File save their data and can determine the size that way. You  
would have to have knowledge about each type of object encountered.

jens




_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@...
https://mail.zope.org/mailman/listinfo/zodb-dev

smime.p7s (6K) Download Attachment

Re: How do I find out the size of a subtree of the object graph?

by Jim Fulton :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Mon, Sep 14, 2009 at 7:10 AM, Chris Withers <chris@...> wrote:
> Jim Fulton wrote:
>>
>> ZODB doesn't provide any direct support because it has no notion of tree.
>
> Does it have any notion of object size on disk for a given object?

Not directly, but you can do:

  len(ob._p_jar.db().storage.load(ob._p_oid, '')[0])+42

Jim

--
Jim Fulton
_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@...
https://mail.zope.org/mailman/listinfo/zodb-dev

Re: How do I find out the size of a subtree of the object graph?

by davisagli :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Sep 14, 2009, at 7:04 AM, Jim Fulton wrote:

> On Mon, Sep 14, 2009 at 7:10 AM, Chris Withers  
> <chris@...> wrote:
>> Jim Fulton wrote:
>>>
>>> ZODB doesn't provide any direct support because it has no notion  
>>> of tree.
>>
>> Does it have any notion of object size on disk for a given object?
>
> Not directly, but you can do:
>
>  len(ob._p_jar.db().storage.load(ob._p_oid, '')[0])+42


What's the extra 42 for?  Aside from life, the universe, and  
everything...


David Glick
Web Developer
ONE/Northwest

New tools and strategies for engaging people in protecting the  
environment

http://www.onenw.org
davidglick@...
work: (206) 286-1235 x32
mobile: (206) 679-3833

Subscribe to ONEList, our email newsletter!
Practical advice for effective online engagement
http://www.onenw.org/full_signup




_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@...
https://mail.zope.org/mailman/listinfo/zodb-dev

Re: How do I find out the size of a subtree of the object graph?

by Jim Fulton :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Mon, Sep 14, 2009 at 10:43 AM, David Glick <davidglick@...> wrote:

> On Sep 14, 2009, at 7:04 AM, Jim Fulton wrote:
>>
>> On Mon, Sep 14, 2009 at 7:10 AM, Chris Withers <chris@...>
>> wrote:
>>>
>>> Jim Fulton wrote:
>>>>
>>>> ZODB doesn't provide any direct support because it has no notion of
>>>> tree.
>>>
>>> Does it have any notion of object size on disk for a given object?
>>
>> Not directly, but you can do:
>>
>>  len(ob._p_jar.db().storage.load(ob._p_oid, '')[0])+42
>
>
> What's the extra 42 for?  Aside from life, the universe, and everything...

Database record header. There's also space for transaction meta data,
but that's amorized over an unknown number of records.

Jim

--
Jim Fulton
_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@...
https://mail.zope.org/mailman/listinfo/zodb-dev