« Return to Thread: Caching of describeTable
Re: Caching of describeTable
by Shahar Evron
::
Rate this Message:
Personally I think there's a room for both solutions.
I think that in Zend_Db_Table_* classes, we just need to add an optional
parameter initialized to null, holding the table schema (let's call it
$_schema). If not defined, this parameter would be populated in runtime
by describeTable() as happens now - and this can be cached or not
cached, according to another parameter ($_schema_cache_ttl = 0 for
example).
If someone wants code generation a-la-Qcodo, they can do it by writing
some tool to automatically populate the $_schema property - but I don't
think we *must* provide such tool, at least not it the first stage.
In my opinion, we get both enough simplicity and enough flexibility this
way. I do think that allowing the developer to specify the table
structure in advance is a good idea - after all, performance wise on
production this would be better even than caching.
Shahar.
Gavin Vess wrote:
> Where the schema information goes is more a matter of perspective and
> preference, whether cached via Zend_Cache, or cached the Qcodo way.
> Qcodo has a much more robust API for managing changes and
> programmatically interacting with the schema, than just a serialization
> of describeTable()'s output cached in Zend_Cache, but then I'm bordering
> on comparing an apple pie to an orange ;)
>
> BTW, the solutions I've seen using TTL's have extra complexity to work
> around distributed environments (e.g. more than one edge/web server).
>
> Cheers,
> Gavin
>
> Bill Karwin wrote:
>> It's true that requiring a describe() on every page request is
>> costly. I've seen reports that performance is poor for describe and
>> other usage of the INFORMATION_SCHEMA virtual tables, in MySQL. Not
>> surprising that the feature was not designed for high performance,
>> under the assumption that will be queried infrequently.
>>
>> Caching the metadata automagically is one solution. TTL is probably
>> of the least expensive way to get some benefit out of the cache, but
>> allow for changes to occur without requiring manual intervention to
>> update the app's view of the metadata. There should also be some
>> method to purge the cache before the TTL expires. I'd hate to be that
>> developer telling the Marketing VP that we've changed the schema, but
>> the app won't notice the change for another two hours!
>>
>> One should be able to tune the TTL down to zero or near zero during
>> development. Schema changes are much more frequent during
>> development, and making the TTL tunable would be the most transparent
>> way to handle this.
>>
>> I don't think putting schema details into the code is the best
>> solution. This would be equivalent to putting a schema description
>> into a config file, which is one thing we're committed to avoiding in
>> Zend Framework. With that solution, it is difficult to keep the
>> schema description in sync with the actual schema. Letting the Db
>> component refresh this for you and cache the results for you would be
>> a better solution.
>>
>> Bill
>>
>> Andi Gutmans wrote:
>>> I think there are a few options here which we discussed in the past:
>>> a) Use Zend_Cache to cache the info with a TTL.
>>> b) Implement a very lightweight Zend_Db specific cache with TTL.
>>> c) Auto-generate classes (i.e. manual update). Theoretically that and
>>> Cache
>>> are similar solutions as Cache can also be manually generated.
>>>
>>> Anyway, Bill is working right now to come with a suggestion for what the
>>> scope of Zend_Db is and if we need an additional layer, what that
>>> would be.
>>> This issue will definitely be taken into account.
>>>
>>> Thanks.
>>>
>>> Andi
>>>
>>>
>>>> -----Original Message-----
>>>> From: Shahar Evron [mailto:shahar.e@...] Sent: Monday, January
>>>> 01, 2007 6:15 AM
>>>> To: fw-db@...
>>>> Subject: [fw-db] Caching of describeTable
>>>>
>>>> Hi,
>>>>
>>>> Maybe I just need to RTFM but this one is a quickie: Is there a
>>>> caching solution for Zend_Db_Adapter_xxx::describeTable() ?
>>>>
>>>> That one gets called quite frequently, and is very unlikely to
>>>> change in production environemtns. Caching it somehow would be a
>>>> smart idea.
>>>>
>>>> If nobody has suggestions, I'll do some hacking later and post an
>>>> article about it.
>>>>
>>>> TIA,
>>>>
>>>> Shahar.
>
I think that in Zend_Db_Table_* classes, we just need to add an optional
parameter initialized to null, holding the table schema (let's call it
$_schema). If not defined, this parameter would be populated in runtime
by describeTable() as happens now - and this can be cached or not
cached, according to another parameter ($_schema_cache_ttl = 0 for
example).
If someone wants code generation a-la-Qcodo, they can do it by writing
some tool to automatically populate the $_schema property - but I don't
think we *must* provide such tool, at least not it the first stage.
In my opinion, we get both enough simplicity and enough flexibility this
way. I do think that allowing the developer to specify the table
structure in advance is a good idea - after all, performance wise on
production this would be better even than caching.
Shahar.
Gavin Vess wrote:
> Where the schema information goes is more a matter of perspective and
> preference, whether cached via Zend_Cache, or cached the Qcodo way.
> Qcodo has a much more robust API for managing changes and
> programmatically interacting with the schema, than just a serialization
> of describeTable()'s output cached in Zend_Cache, but then I'm bordering
> on comparing an apple pie to an orange ;)
>
> BTW, the solutions I've seen using TTL's have extra complexity to work
> around distributed environments (e.g. more than one edge/web server).
>
> Cheers,
> Gavin
>
> Bill Karwin wrote:
>> It's true that requiring a describe() on every page request is
>> costly. I've seen reports that performance is poor for describe and
>> other usage of the INFORMATION_SCHEMA virtual tables, in MySQL. Not
>> surprising that the feature was not designed for high performance,
>> under the assumption that will be queried infrequently.
>>
>> Caching the metadata automagically is one solution. TTL is probably
>> of the least expensive way to get some benefit out of the cache, but
>> allow for changes to occur without requiring manual intervention to
>> update the app's view of the metadata. There should also be some
>> method to purge the cache before the TTL expires. I'd hate to be that
>> developer telling the Marketing VP that we've changed the schema, but
>> the app won't notice the change for another two hours!
>>
>> One should be able to tune the TTL down to zero or near zero during
>> development. Schema changes are much more frequent during
>> development, and making the TTL tunable would be the most transparent
>> way to handle this.
>>
>> I don't think putting schema details into the code is the best
>> solution. This would be equivalent to putting a schema description
>> into a config file, which is one thing we're committed to avoiding in
>> Zend Framework. With that solution, it is difficult to keep the
>> schema description in sync with the actual schema. Letting the Db
>> component refresh this for you and cache the results for you would be
>> a better solution.
>>
>> Bill
>>
>> Andi Gutmans wrote:
>>> I think there are a few options here which we discussed in the past:
>>> a) Use Zend_Cache to cache the info with a TTL.
>>> b) Implement a very lightweight Zend_Db specific cache with TTL.
>>> c) Auto-generate classes (i.e. manual update). Theoretically that and
>>> Cache
>>> are similar solutions as Cache can also be manually generated.
>>>
>>> Anyway, Bill is working right now to come with a suggestion for what the
>>> scope of Zend_Db is and if we need an additional layer, what that
>>> would be.
>>> This issue will definitely be taken into account.
>>>
>>> Thanks.
>>>
>>> Andi
>>>
>>>
>>>> -----Original Message-----
>>>> From: Shahar Evron [mailto:shahar.e@...] Sent: Monday, January
>>>> 01, 2007 6:15 AM
>>>> To: fw-db@...
>>>> Subject: [fw-db] Caching of describeTable
>>>>
>>>> Hi,
>>>>
>>>> Maybe I just need to RTFM but this one is a quickie: Is there a
>>>> caching solution for Zend_Db_Adapter_xxx::describeTable() ?
>>>>
>>>> That one gets called quite frequently, and is very unlikely to
>>>> change in production environemtns. Caching it somehow would be a
>>>> smart idea.
>>>>
>>>> If nobody has suggestions, I'll do some hacking later and post an
>>>> article about it.
>>>>
>>>> TIA,
>>>>
>>>> Shahar.
>
« Return to Thread: Caching of describeTable
| Free embeddable forum powered by Nabble | Forum Help |
