Problem with Annotations

[PAW]

Hi there,

I've been having difficulty thinking this one through, any hints appreciated.

I'm writing a hobby framework (and content management system) with
Grok. The content will have meta-data associated with it. Naturally,
I've decided to use annotations to model this meta data. I don't know
what meta data will be stored up front; opting instead to allow such a
facility to be plugged in. Let's say a have an object called 'foo'
that is found at location:

http://localhost/container/foo

A plugin may, for example, define an IMetaData interface and MetaData
annotation that stores the X and Y coordinate of foo. When a client
accesses the metadata, it is the interface's schema that is used to
deserialise it. So, what I need is some URL format or mechanism that
allows me to reference an interface so that I can adapt the content
object to it and retrieve the object and then go on to deserialise it.

So the first question is, is there some way to look up (in the ZCA) an
interface by name? This would allow:

http://localhost/container/foo/_metadata

I would then go on to define a grok.Traverser that strips the leading
slash and looks up the interface.

My second question is related. Is there a way to avoid having this
naming convention of the leading underscore or similar. Views have @@,
and layers have ++Layer++, is there some way of doing something
similar in grok to say 'this is not an object, please treat it
differently'?

Thanks in advance.
Regards,
Paul
_______________________________________________
Grok-dev mailing list
Grok-dev@...
https://mail.zope.org/mailman/listinfo/grok-dev

[Sebastian Ware]

Personally I would probably have one Annotation class that handles all  
the meta data. I would define a view (orinary or JSON) to access the  
meta data (rather than using traverse).

   http://server:port/path/to/object/meta_data_view

For simple meta data you might want to store them in a PersistentDict  
in order to avoid multiple db-lookups. If the meta data is large, you  
will want a BTree-style container (such as grok.Container).

Mvh Sebastian

On 23 sep 2009, at 12.15, Paul Wilson wrote:

> Hi there,
>
> I've been having difficulty thinking this one through, any hints  
> appreciated.
>
> I'm writing a hobby framework (and content management system) with
> Grok. The content will have meta-data associated with it. Naturally,
> I've decided to use annotations to model this meta data. I don't know
> what meta data will be stored up front; opting instead to allow such a
> facility to be plugged in. Let's say a have an object called 'foo'
> that is found at location:
>
> http://localhost/container/foo
>
> A plugin may, for example, define an IMetaData interface and MetaData
> annotation that stores the X and Y coordinate of foo. When a client
> accesses the metadata, it is the interface's schema that is used to
> deserialise it. So, what I need is some URL format or mechanism that
> allows me to reference an interface so that I can adapt the content
> object to it and retrieve the object and then go on to deserialise it.
>
> So the first question is, is there some way to look up (in the ZCA) an
> interface by name? This would allow:
>
> http://localhost/container/foo/_metadata
>
> I would then go on to define a grok.Traverser that strips the leading
> slash and looks up the interface.
>
> My second question is related. Is there a way to avoid having this
> naming convention of the leading underscore or similar. Views have @@,
> and layers have ++Layer++, is there some way of doing something
> similar in grok to say 'this is not an object, please treat it
> differently'?
>
> Thanks in advance.
> Regards,
> Paul
> _______________________________________________
> Grok-dev mailing list
> Grok-dev@...
> https://mail.zope.org/mailman/listinfo/grok-dev

_______________________________________________
Grok-dev mailing list
Grok-dev@...
https://mail.zope.org/mailman/listinfo/grok-dev

[Souheil CHELFOUH]

Paul, first of all, i'm glad to learn that you are building a
framework/CMS for Grok
I didn't hear about it and I'm also working on a similar project : Dolmen
http://gitweb.dolmen-project.org/


About your problem, I'd rather use a namespace traverser as they are
elegant and readable:
url/to/object/++ns++name/rest/of/url

But, your metadata issue sounds weird to me
Can you explain in more details ?


2009/9/23 Sebastian Ware <sebastian@...>:

> Personally I would probably have one Annotation class that handles all
> the meta data. I would define a view (orinary or JSON) to access the
> meta data (rather than using traverse).
>
>   http://server:port/path/to/object/meta_data_view
>
> For simple meta data you might want to store them in a PersistentDict
> in order to avoid multiple db-lookups. If the meta data is large, you
> will want a BTree-style container (such as grok.Container).
>
> Mvh Sebastian
>
> On 23 sep 2009, at 12.15, Paul Wilson wrote:
>
>> Hi there,
>>
>> I've been having difficulty thinking this one through, any hints
>> appreciated.
>>
>> I'm writing a hobby framework (and content management system) with
>> Grok. The content will have meta-data associated with it. Naturally,
>> I've decided to use annotations to model this meta data. I don't know
>> what meta data will be stored up front; opting instead to allow such a
>> facility to be plugged in. Let's say a have an object called 'foo'
>> that is found at location:
>>
>> http://localhost/container/foo
>>
>> A plugin may, for example, define an IMetaData interface and MetaData
>> annotation that stores the X and Y coordinate of foo. When a client
>> accesses the metadata, it is the interface's schema that is used to
>> deserialise it. So, what I need is some URL format or mechanism that
>> allows me to reference an interface so that I can adapt the content
>> object to it and retrieve the object and then go on to deserialise it.
>>
>> So the first question is, is there some way to look up (in the ZCA) an
>> interface by name? This would allow:
>>
>> http://localhost/container/foo/_metadata
>>
>> I would then go on to define a grok.Traverser that strips the leading
>> slash and looks up the interface.
>>
>> My second question is related. Is there a way to avoid having this
>> naming convention of the leading underscore or similar. Views have @@,
>> and layers have ++Layer++, is there some way of doing something
>> similar in grok to say 'this is not an object, please treat it
>> differently'?
>>
>> Thanks in advance.
>> Regards,
>> Paul
>> _______________________________________________
>> Grok-dev mailing list
>> Grok-dev@...
>> https://mail.zope.org/mailman/listinfo/grok-dev
>
> _______________________________________________
> Grok-dev mailing list
> Grok-dev@...
> https://mail.zope.org/mailman/listinfo/grok-dev
>

_______________________________________________
Grok-dev mailing list
Grok-dev@...
https://mail.zope.org/mailman/listinfo/grok-dev

[PAW]

2009/9/23 Souheil CHELFOUH <trollfot@...>:
> Paul, first of all, i'm glad to learn that you are building a
> framework/CMS for Grok
> I didn't hear about it and I'm also working on a similar project : Dolmen
> http://gitweb.dolmen-project.org/

It's a very domain-specific project and certainly not a general
purpose affair. It is more towards a framework that allows you to plug
in states as part of a statemachine (thanks martian!!) and have them
executed by some a runtime. It's very early but I'm having lots of fun
with it!

> About your problem, I'd rather use a namespace traverser as they are
> elegant and readable:
> url/to/object/++ns++name/rest/of/url
>
> But, your metadata issue sounds weird to me
> Can you explain in more details ?

Imagine a statemachine system that is constructed through the web, and
then executed later using a statemachine runtime. The states are the
"content" aspect, since uses can edit, version and manipulated these
according to permissions. The states are wired as plugins, stored in
the ZODB and serialized according to their interface schema into JSON.
The web client can view these states, but might want to additionally
store X,Y coordinates and other such metadata along with it for its
own purposes, whereas the runtime might want to track how many times
the state has been executed, and how many exceptions it has thrown (or
whatever) for statistics purposes. Both access the information
RESTfully.

The states (and transitions) are in a nested hierarchy of containment
(which is a state diagram), which lends itself naturally to models and
containers. So the web client get a JSON representation of a state
something like this:

http://localhost/state_diagram/state1

I've got this working nicely as it stands. But how to access meta
data? I don't want metadata to be an intrinsic part of the
statemachine code, rather it should be added through a series of
plugins. So when a client requests:

http://localhost/state_diagram/state1/positional

I want "positional" to map _somehow_ to IPositionalAnnotation, such
that I can adapt 'state1' to it, and then use it to serialize the
object and return it to the agent. Like wise with PUT. The important
thing here is that the rest code doesn't have hard coded knowledge of
what annotations it is to support, and should instead retrieve it
dynamically through plugged-in facilities.

There are two problems I can see with this;

  1) How do I prevent traversal thinking that state1 is actually a
diagram, and that positional is a state in that diagram.
  2) How do I get the interface from the "positional" string?

Actually this has just given me an idea regarding 2) ...does Grok
support named annotations? Looking at:

http://svn.zope.org/grokcore.annotation/trunk/src/grokcore/annotation/tests/annotation/name.py?rev=104270&view=markup

...it looks like it does.

Perhaps I could supply a custom traverser that does:

availables_ann = IAnnotations(self.context)
try:
  ann_class = availables_ann[name]
except: ...

iface = list(providedBy(ann_class).flattened())[0]

And then have the web client and runtime implement their own named
annotations.... I'll give it a try, see how it works.

Regards,
Paul
_______________________________________________
Grok-dev mailing list
Grok-dev@...
https://mail.zope.org/mailman/listinfo/grok-dev

[Ethan Jucovy-3]

On Wed, Sep 23, 2009 at 9:57 AM, Paul Wilson <paulalexwilson@...> wrote:
[snip]
>> About your problem, I'd rather use a namespace traverser as they are
>> elegant and readable:
>> url/to/object/++ns++name/rest/of/url
>>
>> But, your metadata issue sounds weird to me
>> Can you explain in more details ?
[snip]

> The states (and transitions) are in a nested hierarchy of containment
> (which is a state diagram), which lends itself naturally to models and
> containers. So the web client get a JSON representation of a state
> something like this:
>
> http://localhost/state_diagram/state1
>
> I've got this working nicely as it stands. But how to access meta
> data? I don't want metadata to be an intrinsic part of the
> statemachine code, rather it should be added through a series of
> plugins. So when a client requests:
>
> http://localhost/state_diagram/state1/positional
>
> I want "positional" to map _somehow_ to IPositionalAnnotation, such
> that I can adapt 'state1' to it, and then use it to serialize the
> object and return it to the agent. Like wise with PUT. The important
> thing here is that the rest code doesn't have hard coded knowledge of
> what annotations it is to support, and should instead retrieve it
> dynamically through plugged-in facilities.
>
> There are two problems I can see with this;
>
>  1) How do I prevent traversal thinking that state1 is actually a
> diagram, and that positional is a state in that diagram.
>  2) How do I get the interface from the "positional" string?
>
> Actually this has just given me an idea regarding 2) ...does Grok
> support named annotations? Looking at:
>
> http://svn.zope.org/grokcore.annotation/trunk/src/grokcore/annotation/tests/annotation/name.py?rev=104270&view=markup
>
> ...it looks like it does.
>
> Perhaps I could supply a custom traverser that does:
>
> availables_ann = IAnnotations(self.context)
> try:
>  ann_class = availables_ann[name]
> except: ...
>
> iface = list(providedBy(ann_class).flattened())[0]

I'm not sure I'm following fully, and it's been way too long since
I've used Grok at all, but ..

I think I'd do this with named adapters, and use the URL component as
the name.  It adds an extra layer of indirection, though, since you
wouldn't be adapting the content object directly to the
metadata/annotation interface -- instead to a generic metadata
provider interface.  Very shaky pseudocode:

class IMetadataProvider(Interface):
  def get_metadata():
    """ returns an IAnnotations """
  def put_metadata(**kw):
    """ hmm, what? """

class PositionalMetadataProvider(grok.Adapter):
  grok.context(ContentObject)
  grok.implements(IMetadataProvider)
  grok.name('positional')

class FolksonomyMetadataProvider(object):
  grok.context(ContentObject)
  grok.implements(IMetadataProvider)
  grok.name('tags')

And then in your traverser, `return
annotation_converted_to_response(IMetadataProvider(content_object,
name=name).get_metadata())`.

--- actually, having written that out, the named adapters look like
they are just views -- IMetadataProvider's get_metadata and
put_metadata could just interact with a browser request & response
directly, and each plugin would define a view that knows how to get
the right annotation.  Hand-waving furiously here, I think most of the
logic (looking up the annotation, de/serializing, error handling)
could be centralized in a base class.  Each plugged-in subclass could
then house all the knowledge about that plugin's URL component and
annotation key & format, and register itself as a view.

Not sure if this is making any sense, or is relevant to your problem.
:)  At any rate, good luck!

egj
_______________________________________________
Grok-dev mailing list
Grok-dev@...
https://mail.zope.org/mailman/listinfo/grok-dev

[PAW]

2009/9/23 Ethan Jucovy <ejucovy@...>:
> On Wed, Sep 23, 2009 at 9:57 AM, Paul Wilson <paulalexwilson@...> wrote:
> [snip]

> --- actually, having written that out, the named adapters look like
> they are just views -- IMetadataProvider's get_metadata and
> put_metadata could just interact with a browser request & response
> directly, and each plugin would define a view that knows how to get
> the right annotation.  Hand-waving furiously here, I think most of the
> logic (looking up the annotation, de/serializing, error handling)
> could be centralized in a base class.  Each plugged-in subclass could
> then house all the knowledge about that plugin's URL component and
> annotation key & format, and register itself as a view.
>
> Not sure if this is making any sense, or is relevant to your problem.
> :)  At any rate, good luck!

Hehe. Wow - firstly it looks like you've understood me, and secondly I
like your solution. I think I've been over engineering my solution and
worrying too much about naming issues. I should have realised this
when I started talking about 'pluggable named adapters' :-)

Thanks for clearing it up for me!

Regards,
Paul
_______________________________________________
Grok-dev mailing list
Grok-dev@...
https://mail.zope.org/mailman/listinfo/grok-dev

[Ethan Jucovy-3]

On Wed, Sep 23, 2009 at 3:44 PM, Paul Wilson <paulalexwilson@...> wrote:
> 2009/9/23 Ethan Jucovy <ejucovy@...>:
>> On Wed, Sep 23, 2009 at 9:57 AM, Paul Wilson <paulalexwilson@...> wrote:
>> [snip]
>
>> --- actually, having written that out, the named adapters look like
>> they are just views -- IMetadataProvider's get_metadata and
>> put_metadata could just interact with a browser request & response
>> directly, and each plugin would define a view that knows how to get
>> the right annotation.  Hand-waving furiously here, I think most of the
>> logic (looking up the annotation, de/serializing, error handling)
>> could be centralized in a base class.  Each plugged-in subclass could
>> then house all the knowledge about that plugin's URL component and
>> annotation key & format, and register itself as a view.
>>
>> Not sure if this is making any sense, or is relevant to your problem.
>> :)  At any rate, good luck!
>
> Hehe. Wow - firstly it looks like you've understood me, and secondly I
> like your solution. I think I've been over engineering my solution and
> worrying too much about naming issues. I should have realised this
> when I started talking about 'pluggable named adapters' :-)
>
> Thanks for clearing it up for me!

Happy to help!  I've been trying to think through a similar conceptual problem on my backburner for a while now, but it's been hard for me to get a handle on it in the first place, so you've helped me too by spelling out the problem. :)

It's funny how often I'll think through a solution that sounds overly complex and hand-rolled, only to discover at the end that it's already a pretty fundamental Zope/Grok feature, just seen from a different angle.  "Grok is informed by a lot of hard-earned wisdom" indeed. :)

Anyway, good luck with the project!  It sounds very interesting, domain-specific or no.

Regards,
Ethan
_______________________________________________
Grok-dev mailing list
Grok-dev@...
https://mail.zope.org/mailman/listinfo/grok-dev