Link metadata?

[Mike Schinkel-5]

Hi all:


I'm pondering an architectural decision and wanted to run it by the hackers.

I'm building a plugin to provide "Post Links", i.e. "Links associated with a given Post." I've got a site where I'm using posts in ways that would model a Wikipedia entry more than it would a blog post. For example purposes let's say I had a tourism site that had a post for each of the 50 US states (I don't, this is just an example.) For my example I'd like to have the ability to add links to external travel articles related to each state.

WordPress already has the (almost) perfect data structure for these links: the wp_links table along with the taxonomy tables for defining link categories (i.e. a link category "Post Links") I can also define a custom post field in wp_postmeta that contains a list of the link_ids in comma-seperated form. This would allow me to pass the comma-seperated list of link_ids using the 'include' parameter of get_bookmarks() to get the list of links. All good so far. (Right?)

The next step is that I'd like to import blocks of links into my "Post Links" link category (I've already built the plugin for this) and then set up some workflow that would capture a screenshot for the URL (using another plugin I'm building), download the page title and meta description, and then finally present to me in a form to edit any of the links in much the same way Flickr.com lets you edit titles and descriptions when you mass upload photos.

The problem arises in that there are not any fields in wp_links where I can store anything to tell me which links have had their screenshot generated, their title and description downloaded, nor their titles and descriptions manually reviewed and edited (excepting the "notes" field which IMO is not a viable candidate.) Posts have wp_postmeta and Users has wp_usermeta but there is no wp_linkmeta for Links. My usecase needs some metadata for workflow but I'm sure there are many other uses for link metadata as well.

So here are the solutions I've considered:


1.) Storing everything in a 'links_workflow' option as a serialized array (this just doesn't feel right.)
2.) Creating a wp_linksmeta table and hoping that if WordPress ever adds it they add it like what I've added (Don't want to add a table but would be cleanest for SQL.)
3.) Adding a "link_meta" field to the wp_links table and storing serialized array data for each link (This is okay, but can't do a performant SQL query against.)

Maybe there's a fourth option but if so it hasn't occurred to me.

So, what would you do if you had this need? Also would there be any pushback to seeing #2 or #3 added into core?

Thanks in advance for pondering with me.

-Mike Schinkel
Custom Wordpress Plugins
http://mikeschinkel.com/custom-wordpress-plugins 
_______________________________________________
wp-hackers mailing list
wp-hackers@...
http://lists.automattic.com/mailman/listinfo/wp-hackers

[Malaiac]

2009/6/15 Mike Schinkel <mikeschinkel@...>:
> I'm pondering an architectural decision and wanted to run it by the hackers.
>
> I'm building a plugin to provide "Post Links", i.e. "Links associated with a given Post." I've got a site where I'm using posts in ways

You should check zeList plugin
http://wordpress.org/extend/plugins/zelist/ which includes some of the
things you're looking for.

For link extended fields, I strongly recommend the creation of a
link_meta table (aka option 2), and duplication of post_meta
functions, for two reasons :
- adding the table and copying the functions takes 30mn top.
- if and when WP decides to switch to a taxonomy-neutral meta table it
will be easier to switch from a "roughly similar to postmeta "
architecture than from a custom hack.
In zeList, just check zelist/includes/metas.php for the functions and
zelist/zelist.php (function zelist_install) for the database creation.
You may need to work on the Link Edit Page to add custom metas fields,
since I did not add these.

As for link related to posts, it can be done manually, or automatically.
Check http://cadeaux.laporterie.com/camping/18-acheter-une-caravane.htm
(check "Liens connexes") for an example of zeList in action. The
function is "related_links()" in
zelist/includes/template-functions.php

For the description/title/images parsing, check
http://www.screentoaster.com/watch/stUkJcQ0VIR11cSFtaXFJfVFRS/zelist_submit_page
it includes a zelist plugin (zelist-submitter), not released yet,
using the zeList API.

Let me know if some of that is helping you !

Malaiac
_______________________________________________
wp-hackers mailing list
wp-hackers@...
http://lists.automattic.com/mailman/listinfo/wp-hackers

[Casey Bisson-2]

A while ago I proposed a comments_meta table[1], and at WordCamp I was  
introduced to bbPress' meta table architecture: rather than have meta  
tables for different types of content, there's just one meta table  
that includes an extra column describing what content type the ID  
links to.

The implication for the WP world was that an additional column in the  
postmeta table could extend it to serve links and comments (and if  
we're bold, replace usermeta). Our purposes are different, but having  
a standardized place and set of functions to store metadata about  
things other than posts would deliver obvious value to both of us.

[1]: http://comox.textdrive.com/pipermail/wp-hackers/2009-May/026343.html

All of that said, have you considered adding these links as posts?  
Putting content in the posts table gets you free tools to add tags and  
other metadata, makes the content easier to search (no need for  
plugins that scatter the searching to all those other tables), and  
allows you to open up the links to comments and other social activity.

You'd have to do the hard work of making the links work well as posts,  
but everything else comes free after that (rather than having to  
continually hack the link system to accommodate the interactions you'd  
like to see). But that's not as hard as it might appear -- I did it  
with photos and library records[2] and released the plugin[3].

[2]: see http://archives.colby-sawyer.edu/archives/2552/ and http://collingswoodlib.org/library/12023/
[3]: http://about.scriblio.net/

--Casey Bisson
http://maisonbisson.com/




On Jun 15, 2009, at 2:28 AM, Mike Schinkel wrote:

> The problem arises in that there are not any fields in wp_links  
> where I can store anything to tell me which links have had their  
> screenshot generated, their title and description downloaded, nor  
> their titles and descriptions manually reviewed and edited  
> (excepting the "notes" field which IMO is not a viable candidate.)  
> Posts have wp_postmeta and Users has wp_usermeta but there is no  
> wp_linkmeta for Links. My usecase needs some metadata for workflow  
> but I'm sure there are many other uses for link metadata as well.
>
> So here are the solutions I've considered:
>
> 1.) Storing everything in a 'links_workflow' option as a serialized  
> array (this just doesn't feel right.)
> 2.) Creating a wp_linksmeta table and hoping that if WordPress ever  
> adds it they add it like what I've added (Don't want to add a table  
> but would be cleanest for SQL.)
> 3.) Adding a "link_meta" field to the wp_links table and storing  
> serialized array data for each link (This is okay, but can't do a  
> performant SQL query against.)

_______________________________________________
wp-hackers mailing list
wp-hackers@...
http://lists.automattic.com/mailman/listinfo/wp-hackers

[Mike Schinkel-5]

Casey,

Thanks for the background on the all-in-one meta table. I agree that a standardized place would benefit us both as well as benefit others who may have different needs.

As for making the links posts themselves, I considered but for my needs posts would be far too "heavy" and provide far more capability than I need. Links are the perfect structure for my needs assuming I have link_meta available. BTW, I'm already hacking posts to serve purposes other than typical blog posts, and I don't want to have to worry about yet another special post type.

But that's for the suggestions, it is definitely an option for another use-case.

BTW, Scriblio looks rather interesting, good work.  

-Mike Schinkel
Custom Wordpress Plugins
http://mikeschinkel.com/custom-wordpress-plugins

----- Original Message -----
From: "Casey Bisson" <casey.bisson@...>
To: wp-hackers@...
Sent: Monday, June 15, 2009 12:27:22 PM GMT -05:00 US/Canada Eastern
Subject: Re: [wp-hackers] Link metadata?


A while ago I proposed a comments_meta table[1], and at WordCamp I was  
introduced to bbPress' meta table architecture: rather than have meta  
tables for different types of content, there's just one meta table  
that includes an extra column describing what content type the ID  
links to.

The implication for the WP world was that an additional column in the  
postmeta table could extend it to serve links and comments (and if  
we're bold, replace usermeta). Our purposes are different, but having  
a standardized place and set of functions to store metadata about  
things other than posts would deliver obvious value to both of us.

[1]: http://comox.textdrive.com/pipermail/wp-hackers/2009-May/026343.html

All of that said, have you considered adding these links as posts?  
Putting content in the posts table gets you free tools to add tags and  
other metadata, makes the content easier to search (no need for  
plugins that scatter the searching to all those other tables), and  
allows you to open up the links to comments and other social activity.

You'd have to do the hard work of making the links work well as posts,  
but everything else comes free after that (rather than having to  
continually hack the link system to accommodate the interactions you'd  
like to see). But that's not as hard as it might appear -- I did it  
with photos and library records[2] and released the plugin[3].

[2]: see http://archives.colby-sawyer.edu/archives/2552/ and http://collingswoodlib.org/library/12023/
[3]: http://about.scriblio.net/

--Casey Bisson
http://maisonbisson.com/




On Jun 15, 2009, at 2:28 AM, Mike Schinkel wrote:

> The problem arises in that there are not any fields in wp_links  
> where I can store anything to tell me which links have had their  
> screenshot generated, their title and description downloaded, nor  
> their titles and descriptions manually reviewed and edited  
> (excepting the "notes" field which IMO is not a viable candidate.)  
> Posts have wp_postmeta and Users has wp_usermeta but there is no  
> wp_linkmeta for Links. My usecase needs some metadata for workflow  
> but I'm sure there are many other uses for link metadata as well.
>
> So here are the solutions I've considered:
>
> 1.) Storing everything in a 'links_workflow' option as a serialized  
> array (this just doesn't feel right.)
> 2.) Creating a wp_linksmeta table and hoping that if WordPress ever  
> adds it they add it like what I've added (Don't want to add a table  
> but would be cleanest for SQL.)
> 3.) Adding a "link_meta" field to the wp_links table and storing  
> serialized array data for each link (This is okay, but can't do a  
> performant SQL query against.)

_______________________________________________
wp-hackers mailing list
wp-hackers@...
http://lists.automattic.com/mailman/listinfo/wp-hackers
_______________________________________________
wp-hackers mailing list
wp-hackers@...
http://lists.automattic.com/mailman/listinfo/wp-hackers

[Mike Schinkel-5]

Malaiac:

Thanks for the pointer to zeList.  Are you going to test/update it for use with v2.8?

When I started writing my email I had been planning to add an option #3, a link_meta field. After completing the email I had changed my mind and decided a wp_linkmeta table a.k.a. option #2 would probably be the best. Thanks for confirming my decision to use a wp_linkmeta table, that is assuming I don't end up using zeList directly.

-Mike Schinkel
Custom Wordpress Plugins
http://mikeschinkel.com/custom-wordpress-plugins

----- Original Message -----
From: "Malaiac" <malaiac@...>
To: wp-hackers@...
Sent: Monday, June 15, 2009 4:09:59 AM GMT -05:00 US/Canada Eastern
Subject: Re: [wp-hackers] Link metadata?

2009/6/15 Mike Schinkel <mikeschinkel@...>:
> I'm pondering an architectural decision and wanted to run it by the hackers.
>
> I'm building a plugin to provide "Post Links", i.e. "Links associated with a given Post." I've got a site where I'm using posts in ways

You should check zeList plugin
http://wordpress.org/extend/plugins/zelist/ which includes some of the
things you're looking for.

For link extended fields, I strongly recommend the creation of a
link_meta table (aka option 2), and duplication of post_meta
functions, for two reasons :
- adding the table and copying the functions takes 30mn top.
- if and when WP decides to switch to a taxonomy-neutral meta table it
will be easier to switch from a "roughly similar to postmeta "
architecture than from a custom hack.
In zeList, just check zelist/includes/metas.php for the functions and
zelist/zelist.php (function zelist_install) for the database creation.
You may need to work on the Link Edit Page to add custom metas fields,
since I did not add these.

As for link related to posts, it can be done manually, or automatically.
Check http://cadeaux.laporterie.com/camping/18-acheter-une-caravane.htm
(check "Liens connexes") for an example of zeList in action. The
function is "related_links()" in
zelist/includes/template-functions.php

For the description/title/images parsing, check
http://www.screentoaster.com/watch/stUkJcQ0VIR11cSFtaXFJfVFRS/zelist_submit_page
it includes a zelist plugin (zelist-submitter), not released yet,
using the zeList API.

Let me know if some of that is helping you !

Malaiac
_______________________________________________
wp-hackers mailing list
wp-hackers@...
http://lists.automattic.com/mailman/listinfo/wp-hackers
_______________________________________________
wp-hackers mailing list
wp-hackers@...
http://lists.automattic.com/mailman/listinfo/wp-hackers

[Malaiac]

2009/6/15 Casey Bisson <casey.bisson@...>:
> [1]: http://comox.textdrive.com/pipermail/wp-hackers/2009-May/026343.html
> All of that said, have you considered adding these links as posts? Putting
> content in the posts table gets you free tools to add tags and other
> metadata, makes the content easier to search (no need for plugins that
> scatter the searching to all those other tables), and allows you to open up
> the links to comments and other social activity.

He'll just have to recode the whole post (link) categories taxonomy.
Been there. Never doing that again.

Beyond that, I agree with the much needed central meta table.

Malaiac
_______________________________________________
wp-hackers mailing list
wp-hackers@...
http://lists.automattic.com/mailman/listinfo/wp-hackers

[Malaiac]

2009/6/15 Mike Schinkel <mikeschinkel@...>:
> Malaiac:
> Thanks for the pointer to zeList.  Are you going to test/update it for use with v2.8?
this week !

> When I started writing my email I had been planning to add an option #3, a link_meta field. After completing the email I had changed my mind and decided a wp_linkmeta table a.k.a. option #2 would probably be the best. Thanks for confirming my decision to use a wp_linkmeta table, that is assuming I don't end up using zeList directly.

#3 is good if you have very few values (like 1 to 3 metas)
and even in this case, #2 won't be much more work.
in link_meta, I store : link_updated, link_published, pagerank, and
some custom metas (like zipcode).

as for using zeList, please take it for a test drive, I feel it may
fill your specs quite closely. I'm available for further
doc/explanation on the plugin.

Malaiac
_______________________________________________
wp-hackers mailing list
wp-hackers@...
http://lists.automattic.com/mailman/listinfo/wp-hackers

[Dougal Campbell]

Casey Bisson wrote:
>
> A while ago I proposed a comments_meta table[1], and at WordCamp I was
> introduced to bbPress' meta table architecture: rather than have meta
> tables for different types of content, there's just one meta table
> that includes an extra column describing what content type the ID
> links to.
Ahhh, I had forgotten about that. That's a good suggestion. I've also
pondered the lack of link- and comment- meta, and a unified metadata
table, indexed by content type would be a good evolution, I think. It
would also aid in the future migration towards more generic CMS
capabilities in core WP, since it would be extensible for future data types.

> All of that said, have you considered adding these links as posts?
> Putting content in the posts table gets you free tools to add tags and
> other metadata, makes the content easier to search (no need for
> plugins that scatter the searching to all those other tables), and
> allows you to open up the links to comments and other social activity.

That's what I was about to suggest, too (I think). Or more specifically,
I was going to suggest either adding metadata to the main post, with
info about the child links (could connect by linkid), or even by using a
child-post (more technically, an attachment) to house the extra data.
This isn't as elegant as attaching the metadata directly to the links in
some way, but is pragmatic in that it shouldn't be too hard to
implement. Done right, you could even reference the same link (by its
link ID) from different posts, but using different link metadata in each
instance. I could probably think of some insteresting use-cases for that.

--
Dougal Campbell <dougal@...>
http://dougal.gunters.org/
http://twitter.com/dougal
http://twitual.com/
_______________________________________________
wp-hackers mailing list
wp-hackers@...
http://lists.automattic.com/mailman/listinfo/wp-hackers

[Mike Schinkel-5]

"Dougal Campbell" <dougal@...> wrote:
> It would also aid in the future migration towards more generic
> CMS capabilities in core WP, since it would be extensible for
> future data types.

So is this something the community is ready to start moving toward, with 2.9?  Should we minimally add a wp_meta table to allow support for link, comments, and other meta even if we don't immediately migrate away from wp_postmeta and wp_usermeta?

> I was going to suggest either adding metadata to the main post,
> with info about the child links (could connect by linkid),

I *think* that's what I described as an option, i.e. in wp_postmeta a 'post_links' field that would store the comma-separated link_ids in a single string meta value. If that's not what you where thinking can you explain (in code?)

> or even by using a child-post (more technically, an attachment) to
> house the extra data.

Are you suggesting a post_type='attachment' or instead adding something like post_type='link'?
 
> This isn't as elegant as attaching the metadata directly to the
> links in some way,

Even with link metadata I still somehow need (to simulate) a linking table to associate the links with a post, hence the 'post_links' value in wp_postmeta.

> Done right, you could even reference the same link (by its
link ID) from different posts, but using different link
> metadata in each instance. I could probably think of some
> interesting use-cases for that.

Now that's another level I didn't need for my use-case but it does make it more interesting. We could use the wp_postmeta values for the 'attachment/link' in wp_posts to give each link different metadata than the metadata for the link that would be shared by all.  It actually works well form a database perspective. Of course the "hitch" is building a good UI so an admin user can access all that power. :)


-Mike Schinkel
Custom Wordpress Plugins
http://mikeschinkel.com/custom-wordpress-plugins
_______________________________________________
wp-hackers mailing list
wp-hackers@...
http://lists.automattic.com/mailman/listinfo/wp-hackers

[scribu]

Submitted a ticket on the topic: http://core.trac.wordpress.org/ticket/10233

--
http://scribu.net
_______________________________________________
wp-hackers mailing list
wp-hackers@...
http://lists.automattic.com/mailman/listinfo/wp-hackers

[Mike Schinkel-5]

scribu:

Thanks for submitting.  Unfortunately trying to decipher the ones that came before (not your fault) is like try to find the holy grail!

   http://core.trac.wordpress.org/ticket/10233
   http://core.trac.wordpress.org/ticket/5183
   http://core.trac.wordpress.org/ticket/5153
   http://core.trac.wordpress.org/ticket/2659   

So what the heck is the resolution?  One table, multiple tables, overload taxonomy, overload wp_options, do without?!?  Of course you may be as confused as me.

-Mike Schinkel
Custom Wordpress Plugins
http://mikeschinkel.com/custom-wordpress-plugins

----- Original Message -----
From: "scribu" <scribu@...>
To: wp-hackers@...
Sent: Monday, June 22, 2009 9:24:29 AM GMT -05:00 US/Canada Eastern
Subject: Re: [wp-hackers] Link metadata?

Submitted a ticket on the topic: http://core.trac.wordpress.org/ticket/10233

--
http://scribu.net
_______________________________________________
wp-hackers mailing list
wp-hackers@...
http://lists.automattic.com/mailman/listinfo/wp-hackers
_______________________________________________
wp-hackers mailing list
wp-hackers@...
http://lists.automattic.com/mailman/listinfo/wp-hackers

[scribu]

>
> So what the heck is the resolution?  One table, multiple tables, overload
> taxonomy, overload wp_options, do without?!?  Of course you may be as
> confused as me.
>
> -Mike Schinkel
>

There doens't seem to be any agreement on the matter yet.

--
http://scribu.net
_______________________________________________
wp-hackers mailing list
wp-hackers@...
http://lists.automattic.com/mailman/listinfo/wp-hackers

[Mike Schinkel-5]

On Jun 23, 2009, at 7:57 AM, scribu wrote:
> There doens't seem to be any agreement on the matter yet.

Next step?  Discuss it here?  There?  On which ticket?
Thanks in advance.

-Mike Schinkel
WordPress Custom Plugins
http://mikeschinkel.com/custom-wordpress-plugins/

_______________________________________________
wp-hackers mailing list
wp-hackers@...
http://lists.automattic.com/mailman/listinfo/wp-hackers

[scribu]

The main ticket seems to be http://core.trac.wordpress.org/ticket/5183

I think it's better to discuss it there since it won't get lost as easily as
previous discussions on wp-hackers. Also, it has better formatting.

--
http://scribu.net
_______________________________________________
wp-hackers mailing list
wp-hackers@...
http://lists.automattic.com/mailman/listinfo/wp-hackers

[Mike Schinkel-5]

The discussion on trac said a single meta table would be too abstract  
and create indexes that were too large and thus too slow.
So I suggested keeping wp_postmeta and wp_usermeta and adding a more  
generic table called wp_othermeta for comments, links, or whatever.

Discussion here:  http://core.trac.wordpress.org/ticket/5183

-Mike Schinkel
WordPress Custom Plugins
http://mikeschinkel.com/custom-wordpress-plugins/



On Jun 23, 2009, at 12:43 PM, scribu wrote:

> The main ticket seems to be http://core.trac.wordpress.org/ticket/5183
>
> I think it's better to discuss it there since it won't get lost as  
> easily as
> previous discussions on wp-hackers. Also, it has better formatting.
>
> --
> http://scribu.net
> _______________________________________________
> wp-hackers mailing list
> wp-hackers@...
> http://lists.automattic.com/mailman/listinfo/wp-hackers

_______________________________________________
wp-hackers mailing list
wp-hackers@...
http://lists.automattic.com/mailman/listinfo/wp-hackers