an alternate xspf over json implementation

[Sebastian Pipping-4]

I just opened a wiki page as Ivo suggested.
It currently holds an unfinished JSPF example file.
My idea is we finish the example and then
derive the spec from it. This way we all know
what it looks like before we go to the dirty
spec work.

http://wiki.xiph.org/index.php/JSPF_Draft



Sebastian

_______________________________________________
Playlist mailing list
Playlist@...
http://lists.musicbrainz.org/mailman/listinfo/playlist

[Chris Anderson-4]

On 6/13/07, Sebastian Pipping <webmaster@...> wrote:

> One more thing: I've seen key names with quotes
> and without quotes: is both legal?
> What I mean is
>
>   {
>     "playlist" : { .. }
>   }
>
> versus
>
>   {
>     playlist : { .. }
>   }
>

According to [1] keys and strings must be double-quoted.

Grabb.it isn't valid 100% of the time by that measure. But we're just
using Rails' default to_json output, and it evals fine. The nice thing
about json is that parsing idiosyncrasies don't usually turn into data
idiosyncrasies. That said, we should probably get valid. :)

[1] http://simonwillison.net/2006/Oct/11/json/

--
Chris Anderson
http://jchris.mfdz.com

_______________________________________________
Playlist mailing list
Playlist@...
http://lists.musicbrainz.org/mailman/listinfo/playlist

[Chris Anderson-4]

On 6/13/07, Sebastian Pipping <webmaster@...> wrote:
> My idea is we finish the example and then
> derive the spec from it. This way we all know
> what it looks like before we go to the dirty
> spec work.

I had floated the idea of creating a reference parser in Javascript
that turns valid XSPF into valid JSPF. We can still go that route, as
well. I've been having a little trouble getting my existing parser to
work on IE 6, so I haven't checked it into the repository yet.

I'm not sure how one would spec a JSON format. I'm sure it can be done
- but some of the key differences from XML are that a JSON format
ought to allow arbitrary keys to be added to the document, as long as
the required keys are present and any keys that are part of the spec
are valid.



--
Chris Anderson
http://jchris.mfdz.com

_______________________________________________
Playlist mailing list
Playlist@...
http://lists.musicbrainz.org/mailman/listinfo/playlist

[Chris Anderson-4]

On 6/13/07, Sebastian Pipping <webmaster@...> wrote:
> I just opened a wiki page as Ivo suggested.
> It currently holds an unfinished JSPF example file.
>
> http://wiki.xiph.org/index.php/JSPF_Draft
>

I updated the page to handle extension elements. I was surprised to
see how easy it was to handle multiple extensions from the same
application. One interesting deviation from the XML, and worth
discussing: Rather than have the extensions contain a JSON conversion
of arbitrary XML - which is the stuff that can get complicated quick -
I've got them containing arbitrary JSON.

What this means is that it will be up to each extension developer to
decide how they want to express their extensions in JSON. Should they
choose a verbose raw transform from XML, they are able to do that, but
should they choose a tighter format more native to JSON, we'll support
that as well.

To me this seems to best parallel the intentions and data model of
XSPF's extension element.

Thoughts?


--
Chris Anderson
http://jchris.mfdz.com

_______________________________________________
Playlist mailing list
Playlist@...
http://lists.musicbrainz.org/mailman/listinfo/playlist

[Sebastian Pipping-4]

Chris Anderson wrote:

> On 6/13/07, Sebastian Pipping <webmaster@...> wrote:
>> My idea is we finish the example and then
>> derive the spec from it. This way we all know
>> what it looks like before we go to the dirty
>> spec work.
>
> I had floated the idea of creating a reference parser in Javascript
> that turns valid XSPF into valid JSPF. We can still go that route, as
> well. I've been having a little trouble getting my existing parser to
> work on IE 6, so I haven't checked it into the repository yet.

---------------------------------------------
A reference converter would be nice.
I think we should finish the format
before coding though so no code has to
b thrown away and written again.
---------------------------------------------



> I'm not sure how one would spec a JSON format. I'm sure it can be done
> - but some of the key differences from XML are that a JSON format
> ought to allow arbitrary keys to be added to the document, as long as
> the required keys are present and any keys that are part of the spec
> are valid.

---------------------------------------------
This conflicts to the extension concept
of XSPF. While I'm not sure which concept
is better, I think we should try to
make the semantics of JSPF as close to
that of XSPF to be able to convert from
JSPF to XSPF without problems.



Sebastian

_______________________________________________
Playlist mailing list
Playlist@...
http://lists.musicbrainz.org/mailman/listinfo/playlist

[Sebastian Pipping-4]

Chris Anderson wrote:
> I updated the page to handle extension elements. I was surprised to
> see how easy it was to handle multiple extensions from the same
> application.

--------------------------------------------------------------
With mixed extension types we lose the extension order this
way, right? Is that order important to preserve?
--------------------------------------------------------------

> One interesting deviation from the XML, and worth
> discussing: Rather than have the extensions contain a JSON conversion
> of arbitrary XML - which is the stuff that can get complicated quick -
> I've got them containing arbitrary JSON.
>
> What this means is that it will be up to each extension developer to
> decide how they want to express their extensions in JSON. Should they
> choose a verbose raw transform from XML, they are able to do that, but
> should they choose a tighter format more native to JSON, we'll support
> that as well.

--------------------------------------------------------------
You mean an auto conversion for an extension from XML to JSON
will work but maybe look bad but extension authors can
define a better JSON edition as well?



Sebastian

_______________________________________________
Playlist mailing list
Playlist@...
http://lists.musicbrainz.org/mailman/listinfo/playlist

[Chris Anderson-4]

On 6/13/07, Sebastian Pipping <webmaster@...> wrote:

> Chris Anderson wrote:
> > I'm not sure how one would spec a JSON format. I'm sure it can be done
> > - but some of the key differences from XML are that a JSON format
> > ought to allow arbitrary keys to be added to the document, as long as
> > the required keys are present and any keys that are part of the spec
> > are valid.
>
> ---------------------------------------------
> This conflicts to the extension concept
> of XSPF. While I'm not sure which concept
> is better, I think we should try to
> make the semantics of JSPF as close to
> that of XSPF to be able to convert from
> JSPF to XSPF without problems.
>

I understand what you are getting at here. From [1]: "The JSON
approach of just ignoring keys you don't understand can get you a long
way, but I don't think it scales."

Probably the best course of action is for the JSPF spec to remain
silent on the issue. Conversion from the Javascript object to an XSPF
document will by its nature ignore keys that aren't needed, so the
only need for the spec to disallow extra keys would be to ease
manipulations of the JSPF that don't first read it into a data
structure. Stream parsing JSPF directly, rather than explictly as JSON
first, seems an edge case to me.

The standard JSON behavior is to parse the whole document into a data
structure. Code which is expecting a JSPF structure won't be bothered
by extra keys, so it's convention in JSON to allow those keys.

[1] http://blog.jclark.com/2007/04/xml-and-json.html

--
Chris Anderson
http://jchris.mfdz.com

_______________________________________________
Playlist mailing list
Playlist@...
http://lists.musicbrainz.org/mailman/listinfo/playlist

[Chris Anderson-4]

On 6/13/07, Sebastian Pipping <webmaster@...> wrote:
> Chris Anderson wrote:
> > I updated the page to handle extension elements. I was surprised to
> > see how easy it was to handle multiple extensions from the same
> > application.
>
> --------------------------------------------------------------
> With mixed extension types we lose the extension order this
> way, right? Is that order important to preserve?
> --------------------------------------------------------------

I'm not sure I follow you here... I was imagining that if the
extension for say "http://grabb.it" showed up twice in a track, it
would show up as two elements of the "http://grabb.it" extension
array. The order could be preserved, but I don't think there's any
reason to need it to be. As far as the insides of the extensions go -
that's up to individual applications.


> --------------------------------------------------------------
> You mean an auto conversion for an extension from XML to JSON
> will work but maybe look bad but extension authors can
> define a better JSON edition as well?

Exactly. The nice thing is that simple key-value pair extensions
convert cleanly and easily, so applications that limit themselves to
that vocabulary won't have to do any extra work. Anything more complex
will require something like the conversion algorithm described in the
XML.com article.

--
Chris Anderson
http://jchris.mfdz.com

_______________________________________________
Playlist mailing list
Playlist@...
http://lists.musicbrainz.org/mailman/listinfo/playlist

[Sebastian Pipping-4]

Chris Anderson wrote:
> Probably the best course of action is for the JSPF spec to remain
> silent on the issue.

----------------------------------------------------------
I'm not sure if silence about it is a good idea
in a specification. Maybe we just need more input.



Sebastian

_______________________________________________
Playlist mailing list
Playlist@...
http://lists.musicbrainz.org/mailman/listinfo/playlist

[Sebastian Pipping-4]

Chris Anderson wrote:
>> With mixed extension types we lose the extension order this
>> way, right? Is that order important to preserve?
>
> I'm not sure I follow you here... I was imagining that if the
> extension for say "http://grabb.it" showed up twice in a track, it
> would show up as two elements of the "http://grabb.it" extension
> array. The order could be preserved, but I don't think there's any
> reason to need it to be. As far as the insides of the extensions go -
> that's up to individual applications.

--------------------------------------------------------------
I thought about this again. I modified your extension
proposal to actually see the difference:


== ORIGINAL ==
   "extension"     : {
     "http://example.application.com":[
       {"multiple":"instances of"},
       {"an application's":"extension can coexist"}
     ],
     "http://another.application.com":[{"each instance":"arbitrary json"}]
   }

== MOD ==
   "extension"     : [
     { "http://example.com/app/1/" : ARBITRARY_EXTENSION_BODY },
     { "http://example.com/app/2/" : ARBITRARY_EXTENSION_BODY }
   ]


I like the latter much better because
* It fully preserves the order (ABABA will not become AAABB)
* It is simpler, has one level nesting less (in a way "list versus tree")

I finishes the draft in the wiki with that new extension model.
If I didn't overlook anything the at-this-moment JSPF proposal can be
converted back and forth losslessly. In my eyes it also is pretty
readable. What do you think?


Before I forget: If you look at the track locations you find this:

   "location"      : [
     {"location"     : "http://example.com/1/"},
     {"location"     : "http://example.com/2/"}
   ],

It might look a little starnge first, but...
I did it to be consistent to playlist.attribution:

   "attribution"   : [
     {"identifier"   : "http://example.com/"},
     {"location"     : "http://example.com/"}
   ],

We need a key name for it anyway. Also I decided not to append
"s"s to the "container elements" to keep the names 1:1 XSPF.
Additionally we have singular everywhere then. Simplicity,
consistency, readability. Well, in my eyes. I hope you like it.



Sebastian

_______________________________________________
Playlist mailing list
Playlist@...
http://lists.musicbrainz.org/mailman/listinfo/playlist

[Chris Anderson-4]

On 6/14/07, Sebastian Pipping <webmaster@...> wrote:

> Chris Anderson wrote:
> --------------------------------------------------------------
> I thought about this again. I modified your extension
> proposal to actually see the difference:
>
>
> == ORIGINAL ==
>    "extension"     : {
>      "http://example.application.com":[
>        {"multiple":"instances of"},
>        {"an application's":"extension can coexist"}
>      ],
>      "http://another.application.com":[{"each instance":"arbitrary json"}]
>    }
>
> == MOD ==
>    "extension"     : [
>      { "http://example.com/app/1/" : ARBITRARY_EXTENSION_BODY },
>      { "http://example.com/app/2/" : ARBITRARY_EXTENSION_BODY }
>    ]
>
>
> I like the latter much better because
> * It fully preserves the order (ABABA will not become AAABB)
> * It is simpler, has one level nesting less (in a way "list versus tree")

I'm torn here. I see the benefit of your approach, as far as encoding
data goes. The trouble spot is in retrieving the data. Rather than
accessing an application's extensions directly in code, one has to
search through the array of extensions matches. The difference is
significant.

the old way:

my_jspf.playlist.track[0].extension["http://example.com/app/1/"][0];

the new way requires a function to find the first matching extension -
even more work if you want to find more than one extension body for
the same application.

for (var i=0; i < my_jspf.playlist.track[0].extension.length; i++) {
  var extension = my_jspf.playlist.track[0].extension[i];
  if (typeof extension["http://example.com/app/1/"]  != "undefined") {
    return extension["http://example.com/app/1/"];
  }
}

Correct me if I'm wrong, but extension order does not matter in XSPF,
so perhaps the usability gains here outweigh the costs of losing XML
ordering.

>
> Before I forget: If you look at the track locations you find this:
>
>    "location"      : [
>      {"location"     : "http://example.com/1/"},
>      {"location"     : "http://example.com/2/"}
>    ],
>

I'm not sure I understand the benefit here; the Rubyist in me just
wants to remove the duplication of the extra "location" key. I updated
the wiki thusly:

         "location"      : ["http://example.com/1/", "http://example.com/2/"],
         "identifier"    : ["http://example.com/1/", "http://example.com/2/"],

As well as requiring less bytes over the wire, the compact syntax also
means less typing for people using the JSPF object. To access the
first location of a track using the compact syntax, one codes:

my_jspf.playlist.track[0].location[0]

using the previous syntax, it's a little longer:

my_jspf.playlist.track[0].location[0].location

The only benefit I can see to the longer syntax is that one could hang
additional properties (above and beyond the spec) on the location
objects - maybe an application would want to provide average transfer
rates for a given location, or last_seen_at times for the urls - but
I'm not sure we want to encourage extra-spec properties nested so
deeply. eg:

my_jspf.playlist.track[0].location[0].last_seen_at   #=>
"2007-05-15T13:08:37-08:00"

Although I can see the appeal. :)

I really like the way you've handled the rel attributes for link and
meta elements. I was stumped for a method to manage that simply, but I
think you've nailed it here:

        "link"          : [
          {"http://example.com/rel/1/" : "http://example.com/body/1/"},
          {"http://example.com/rel/2/" : "http://example.com/body/2/"}
        ],
        "meta"          : [
          {"http://example.com/rel/1/" : "my meta 14"},
          {"http://example.com/rel/2/" : "345"}
        ],

I'm eager for everyone's feedback.

--
Chris Anderson
http://jchris.mfdz.com

_______________________________________________
Playlist mailing list
Playlist@...
http://lists.musicbrainz.org/mailman/listinfo/playlist

[Sebastian Pipping-4]

Chris Anderson wrote:

> I'm torn here. I see the benefit of your approach, as far as encoding
> data goes. The trouble spot is in retrieving the data. Rather than
> accessing an application's extensions directly in code, one has to
> search through the array of extensions matches. The difference is
> significant.
>
> the old way:
>
> my_jspf.playlist.track[0].extension["http://example.com/app/1/"][0];
>
> the new way requires a function to find the first matching extension -
> even more work if you want to find more than one extension body for
> the same application.
>
> for (var i=0; i < my_jspf.playlist.track[0].extension.length; i++) {
>  var extension = my_jspf.playlist.track[0].extension[i];
>  if (typeof extension["http://example.com/app/1/"]  != "undefined") {
>    return extension["http://example.com/app/1/"];
>  }
> }

-------------------------------------------------------------
I didn't think of that. I agree the first way is much easier.
-------------------------------------------------------------



> Correct me if I'm wrong, but extension order does not matter in XSPF,
> so perhaps the usability gains here outweigh the costs of losing XML
> ordering.

-------------------------------------------------------------
I'm not sure if extension order really is insignificant.
The order of locations for example is significant and
I don't see why it should be different with extensions.
Lucas, does it matter? Is there a third way?
-------------------------------------------------------------

>> Before I forget: If you look at the track locations you find this:
>>
>>    "location"      : [
>>      {"location"     : "http://example.com/1/"},
>>      {"location"     : "http://example.com/2/"}
>>    ],
>>
>
> I'm not sure I understand the benefit here; the Rubyist in me just
> wants to remove the duplication of the extra "location" key. I updated
> the wiki thusly:
>
>         "location"      : ["http://example.com/1/",
> "http://example.com/2/"],
>         "identifier"    : ["http://example.com/1/",
> "http://example.com/2/"],

-------------------------------------------------------------
I didn't know this was legal JSON. No need to duplicate
the keys then. Cool!
-------------------------------------------------------------

> The only benefit I can see to the longer syntax is that one could hang
> additional properties (above and beyond the spec) on the location
> objects - maybe an application would want to provide average transfer
> rates for a given location, or last_seen_at times for the urls - but
> I'm not sure we want to encourage extra-spec properties nested so
> deeply. eg:
>
> my_jspf.playlist.track[0].location[0].last_seen_at   #=>
> "2007-05-15T13:08:37-08:00"
>
> Although I can see the appeal. :)

-------------------------------------------------------------
I think we should not introduce additional features
for JSPF. I think we should keep XSPF and JSON in sync.
If we really need more features we should make a successor
version of both JSON and XSPF in my eyes.



Sebastian


_______________________________________________
Playlist mailing list
Playlist@...
http://lists.musicbrainz.org/mailman/listinfo/playlist

[Lucas Gonze-2]

> Chris Anderson wrote:
>> Correct me if I'm wrong, but extension order does not matter in XSPF,
>> so perhaps the usability gains here outweigh the costs of losing XML
>> ordering.

Sebastian Pipping wrote:
> I'm not sure if extension order really is insignificant.
> The order of locations for example is significant and
> I don't see why it should be different with extensions.
> Lucas, does it matter? Is there a third way?

I could imagine that the rel and meta elements would need to preserve
order, but because the extension element contains a complex structure
with multiple nodes inside I don't think that the order of extensions
should matter.

BTW, I notice a pattern with how much of our time we spend on the
extension elements, which are not central to XSPF' mission, and which
are relatively little used.  The attribution element matters a lot
(because it enables sharing licenses which require attribution), but it
gets much less time.  These extension elements are consistently on the
wrong side of the 80/20 rule.

Sebastian Pipping wrote:
> I think we should not introduce additional features
> for JSPF. I think we should keep XSPF and JSON in sync.
> If we really need more features we should make a successor
> version of both JSON and XSPF in my eyes.

I strongly agree.  There is no bigger mistake.



_______________________________________________
Playlist mailing list
Playlist@...
http://lists.musicbrainz.org/mailman/listinfo/playlist

[Chris Anderson-4]

Unless there's more input from other list members, I think we've got
ourselves a format. I'll start updating my Javascript XSPF parser to
produce this version.

On 6/16/07, Lucas Gonze <lgonze@...> wrote:
> Sebastian Pipping wrote:
> > I think we should not introduce additional features
> > for JSPF. I think we should keep XSPF and JSON in sync.
> > If we really need more features we should make a successor
> > version of both JSON and XSPF in my eyes.
>
> I strongly agree.  There is no bigger mistake.

Here here!

But we've got to remember that hanging additional attributes on the
Javascript object will be the normal course of action when people are
using JSPF in the wild. I'll suggest that an object which has all the
fields in the specification, and is valid on those fields, but which
has some additional keys thrown on, is still valid JSPF. I see this as
a structural hallmark of any Javascript process which would use the
format, as well as defacto fallout from the existing world of JSON
applications.

I realize this practice is not nearly as scalable as XML namespaces -
but it think any kind of validator we create will be better served by
validating on the Javascript object result of parsing a JSPF document,
rather than on the JSPF document-string itself.

(I'm playing a little bit of devil's advocate here - I would avoid
private enrichments of the JSPF schema in my own projects - but I
think the fast and loose world of JSON really is better served by this
sort of permissiveness. Think of it as duck-typed validation.)

In other words, valid JSPF is valid JSON that when parsed yields an
object that responds to the correct set of messages.

eg.

my_jspf.playlist.track MUST be an array with zero or more elements

and

my_jspf.playlist.track[3].meta MAY be null or an array of zero or more elements.

to specify that for instance

my_jspf.playlist.nonstandard_property must be null

I think is an overspecification.

Thoughts?

--
Chris Anderson
http://jchris.mfdz.com

_______________________________________________
Playlist mailing list
Playlist@...
http://lists.musicbrainz.org/mailman/listinfo/playlist

[Sebastian Pipping-4]

Chris Anderson wrote:
> Unless there's more input from other list members, I think we've got
> ourselves a format. I'll start updating my Javascript XSPF parser to
> produce this version.

------------------------------------------------------------
Sounds good. I just had a look at the current state
again and I think we came to a good solution.
Looking forward to JSPF v1 in action!

One more thing: I stumbled over these single lines on
the XSPF.toJSPF() page:

  duration: null
  trackNum: null
  album: null

To me it seemed logical to map '"key" : null' to an empty
tag '<key></key>' but what does '"key" : ""' map to then?
So '"key" : null' is equivalent to omitting that element?



Sebastian


_______________________________________________
Playlist mailing list
Playlist@...
http://lists.musicbrainz.org/mailman/listinfo/playlist

[Chris Anderson-4]

On 6/16/07, Sebastian Pipping <webmaster@...> wrote:
> To me it seemed logical to map '"key" : null' to an empty
> tag '<key></key>' but what does '"key" : ""' map to then?
> So '"key" : null' is equivalent to omitting that element?

I think the null values you see there are just artifacts of the
parsing process. I'm not sure what the behaviour ought to be. It might
be more user-friendly to provide null values rather than omitting the
element, but it should be an easy tweak to make either way once we
have a working parser.

--
Chris Anderson
http://jchris.mfdz.com

_______________________________________________
Playlist mailing list
Playlist@...
http://lists.musicbrainz.org/mailman/listinfo/playlist

[Chris Anderson-4]

> On 5/21/07, Sebastian Pipping <webmaster@...> wrote:
> > Lucas Gonze wrote:
> > > First shot at an extension for licenses in tracks:
> > >
> > > http://gonze.com/xspf/rellicense/
> > -------------------------------------------------------------
> > Had a first look - nice!
> >
> > How about one of these for the URI?:
> >  * http://xspf.org/track/license/
> >  * http://xspf.org/track/link/license/1/0/
> >

I'm finally writing the code to put the cc-license data Grabb.it
aggregates into our xspf/jspf. I'm planning to use this proposal, but
we should pick a URI for the rel. I'm partial to the versioned one,
for what its worth.

Chris



--
Chris Anderson
http://jchris.mfdz.com

_______________________________________________
Playlist mailing list
Playlist@...
http://lists.musicbrainz.org/mailman/listinfo/playlist

[Saoshyant]

On 11/13/07, Chris Anderson <jchris@...> wrote:
> I'm planning to use this proposal, but
> we should pick a URI for the rel. I'm partial to the versioned one,
> for what its worth.

Like this:

http://creativecommons.org/licenses/by/3.0/

you mean?  That would be the right way to use this extension, in my opinion.

-Ivo

_______________________________________________
Playlist mailing list
Playlist@...
http://lists.musicbrainz.org/mailman/listinfo/playlist

[Chris Anderson-4]

On Nov 13, 2007 6:08 AM, Ivo Emanuel Gonçalves <justivo@...> wrote:
>
> Like this:
>
> http://creativecommons.org/licenses/by/3.0/
>
> you mean?  That would be the right way to use this extension, in my opinion.
>
> -Ivo

Yes, I agree about that - I presume you are talking about the text
content of the link element. The original thread was about what the
<link rel="uri"> uri ought to be, to signify that the link itself
specifies a license.



--
Chris Anderson
http://jchris.mfdz.com

_______________________________________________
Playlist mailing list
Playlist@...
http://lists.musicbrainz.org/mailman/listinfo/playlist

[Agentbleu]

I am developing a new feature for my xspf player and want to pool the  
knowledge here.

What I need is a way to find out the most popular searched for bands  
in a given period. say last 3 months, etc.

are there any mp3 search engines here listening ?

suggestions anyone?




_______________________________________________
Playlist mailing list
Playlist@...
http://lists.musicbrainz.org/mailman/listinfo/playlist