Re: "scheme" attribute of META element

[Tom Dent]

Ian Hickson wrote:
>> I can't speak for Elliot, but the Web repository connector inside SAP
>> Netweaver's Knowledge Management has supported RFC2731-style encoded
>> metadata (as shown above) for many years now.
>
> Could you elaborate on how this tool consumes this data? Any information

> you may have would be very useful. Could you walk us through an example
of
> how this information gets used? How do the various schemes affect the
> handling of the metadata? Have you found particular processing is needed

> to process invalid values? Is the tool's input limited to files
generated
> by one organisation, or does it process input from arbitrary Web sites?

The 'scheme' attribute is also used by UK Government websites. The
e-Government Metadata Standard gives elements and refinements to be used
in metadata for information resources and is available here:
http://www.govtalk.gov.uk/schemasstandards/metadata_document.asp?docnum=10
17

This gives several examples where the 'scheme' element is mentioned, and
uses different schemes, such as Dublin Core (on page 19 of the document):
'<meta name="DCTERMS.temporal" scheme="DCTERMS.W3CDTF"
content="2006-04-20"/>' where the date will follow the guidelines here:
http://www.w3.org/TR/NOTE-datetime or (on page 34): '<meta
name="DC.identifier" scheme="DCTERMS.URI"
content="http://purl.oclc.org/NET/e-GMS_v1"/>' where the URI guidelines
should be used: http://purl.org/dc/terms/URI

As well as using Dublin Core, other schemes are used, such as IPSV
(Integrated Public Sector Vocabulary): '<meta name="DC.subject"
scheme="eGMS.IPSV" content="Youth centres"/>'. IPSV is a controlled list
that was developed with the backing of the Department for Communities and
Local Government (CLG - formerly the Office of the Deputy Prime Minister)
and the e-Government Unit (e-GU) of the Cabinet Office for use by UK
public sector organisations and is available in machine-readable format
from here: http://www.esd.org.uk/standards/ipsv/

The IPSV format referenced in the e-GMS guidance is used by many sites,
such as
http://www.poole.gov.uk/children_integrated/services/ref:S4649B18BD5605/ak
a:Bourne+Valley+Youth+Centre/ and the metadata can be used to index the
resources, such as on this page:
http://www.esd.org.uk/standards/ipsv/viewer/TermUsageList.aspx?term=Youth%
20centres&id=4665&treeid=AlternateTree&referrer=main.aspx

Due to the fact that IPSV uses a controlled set of terms, these pages can
be referenced in a way which is relatively easy for a machine to
reference, as the scheme gives the name of the list to be used and the
content gives the name of the item, making it a more reliable way of
sourcing the exact content of the page and matching it to others that are
used. The extent of its use means that Sitemorse, a company who check page
validity of UK sites in several areas, use this metadata to check content:
http://www.sitemorse.com/kb.html?kb=1266176694 and it is a feature built
into a CMS used by UK sites:
http://www.jadu.co.uk/info/20029/government/45/integrated_public_sector_vo
cabulary_ipsv/1

Tom Dent
Porism Limited
tom.dent@...

[Ian Hickson]

On Wed, 29 Jul 2009, Tom Dent wrote:
>
> As well as using Dublin Core, other schemes are used, such as IPSV
> (Integrated Public Sector Vocabulary): '<meta name="DC.subject"
> scheme="eGMS.IPSV" content="Youth centres"/>'. IPSV is a controlled list
> that was developed with the backing of the Department for Communities and
> Local Government (CLG - formerly the Office of the Deputy Prime Minister)
> and the e-Government Unit (e-GU) of the Cabinet Office for use by UK
> public sector organisations and is available in machine-readable format
> from here: http://www.esd.org.uk/standards/ipsv/

What other scemes are used with this name?

What happens when the value used doesn't come from the allowed list? Or
when the scheme doesn't match "eGMS.IPSV" but the value is still one of
the allowed values?

> Due to the fact that IPSV uses a controlled set of terms, these pages can
> be referenced in a way which is relatively easy for a machine to
> reference, as the scheme gives the name of the list to be used and the
> content gives the name of the item, making it a more reliable way of
> sourcing the exact content of the page and matching it to others that are
> used. The extent of its use means that Sitemorse, a company who check page
> validity of UK sites in several areas, use this metadata to check content:
> http://www.sitemorse.com/kb.html?kb=1266176694 and it is a feature built
> into a CMS used by UK sites:
> http://www.jadu.co.uk/info/20029/government/45/integrated_public_sector_vocabulary_ipsv/1

Could you show an example of how the scheme attribute affects the user
experience? I looked around but couldn't find any pages that actually
expose any of this data, so it wasn't clear to me whether any code other
than the validators and CMSes actually used it.

--
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

[Tom Dent]

On Wed, Jul 29, 2009 at 7:27 PM, Ian Hickson <ian@...> wrote:

>
> On Wed, 29 Jul 2009, Tom Dent wrote:
> >
> > As well as using Dublin Core, other schemes are used, such as IPSV
> > (Integrated Public Sector Vocabulary): '<meta name="DC.subject"
> > scheme="eGMS.IPSV" content="Youth centres"/>'. IPSV is a controlled list
> > that was developed with the backing of the Department for Communities and
> > Local Government (CLG - formerly the Office of the Deputy Prime Minister)
> > and the e-Government Unit (e-GU) of the Cabinet Office for use by UK
> > public sector organisations and is available in machine-readable format
> > from here: http://www.esd.org.uk/standards/ipsv/
>
> What other scemes are used with this name?

I'm not sure what you mean, but any controlled list (eGMS also
references the Local Government Service List (available at
http://www.esd.org.uk/standards/lgsl)) or data format pattern might
constitute a scheme. The Dublin Core metadata, which is also
referenced, is designed to be universal, as are the schemes which it
employs (such as date and URI), and references other more universal
schemes such as ISO 3166 http://www.iso.org/iso/country_codes.htm

> What happens when the value used doesn't come from the allowed list? Or
> when the scheme doesn't match "eGMS.IPSV" but the value is still one of
> the allowed values?
If a value does not reflect a concept id or preferred label for a
concept in a controlled list, the page reference is ignored as
invalid.  If the value is an allowed value but the scheme is not
specified, we assume that the value does not necessarily mean the same
as we mean within the scheme.  Hence again we ignore the reference.
So we could think of the scheme as a namespace for the value.

> > Due to the fact that IPSV uses a controlled set of terms, these pages can
> > be referenced in a way which is relatively easy for a machine to
> > reference, as the scheme gives the name of the list to be used and the
> > content gives the name of the item, making it a more reliable way of
> > sourcing the exact content of the page and matching it to others that are
> > used. The extent of its use means that Sitemorse, a company who check page
> > validity of UK sites in several areas, use this metadata to check content:
> > http://www.sitemorse.com/kb.html?kb=1266176694 and it is a feature built
> > into a CMS used by UK sites:
> > http://www.jadu.co.uk/info/20029/government/45/integrated_public_sector_vocabulary_ipsv/1
>
> Could you show an example of how the scheme attribute affects the user
> experience? I looked around but couldn't find any pages that actually
> expose any of this data, so it wasn't clear to me whether any code other
> than the validators and CMSes actually used it.

The scheme does not directly impact on the user, but can (for example)
do so via search software.  You can see a view of IPSV at
http://www.esd.org.uk/standards/ipsv/viewer/ or LGSL at
http://www.esd.org.uk/standards/lgsl/viewer .

If a user searched for pages by using a "non-preferred term", the
search engine should convert that to an IPSV preferred term and then
return all content with that preferred term in its metadata (with the
IPSV scheme).  The search software might also suggest showing content
for broader and narrower subject headings from the IPSV hierarchy.

The LGSL is being used to reference local authority sites in the UK
Government's Directgov site (at http://www.direct.gov.uk/) which is
used as a portal to local authority sites, and uses LGSL to reference
pages.

Tom Dent
Porism Limited
tom.dent@...

[Ian Hickson]

On Tue, 4 Aug 2009, Tom Dent wrote:

> On Wed, Jul 29, 2009 at 7:27 PM, Ian Hickson <ian@...> wrote:
> >
> > Could you show an example of how the scheme attribute affects the user
> > experience? I looked around but couldn't find any pages that actually
> > expose any of this data, so it wasn't clear to me whether any code other
> > than the validators and CMSes actually used it.
>
> The scheme does not directly impact on the user, but can (for example)
> do so via search software.  You can see a view of IPSV at
> http://www.esd.org.uk/standards/ipsv/viewer/ or LGSL at
> http://www.esd.org.uk/standards/lgsl/viewer .
>
> If a user searched for pages by using a "non-preferred term", the search
> engine should convert that to an IPSV preferred term and then return all
> content with that preferred term in its metadata (with the IPSV scheme).  
> The search software might also suggest showing content for broader and
> narrower subject headings from the IPSV hierarchy.
>
> The LGSL is being used to reference local authority sites in the UK
> Government's Directgov site (at http://www.direct.gov.uk/) which is used
> as a portal to local authority sites, and uses LGSL to reference pages.

This doesn't really seem like a very compelling use case. Surely more
modern search technologies would be a significantly more effective way of
addressing the same problem at the user level, but with a significantly
better user experience? Can you walk me through what a typical user might
search for in a way that metadata with a scheme would affect the result?
I'm having trouble figuring out what a suitable "non-preferred term" might
be. For example, I did a search on direct.gov.uk for "how do i report
foreign income for tax purposes", which gave me the same top result as
Google did for that same search (with a site restrict), and did not appear
to be in any way affected by metadata terms that used schemes.

--
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

[Tom Dent]

On Thu, Aug 13, 2009 at 12:19 PM, Ian Hickson<ian@...> wrote:

> On Tue, 4 Aug 2009, Tom Dent wrote:
>> On Wed, Jul 29, 2009 at 7:27 PM, Ian Hickson <ian@...> wrote:
>> >
>> > Could you show an example of how the scheme attribute affects the user
>> > experience? I looked around but couldn't find any pages that actually
>> > expose any of this data, so it wasn't clear to me whether any code other
>> > than the validators and CMSes actually used it.
>>
>> The scheme does not directly impact on the user, but can (for example)
>> do so via search software.  You can see a view of IPSV at
>> http://www.esd.org.uk/standards/ipsv/viewer/ or LGSL at
>> http://www.esd.org.uk/standards/lgsl/viewer .
>>
>> If a user searched for pages by using a "non-preferred term", the search
>> engine should convert that to an IPSV preferred term and then return all
>> content with that preferred term in its metadata (with the IPSV scheme).
>> The search software might also suggest showing content for broader and
>> narrower subject headings from the IPSV hierarchy.
>>
>> The LGSL is being used to reference local authority sites in the UK
>> Government's Directgov site (at http://www.direct.gov.uk/) which is used
>> as a portal to local authority sites, and uses LGSL to reference pages.
>
> This doesn't really seem like a very compelling use case. Surely more
> modern search technologies would be a significantly more effective way of
> addressing the same problem at the user level, but with a significantly
> better user experience? Can you walk me through what a typical user might
> search for in a way that metadata with a scheme would affect the result?
> I'm having trouble figuring out what a suitable "non-preferred term" might
> be. For example, I did a search on direct.gov.uk for "how do i report
> foreign income for tax purposes", which gave me the same top result as
> Google did for that same search (with a site restrict), and did not appear
> to be in any way affected by metadata terms that used schemes.

Using the LGSL metadata can make the seach at the Directgov site more
powerful than Google. For example, this page about abandoned vehicles
references LGSL:
http://www.chiltern.gov.uk/site/scripts/services_info.php?serviceID=306&startsWith=A
<meta name="eGMS.subject.service" lang="en" scheme="LGSL"
content="Abandoned vehicles" />
Doing a Google search for that abandoned vehicles and a postcode in
that authority does not return the correct page:
http://www.google.co.uk/search?hl=de&q=HP5+1RS+abandoned+vehicles
However entering your postcode into the Directgov site (which also
references the LGSL in the URL) does:
 http://local.direct.gov.uk/LDGRedirect/LocationSearch.do?searchtype=1&LGSL=372&LGIL=0&Style=&formsub=t&text=HP5+1RS
This is particulaly import for people who live in areas which are
covered by more than one authority, as they may not know which of the
authorities supplies the service they need. Searching the site for
this text returns many results - using the LGSL allows authorities to
mark which page most appropriately represents this:
http://www.google.co.uk/search?q=+site:www.chiltern.gov.uk+abandoned+vehicles
As Google does not search this metadata, it is unable to determine
which page best represents this service.

As an example of non-preferred terms, IPSV 219 is "Taxis", but has
non-preferred terms (terms which apply to the same concept) of
"Minicabs", "London cabs", "Cabs" and "Taxicabs". All the terms apply
to the same concept but may be used in different situations. However
to give a common vacabulary to know that the same concept is being
referenced, the concept is given a preferred term.

[Mike Thacker-2]

As I understand it, this thread is concerned with the usefulness of keeping the "scheme" attribute of the META element.

The discussion comes down to whether or not schemes defining the format of values or, as described here, controlled lists adding definition and precise meaning to values (in the "content" attribute) are useful.

To me adding defining the controlled list from which a precise value is taken has tremendous value, as described by Tom.

Ian you say:

>>Surely more modern search technologies would be a significantly more effective way of addressing the same problem at the user level, but with a significantly
better user experience?<<

Can you say (or provide a link to indicate) what these modern search technologies might be?

Thanks
Mike

[Ian Hickson]

On Wed, 19 Aug 2009, Tom Dent wrote:

> On Thu, Aug 13, 2009 at 12:19 PM, Ian Hickson<ian@...> wrote:
> > On Tue, 4 Aug 2009, Tom Dent wrote:
> >> On Wed, Jul 29, 2009 at 7:27 PM, Ian Hickson <ian@...> wrote:
> >>
> >> The scheme does not directly impact on the user, but can (for
> >> example) do so via search software. �You can see a view of IPSV at
> >> http://www.esd.org.uk/standards/ipsv/viewer/ or LGSL at
> >> http://www.esd.org.uk/standards/lgsl/viewer .
> >>
> >> If a user searched for pages by using a "non-preferred term", the
> >> search engine should convert that to an IPSV preferred term and then
> >> return all content with that preferred term in its metadata (with the
> >> IPSV scheme). The search software might also suggest showing content
> >> for broader and narrower subject headings from the IPSV hierarchy.
> >>
> >> The LGSL is being used to reference local authority sites in the UK
> >> Government's Directgov site (at http://www.direct.gov.uk/) which is
> >> used as a portal to local authority sites, and uses LGSL to reference
> >> pages.
> >
> > This doesn't really seem like a very compelling use case. Surely more
> > modern search technologies would be a significantly more effective way
> > of addressing the same problem at the user level, but with a
> > significantly better user experience? Can you walk me through what a
> > typical user might search for in a way that metadata with a scheme
> > would affect the result? I'm having trouble figuring out what a
> > suitable "non-preferred term" might be. For example, I did a search on
> > direct.gov.uk for "how do i report foreign income for tax purposes",
> > which gave me the same top result as Google did for that same search
> > (with a site restrict), and did not appear to be in any way affected
> > by metadata terms that used schemes.
>
> Using the LGSL metadata can make the seach at the Directgov site more
> powerful than Google. For example, this page about abandoned vehicles
> references LGSL:
>
> http://www.chiltern.gov.uk/site/scripts/services_info.php?serviceID=306&startsWith=A
>
> <meta name="eGMS.subject.service" lang="en" scheme="LGSL"
> content="Abandoned vehicles" />
>
> Doing a Google search for that abandoned vehicles and a postcode in that
> authority does not return the correct page:
> http://www.google.co.uk/search?hl=de&q=HP5+1RS+abandoned+vehicles 
> However entering your postcode into the Directgov site (which also
> references the LGSL in the URL) does:
>
>  http://local.direct.gov.uk/LDGRedirect/LocationSearch.do?searchtype=1&LGSL=372&LGIL=0&Style=&formsub=t&text=HP5+1RS

You don't need the "scheme" attribute to do that, though. This is
basically just synonym matching; you could apply the same synonym matching
to all terms on the page, and to all metadata values, regardless of the
scheme="" attribute.

(How do you map the postcode to that page, by the way?)


On Wed, 19 Aug 2009, Mike Thacker wrote:

>
> As I understand it, this thread is concerned with the usefulness of
> keeping the "scheme" attribute of the META element.
>
> The discussion comes down to whether or not schemes defining the format
> of values or, as described here, controlled lists adding definition and
> precise meaning to values (in the "content" attribute) are useful.
>
> To me adding defining the controlled list from which a precise value is
> taken has tremendous value, as described by Tom.
>
> Ian you say:
>
> >>Surely more modern search technologies would be a significantly more
> effective way of addressing the same problem at the user level, but with
> a significantly better user experience?<<
>
> Can you say (or provide a link to indicate) what these modern search
> technologies might be?

Well for example in this particular case it's not clear that the scheme=""
attribute is helping at all. Surely it would be better, as noted above, to
just apply the same processing to _all_ the keywords on the page, rather
than just those in the metadata headers, regardless of scheme="".

--
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

[Mike Thacker-2]

>>You don't need the "scheme" attribute to do that, though. This is
basically just synonym matching; you could apply the same synonym matching
to all terms on the page, and to all metadata values, regardless of the
scheme="" attribute.<<

Well the scheme defines the meaning of the content in a precise context.

So "Apple" from a scheme defining computers, would be different from "Apple" in a scheme defining fruit.

In this case "Abandoned vehicles" has a precise meaning in LGSL.  It defines a service that UK councils are required to provide under a law which defines the scope of the service and the duties of the council.  Pages with that term in the metadata with the "LGSL" scheme are contributing towards the fulfilment of that duty.


>>(How do you map the postcode to that page, by the way?)<<

The government site in question uses licensed data which identifies the council whose geaographical scope contains the specified post code.

Hope this helps.

Do say if you still feel that I am missing something and that scheme is redundant.

Mike

On Thu, Aug 27, 2009 at 19:58, Ian Hickson <ian@...> wrote:

On Wed, 19 Aug 2009, Tom Dent wrote:
> On Thu, Aug 13, 2009 at 12:19 PM, Ian Hickson<ian@...> wrote:
> > On Tue, 4 Aug 2009, Tom Dent wrote:
> >> On Wed, Jul 29, 2009 at 7:27 PM, Ian Hickson <ian@...> wrote:
> >>

> >> The scheme does not directly impact on the user, but can (for
> >> example) do so via search software.  You can see a view of IPSV at
> >> http://www.esd.org.uk/standards/ipsv/viewer/ or LGSL at
> >> http://www.esd.org.uk/standards/lgsl/viewer .
> >>
> >> If a user searched for pages by using a "non-preferred term", the
> >> search engine should convert that to an IPSV preferred term and then
> >> return all content with that preferred term in its metadata (with the
> >> IPSV scheme). The search software might also suggest showing content
> >> for broader and narrower subject headings from the IPSV hierarchy.
> >>
> >> The LGSL is being used to reference local authority sites in the UK
> >> Government's Directgov site (at http://www.direct.gov.uk/) which is
> >> used as a portal to local authority sites, and uses LGSL to reference
> >> pages.
> >

> > This doesn't really seem like a very compelling use case. Surely more

> > modern search technologies would be a significantly more effective way
> > of addressing the same problem at the user level, but with a

> > significantly better user experience? Can you walk me through what a

> > typical user might search for in a way that metadata with a scheme
> > would affect the result? I'm having trouble figuring out what a
> > suitable "non-preferred term" might be. For example, I did a search on
> > direct.gov.uk for "how do i report foreign income for tax purposes",
> > which gave me the same top result as Google did for that same search
> > (with a site restrict), and did not appear to be in any way affected
> > by metadata terms that used schemes.
>
> Using the LGSL metadata can make the seach at the Directgov site more
> powerful than Google. For example, this page about abandoned vehicles
> references LGSL:
>
> http://www.chiltern.gov.uk/site/scripts/services_info.php?serviceID=306&startsWith=A
>
> <meta name="eGMS.subject.service" lang="en" scheme="LGSL"
> content="Abandoned vehicles" />
>
> Doing a Google search for that abandoned vehicles and a postcode in that
> authority does not return the correct page:
> http://www.google.co.uk/search?hl=de&q=HP5+1RS+abandoned+vehicles
> However entering your postcode into the Directgov site (which also
> references the LGSL in the URL) does:
>
>  http://local.direct.gov.uk/LDGRedirect/LocationSearch.do?searchtype=1&LGSL=372&LGIL=0&Style=&formsub=t&text=HP5+1RS

You don't need the "scheme" attribute to do that, though. This is
basically just synonym matching; you could apply the same synonym matching
to all terms on the page, and to all metadata values, regardless of the
scheme="" attribute.

(How do you map the postcode to that page, by the way?)

On Wed, 19 Aug 2009, Mike Thacker wrote:
>
> As I understand it, this thread is concerned with the usefulness of
> keeping the "scheme" attribute of the META element.
>
> The discussion comes down to whether or not schemes defining the format
> of values or, as described here, controlled lists adding definition and
> precise meaning to values (in the "content" attribute) are useful.
>
> To me adding defining the controlled list from which a precise value is
> taken has tremendous value, as described by Tom.
>
> Ian you say:
>
> >>Surely more modern search technologies would be a significantly more
> effective way of addressing the same problem at the user level, but with
> a significantly better user experience?<<
>
> Can you say (or provide a link to indicate) what these modern search
> technologies might be?

Well for example in this particular case it's not clear that the scheme=""
attribute is helping at all. Surely it would be better, as noted above, to
just apply the same processing to _all_ the keywords on the page, rather
than just those in the metadata headers, regardless of scheme="".

--

Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

[Ian Hickson]

On Fri, 28 Aug 2009, Mike Thacker wrote:

>
> >>You don't need the "scheme" attribute to do that, though. This is
> basically just synonym matching; you could apply the same synonym matching
> to all terms on the page, and to all metadata values, regardless of the
> scheme="" attribute.<<
>
> Well the scheme defines the meaning of the content in a precise context.
>
> So "Apple" from a scheme defining computers, would be different from "Apple"
> in a scheme defining fruit.
>
> In this case "Abandoned vehicles" has a precise meaning in LGSL.  It
> defines a service that UK councils are required to provide under a law
> which defines the scope of the service and the duties of the council.  
> Pages with that term in the metadata with the "LGSL" scheme are
> contributing towards the fulfilment of that duty.

Sure, but what would break if instead of looking at scheme="", you just
applied the synonyms from all the various schemes that are supported?

--
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'