AGAIN: sharing the data

[Giancarlo Macchi]

Well since Vittorio's last words things got extremely quiet in
here:

 > Let's try to get more
 > practical in the next posts; if anyone agrees at all on the several
points
 > that have been raised, let's try to formulate some proposals on how to go
 > about it.
 >  

Are we stuck at this point? Yeah, I definitely think so. :-)

In fact we are able to talk about a lot on nice theoretical
things but when we try to move to a practical level then ...

Well, I think that I will still insist on the
"data sharing" issue. Which I think is actually the ISSUE here.
Just think for a second, if the data of my archaeological models
is available on the public domain then I am forced to be formal
in the quantification and definition processes.  Therefore
whoever would like to follow this path should be formal also in
the definition of the data itself.  And finally one may also be
forced to save his/her files encoded with common open formats.
Roberto has always insisted on this point: "the real problem is
the data". "Where is data?" "Do I have access to your data?"

So, before we start to be practical let us measure if it is worth it.

a) How many of you are willing to share their own data?
b) How many of you are able to share their own data?
c) What kind of data would you eventually share with the community?
   everything? a part of it? only the part related with my published
   models or theories?

Once we know exactly the amount and type of information that
eventually we will be available then probably we will decide if
it is worth to move to a more practical level.

Have a nice W.E.

   giancarlo




--
Mailing list info: http://lists.linux.it/listinfo/archaeology

[Chris Puttick]

Answers!

(a) We are.

(b) We are... With some exceptions due to contractual conditions, mostly only resulting in timelags of years rather than decades.

(c) All of it, clean and dirty, accurate measurements and passing comments.

Is a *lot* of data.

Chris


-----Original Message-----
From: archaeology-bounces+chris.puttick=oxfordarch.co.uk@... on behalf of Giancarlo Macchi
Sent: Fri 22/09/2006 13:05
To: archaeology@...
Subject: AGAIN: sharing the data
 

Well since Vittorio's last words things got extremely quiet in
here:

 > Let's try to get more
 > practical in the next posts; if anyone agrees at all on the several
points
 > that have been raised, let's try to formulate some proposals on how to go
 > about it.
 >  

Are we stuck at this point? Yeah, I definitely think so. :-)

In fact we are able to talk about a lot on nice theoretical
things but when we try to move to a practical level then ...

Well, I think that I will still insist on the
"data sharing" issue. Which I think is actually the ISSUE here.
Just think for a second, if the data of my archaeological models
is available on the public domain then I am forced to be formal
in the quantification and definition processes.  Therefore
whoever would like to follow this path should be formal also in
the definition of the data itself.  And finally one may also be
forced to save his/her files encoded with common open formats.
Roberto has always insisted on this point: "the real problem is
the data". "Where is data?" "Do I have access to your data?"

So, before we start to be practical let us measure if it is worth it.

a) How many of you are willing to share their own data?
b) How many of you are able to share their own data?
c) What kind of data would you eventually share with the community?
   everything? a part of it? only the part related with my published
   models or theories?

Once we know exactly the amount and type of information that
eventually we will be available then probably we will decide if
it is worth to move to a more practical level.

Have a nice W.E.

   giancarlo




--
Mailing list info: http://lists.linux.it/listinfo/archaeology


 





This message has been scanned for viruses by BlackSpider MailControl - www.blackspider.com

--
Mailing list info: http://lists.linux.it/listinfo/archaeology

[Sebastian Rahtz]

> Just think for a second, if the data of my archaeological models
> is available on the public domain then I am forced to be formal
> in the quantification and definition processes.  
This is a rather worrrying statement. You mean that you
won't follow good definition processes unless you
have to reveal the data? Why collect data at all
if it is not in a state to be shared with others?
>  And finally one may also be
> forced to save his/her files encoded with common open formats.
>  
Again, if you don't have this in place already,
it seems like a dereliction of duty. As soon
as you start collecting data, you should define
the procedures for long-term exchange
and archiving, surely?


--
Sebastian Rahtz      

Information Manager, Oxford University Computing Services
13 Banbury Road, Oxford OX2 6NN. Phone [phone]

OSS Watch: JISC Open Source Advisory Service
http://www.oss-watch.ac.uk


--
Mailing list info: http://lists.linux.it/listinfo/archaeology

[Giancarlo Macchi]

Sebastian,
Of course it is worrying! Otherwise why are we posting
here. Please, take into account that things in the U.K. and in
Italy are different. Here most of the archaeological records are
kept hidden to the rest of the scientific community. So basically
there is no need to think to "procedures for long-term exchange
and archiving". We are trying to change things and perhaps
you can help us.
With the words "to be formal" I meant to put things in a
comprehensible and exhaustive way for the public. We would be
very grateful if maybe you can provide and post some links of
repositories where you guys in U.K. have already solved the
problem of archaeological data sharing: metadata, open formats,
readability of the data 100 years from now etc?
Thank you in advance,

  giancarlo

--
Mailing list info: http://lists.linux.it/listinfo/archaeology

[Dorothy Graves]

Giancarlo said:  "We would be
very grateful if maybe you can provide and post some links of
repositories where you guys in U.K. have already solved the
problem of archaeological data sharing: metadata, open formats,
readability of the data 100 years from now etc?"


I can chime in a tiny bit from the UK/Scotland side of things.  All of my preliminary archaeological data came from the RCAHMS, and one of the requirements they made was that I would give them a copy of my thesis/digital data when I completed.  My research would be available to anyone in the public from that moment forward, because it'd be part of the RCAHMS library.  On the other hand, the geographical data I used (i.e. the 1:10k DTMs and vector-based data) is the sole property of the Ordnance Survey, and expensive to get ahold of unless you're part of a university or other scheme allowing access.  That stuff is of course available at Digimap (http://edina.ac.uk/digimap/) 

As to my data, I'm unclear as to what a member of public could do with my data without access to the Ordnance Survey data.  I guess they could visualise my database in the RCAHMS library and may be allowed to take a copy of it with them, but they wouldn't be allowed to take any of the Ordinance Survey data home with them.  However, the main thing is that my archaeological data would be available to anyone - archaeologists included.  There is a massive drive for digitised archaeological data for the UK; the main repository is of course at http://ads.ahds.ac.uk/

Hope that answers a few questions, or at least gives you interesting links to look at.

Dot
Giancarlo Macchi <macchi@...> wrote:

Sebastian,
Of course it is worrying! Otherwise why are we posting
here. Please, take into account that things in the U.K. and in
Italy are different. Here most of the archaeological records are
kept hidden to the rest of the scientific community. So basically
there is no need to think to "procedures for long-term exchange
and archiving". We are trying to change things and perhaps
you can help us.
With the words "to be formal" I meant to put things in a
comprehensible and exhaustive way for the public. We would be
very grateful if maybe you can provide and post some links of
repositories where you guys in U.K. have already solved the
problem of archaeological data sharing: metadata, open formats,
readability of the data 100 years from now etc?
Thank you in advance,

giancarlo

--
Mailing list info: http://lists.linux.it/listinfo/archaeology

---------------------------
Dorothy Graves
Postgraduate Researcher
Department of Archaeology
School of Arts, Culture and Environment
Old High School, Infirmary Street
Edinburgh EH1 1LT, Scotland, UK

---

How low will we go? Check out Yahoo! Messenger’s low PC-to-Phone call rates.

--
Mailing list info: http://lists.linux.it/listinfo/archaeology

[Sebastian Rahtz]

Giancarlo Macchi wrote:
> We would be
> very grateful if maybe you can provide and post some links of
> repositories where you guys in U.K. have already solved the
> problem of archaeological data sharing: metadata, open formats,
> readability of the data 100 years from now etc?
>
As already noted, we have a national data service
(http://ads.ahds.ac.uk/) for archaeology. There's
a lot of stuff around there on this subject. The day to
day problems of physically managing data, migrating
to new systems, etc, are covered by all the components
of the  AHDS;  metadata and open formats are
also common to all of them, though the exact standards
vary.

Coincidentally, the ADS have a workshop next week
on "XML for archaeologists"
(http://ads.ahds.ac.uk/project/workshop/research06.html)
at which I am talking; one of the things I wanted to discuss
was the relationship between working data stores,
and archival forms. My example is from a classical
project with a large clunky relational database, which
does their daily work; but which would be useless handed
over to someone else. So the point is to export to
an archival XML form using a well-understood schema.
No big deal, really; but to illustrate the idea that
your archival/interchange format is not your
working format. One of the side effects is
being able to offer the XML as a machine-accessible
source via web services.

--
Sebastian Rahtz      

Information Manager, Oxford University Computing Services
13 Banbury Road, Oxford OX2 6NN. Phone [phone]

OSS Watch: JISC Open Source Advisory Service
http://www.oss-watch.ac.uk


--
Mailing list info: http://lists.linux.it/listinfo/archaeology

[Chris Puttick]

I would add some possibel controversy to that - the main repository for grant-funded work is at ADS (which is a chargeable service)- contracting units hold far more data, hence our intention to make our own holdings open and others in the UK market seem to be heading in a similar direction.

I tend to assume however this "opening" is done not in a file or files mentality but as a service, with federated search feeds for global searches and local site free text search. This allows the map context to remain available through WMS/WFS layers - some sort of SDI underpinning it. Open formats are important for the services and any accompanying files. For all the geospatial stuff there is the OGC standards, files are now best in ISO 26300 (Open Document Format) and where a standard doesn't exist, make sure the software used to provide and read it is open source.

Metadata is in reality an open question. Dublin core plus some real basics (site location (name/lat,long), relevant historical periods, ?). After that, well, that's a long, heated discussion. I think it's important to remember that we preserve for social reasons, not for the specialists now. Bu then I'm not a specialist.

And no, my limited understanding has it the UK has *not* solved these problems. In archaeology or elsewhere. Theres a starting point for discussion though ;-).

Chris


-----Original Message-----
From: archaeology-bounces+chris.puttick=oxfordarch.co.uk@... on behalf of Dorothy Graves
Sent: Fri 22/09/2006 16:50
To: archaeology@...
Subject: Re: AGAIN: sharing the data
 
Giancarlo said:  "We would be
very grateful if maybe you can provide and post some links of
repositories where you guys in U.K. have already solved the
problem of archaeological data sharing: metadata, open formats,
readability of the data 100 years from now etc?"


I can chime in a tiny bit from the UK/Scotland side of things.  All of my preliminary archaeological data came from the RCAHMS, and one of the requirements they made was that I would give them a copy of my thesis/digital data when I completed.  My research would be available to anyone in the public from that moment forward, because it'd be part of the RCAHMS library.  On the other hand, the geographical data I used (i.e. the 1:10k DTMs and vector-based data) is the sole property of the Ordnance Survey, and expensive to get ahold of unless you're part of a university or other scheme allowing access.  That stuff is of course available at Digimap (http://edina.ac.uk/digimap/)  

As to my data, I'm unclear as to what a member of public could do with my data without access to the Ordnance Survey data.  I guess they could visualise my database in the RCAHMS library and may be allowed to take a copy of it with them, but they wouldn't be allowed to take any of the Ordinance Survey data home with them.  However, the main thing is that my archaeological data would be available to anyone - archaeologists included.  There is a massive drive for digitised archaeological data for the UK; the main repository is of course at http://ads.ahds.ac.uk/

Hope that answers a few questions, or at least gives you interesting links to look at.

Dot
Giancarlo Macchi <macchi@...> wrote: Sebastian,
Of course it is worrying! Otherwise why are we posting
here. Please, take into account that things in the U.K. and in
Italy are different. Here most of the archaeological records are
kept hidden to the rest of the scientific community. So basically
there is no need to think to "procedures for long-term exchange
and archiving". We are trying to change things and perhaps
you can help us.
With the words "to be formal" I meant to put things in a
comprehensible and exhaustive way for the public. We would be
very grateful if maybe you can provide and post some links of
repositories where you guys in U.K. have already solved the
problem of archaeological data sharing: metadata, open formats,
readability of the data 100 years from now etc?
Thank you in advance,

  giancarlo

--
Mailing list info: http://lists.linux.it/listinfo/archaeology



---------------------------
Dorothy Graves
Postgraduate Researcher
Department of Archaeology
School of Arts, Culture and Environment
Old High School, Infirmary Street
Edinburgh EH1 1LT, Scotland, UK
 
---------------------------------
How low will we go? Check out Yahoo! Messenger's low  PC-to-Phone call rates.

 





This message has been scanned for viruses by BlackSpider MailControl - www.blackspider.com

--
Mailing list info: http://lists.linux.it/listinfo/archaeology

[Sebastian Rahtz]

Chris Puttick wrote:
> I would add some possibel controversy to that - the main repository for grant-funded work is at ADS (which is a chargeable service)- contracting units hold far more data, hence our intention to make our own holdings open and others in the UK market seem to be heading in a similar direction.
>  
I should probably know the answer to this, but to what extent are
you obliged to produce the data on demand, if the work is
funded by government money?
>  files are now best in ISO 26300 (Open Document Format)
one might disagree. holding text in what is simply standardized
word-processor
format is not as useful as it might be.
> Metadata is in reality an open question. Dublin core plus some real basics (site location (name/lat,long), relevant historical periods, ?). After that, well, that's a long, heated discussion.
it is indeed :-} "relevant historical periods" is a nice wrapper for a
can of worms.
> I think it's important to remember that we preserve for social reasons, not for the specialists now. Bu then I'm not a specialist.
>
>  
by "social", you mean "political" there?

by why do you collect data if _not_ to preserve?

--
Sebastian Rahtz      

Information Manager, Oxford University Computing Services
13 Banbury Road, Oxford OX2 6NN. Phone [phone]

OSS Watch: JISC Open Source Advisory Service
http://www.oss-watch.ac.uk


--
Mailing list info: http://lists.linux.it/listinfo/archaeology

[Sebastian Rahtz]

Chris Puttick wrote:
> I would add some possibel controversy to that - the main repository for grant-funded work is at ADS (which is a chargeable service)- contracting units hold far more data, hence our intention to make our own holdings open and others in the UK market seem to be heading in a similar direction.
>  
I should probably know the answer to this, but to what extent are
you obliged to produce the data on demand, if the work is
funded by government money?
>  files are now best in ISO 26300 (Open Document Format)
one might disagree. holding text in what is simply standardized
word-processor
format is not as useful as it might be.
> Metadata is in reality an open question. Dublin core plus some real basics (site location (name/lat,long), relevant historical periods, ?). After that, well, that's a long, heated discussion.
it is indeed :-} "relevant historical periods" is a nice wrapper for a
can of worms.
> I think it's important to remember that we preserve for social reasons, not for the specialists now. Bu then I'm not a specialist.
>
>  
by "social", you mean "political" there?

but why do you collect data if _not_ to preserve? to
simply draw inferences from?

--
Sebastian Rahtz      

Information Manager, Oxford University Computing Services
13 Banbury Road, Oxford OX2 6NN. Phone [phone]

OSS Watch: JISC Open Source Advisory Service
http://www.oss-watch.ac.uk


--
Mailing list info: http://lists.linux.it/listinfo/archaeology

[Chris Puttick]

-----Original Message-----
From: Sebastian Rahtz [mailto:sebastian.rahtz@...]
Sent: Fri 22/09/2006 17:36
To: Chris Puttick
Cc: archaeology@...
Subject: Re: AGAIN: sharing the data
 
Chris Puttick wrote:
> I would add some possibel controversy to that - the main repository for grant-funded work is at ADS (which is a chargeable service)- contracting units hold far more data, hence our intention to make our own holdings open and others in the UK market seem to be heading in a similar direction.
>  
I should probably know the answer to this, but to what extent are
you obliged to produce the data on demand, if the work is
funded by government money?

>>> Not at all unless specified in contrsct. With most grant funded work there is a stipulation to publish reports and deposit materials and for English Heritage work, at least at the current time, there is a digital data deposition requirement to ADS. However, by far the great majority of archaeology work is carried out on behalf of developers rather than under grant, and the conditions are mostly limited to reporting and deposit of finds along with the traditional UK archaeological archive i.e. no digital deposit takes place.

>  files are now best in ISO 26300 (Open Document Format)
one might disagree. holding text in what is simply standardized
word-processor format is not as useful as it might be.

>>> Depends on the file type - by files I was implying office productivity files, and for those ISO 26300 is pretty much the only preservation choice, and holds far more meaning than is present just in text, numbers and pictures.

> Metadata is in reality an open question. Dublin core plus some real basics (site location (name/lat,long), relevant historical periods, ?). After that, well, that's a long, heated discussion.
it is indeed :-} "relevant historical periods" is a nice wrapper for a
can of worms.

>>> And that sort of definition work I'll leave to the specialists, with a plea (a demand) that they leave and explain room for interpretation. The further details of a preservation and access solution is the one I'm really interested in further discussion of...

> I think it's important to remember that we preserve for social reasons, not for the specialists now. Bu then I'm not a specialist.
>
>  
by "social", you mean "political" there?

by why do you collect data if _not_ to preserve?

>>> By social I mean for society e.g. for the likes of me, a common non-archaeologist tax payer, rather than just or even primarily for the experts and researchers; and then not just me now but future generations. This a) means we cannot presume what is meaningless and therefore discard it and (b) we should consider the marginal value of the work done as part of the preservation of the record.

>>> And on why do we collect data? Depends who you ask - definitely seems for some archaeologists it is primarily for their own ends - this type demonstrate limited interest in what happens to the data once they have studied it and written about it. Personally I preserve because I should and I can. And it's my job ;-).

Chris

 





This message has been scanned for viruses by BlackSpider MailControl - www.blackspider.com

--
Mailing list info: http://lists.linux.it/listinfo/archaeology

[Stefano Costa]

Il giorno sab, 23/09/2006 alle 19.21 +0100, Chris Puttick ha scritto:
> >  files are now best in ISO 26300 (Open Document Format)
> one might disagree. holding text in what is simply standardized
> word-processor format is not as useful as it might be.
>
> >>> Depends on the file type - by files I was implying office
>  productivity files, and for those ISO 26300 is pretty much the only
>  preservation choice, and holds far more meaning than is present just
>  in text, numbers and pictures.

Current archaeological work can be done without ever using an office
suite. ODF is good for what it was meant for - bureaus and the like. But
I second you on the fact that standards (ISO, or similar ones) are
missing here.

> > Metadata is in reality an open question. Dublin core plus some real
>  basics (site location (name/lat,long), relevant historical periods,
>  ?). After that, well, that's a long, heated discussion. it is indeed
>  :-} "relevant historical periods" is a nice wrapper for a can of
>  worms.
>
> >>> And that sort of definition work I'll leave to the specialists,
>  with a plea (a demand) that they leave and explain room for
>  interpretation. The further details of a preservation and access
>  solution is the one I'm really interested in further discussion of...

There's really a lot of stuff out there about metadata and ontologies
(which are strictly tied just because metadata are not so useful on
their own - machines should be able to "understand" them).

> > I think it's important to remember that we preserve for social
>  reasons, not for the specialists now. Bu then I'm not a specialist. >
> >   by "social", you mean "political" there?
>
> by why do you collect data if _not_ to preserve?
>
> >>> By social I mean for society e.g. for the likes of me, a common
>  non-archaeologist tax payer, rather than just or even primarily for
>  the experts and researchers; and then not just me now but future
>  generations. This a) means we cannot presume what is meaningless and
>  therefore discard it and (b) we should consider the marginal value of
>  the work done as part of the preservation of the record.

> >>> And on why do we collect data? Depends who you ask - definitely
>  seems for some archaeologists it is primarily for their own ends -
>  this type demonstrate limited interest in what happens to the data
>  once they have studied it and written about it. Personally I preserve
>  because I should and I can. And it's my job ;-).

This is not an archaeology-only problem. Public entities collect
archives of everything that is falling under their domain, including
archaeological data, that we are collecting for our purposes (research,
preservation, or whatever one might want to do while being an
archaeologist). This is what can be done for the tax payers, and as one
of them I really don't like archives (especially archaeological ones)
being locked, secret, unmanageable places as they are - at least in
Italy.

Previously on this list I had written about this issue that I value data
sharing far more than preservation per se. Sharing makes sense because
it is a powerful mean for selecting valuable contents and disregarding
others. If everything gets collected but nothing gets evaluated and
used, why collect?

But one cannot collect everything: we have to make selections, it's not
a choice but a must. Digital archives are far less heavy than old ones,
but they don't come at no cost. Data storage needs to be paid and though
its immaterial nature it needs however some physical space. (Though I
admit this is not a strong point).

And if you are worried about data selection and preservation for future
generations, don't forget that there are people in archaeology who use
to think that we are making (sometimes unconscious) selections at every
stage of our research, from early projects through data collecting, to
their publication and dissemination. I don't agree 100% with those
theories, but I just think we can't ignore this issue when we talk about
data.

There are great differences between Italy and UK, but I see there are
subscribers from other countries too that I would like to hear about.

Best regards,

Stefano

--
Stefano Costa
http://www.iosa.it Software Open Source per l'Archeologia
Jabber: steko@...
GnuPG Key ID 1024D/0xD0D30245
Linux Registered User #385969 counter.li.org


--
Mailing list info: http://lists.linux.it/listinfo/archaeology

[Sebastian Rahtz]

Stefano Costa wrote
> And if you are worried about data selection and preservation for future
> generations, don't forget that there are people in archaeology who use
> to think that we are making (sometimes unconscious) selections at every
> stage of our research, from early projects through data collecting, to
> their publication and dissemination. I don't agree 100% with those
> theories, but I just think we can't ignore this issue when we talk about
> data.
Obviously, we make choices when we record archaeological
data - but within the current norms agreed by our peers.
The important thing, presumably, is a) to articulate and
document those choices; and b) once you are aware
of having collected some data, not to throw it
away without a similar process of articulation and
documentation. If we don't record our processes,
we are not scientists any more.

--
Sebastian Rahtz      

Information Manager, Oxford University Computing Services
13 Banbury Road, Oxford OX2 6NN. Phone [phone]

OSS Watch: JISC Open Source Advisory Service
http://www.oss-watch.ac.uk


--
Mailing list info: http://lists.linux.it/listinfo/archaeology

[Giancarlo Macchi]

Hi Dorothy,
Well the point is exactly this. I will use your case as an
example, but also to know if things are different in the UK.
So this mail is full of questions.

The people from the RCAHMS gave you an archaeological
digital data set. Let's call this dataset x.
You will conduct a research. During this research you will
apply a some quantitative methods, approaches etc. Let's
imagine your research can be synthesized as a function: f().
So we may hope that at the end of your research you will
publish a volume, that if I have understood will be available
to everyone through the RCAHMS library. Let's call the outcome
of your research V.
So basically we may say that:

     f(x) -> V

Once V is published how can I validate or refuse V? Will
other people around the world be entitled to have access
to x? I mean the exact data set you have used. But also f()
is important. What are the exact algorithm's you have used?
Are they open or closed source?
So basically my point is that for a real scientific approach
it is important to the public have access to V (obviously)
but also to x and eventually to f().

Sebastian, my original post (AGAIN: sharing the data) wants
to address this point. I want you and everybody else around
the globe to be entitled to challenge my models. To do that
I need do guarantee you access to my data.

giancarlo


 


Dorothy Graves ha scritto:

>
>
> I can chime in a tiny bit from the UK/Scotland side of things.  All of
> my preliminary archaeological data came from the RCAHMS, and one of
> the requirements they made was that I would give them a copy of my
> thesis/digital data when I completed.  My research would be available
> to anyone in the public from that moment forward, because it'd be part
> of the RCAHMS library.  On the other hand, the geographical data I
> used (i.e. the 1:10k DTMs and vector-based data) is the sole property
> of the Ordnance Survey, and expensive to get ahold of unless you're
> part of a university or other scheme allowing access.  That stuff is
> of course available at Digimap (http://edina.ac.uk/digimap/)

--
Mailing list info: http://lists.linux.it/listinfo/archaeology

[Stefano Costa]

Il giorno dom, 24/09/2006 alle 23.32 +0100, Sebastian Rahtz ha scritto:

> Obviously, we make choices when we record archaeological
> data - but within the current norms agreed by our peers.

Yes, and these norms are different in each country (at least). There are
countries where norms are just common sense, and others where norms come
from government officers.

> The important thing, presumably, is a) to articulate and
> document those choices; and b) once you are aware
> of having collected some data, not to throw it
> away without a similar process of articulation and
> documentation. If we don't record our processes,
> we are not scientists any more.

I agree 100% with you. But, in your professional experience, how many
people do like that? Moreover, how many just don't care about "being
scientists"?



--
Stefano Costa
http://www.iosa.it Software Open Source per l'Archeologia
Jabber: steko@...
GnuPG Key ID 1024D/0xD0D30245
Linux Registered User #385969 counter.li.org


--
Mailing list info: http://lists.linux.it/listinfo/archaeology

[Sebastian Rahtz]

>>  If we don't record our processes,
>> we are not scientists any more.
>>    
>
> I agree 100% with you. But, in your professional experience, how many
> people do like that?
when I used to see field archaeologists in the UK
(its been quite a while since I escaped to the safer
confines of IT!), most people at least
_thought_ thats what they were doing.
>  Moreover, how many just don't care about "being
> scientists"?
>  
to be fair, I do not often meet people like that.


--
Sebastian Rahtz      

Information Manager, Oxford University Computing Services
13 Banbury Road, Oxford OX2 6NN. Phone [phone]

OSS Watch: JISC Open Source Advisory Service
http://www.oss-watch.ac.uk


--
Mailing list info: http://lists.linux.it/listinfo/archaeology

[Dorothy Graves]

"Once V is published how can I validate or refuse V? Will
other people around the world be entitled to have access
to x? I mean the exact data set you have used. But also f()
is important. What are the exact algorithm's you have used?
Are they open or closed source?"

I agree that this is a problem.  Any mathematical algorithms used must be published in my thesis (V), and my database (x) would be available in digital format.  But of course, the algorithms used by software I use may not be accessible because I use ArcGIS, Imagine, and Idrisi.  I have tried more than once to learn how to use GRASS, and found the process extremely frustrating.  I am not using it to complete my research, but I did give it a fair go more than once.  I do not have the programming skills to use it even remotely effectively, and at this point in my research, I cannot sacrifice the time to learn.  You could arguably take my database and put it into the different closed-source packages I have used, but there's always the inherent danger of incongruent GISes.  I find that to be extremely worrying, more so than the question of access to data.

Others using this list may feel quite strongly that I - and others like me - am doing a disservice to the field for not using open source software.  (I agree, incidentally.)  As my computer skillls grow, I hope to eventually migrate over to open source, but for now this is simply impossible.  A lack of training in computer science is probably the real problem here, because that knowledge breaks down the real barriers of using open source, and thus resolves the problem of truly verifying/testing others' research.  I know I wasn't the only student at QMDAA who had little or no background in computer science, which may suggest this problem is endemic to archaeology as it's practiced throughout the world.

So I put it to anyone in this list - do you use open source software for your research?  If so, how did you learn to use the software?  If you are self-taught, how long did it take to wield it effectively? 

Dorothy

Giancarlo Macchi <macchi@...> wrote:

Hi Dorothy,
Well the point is exactly this. I will use your case as an
example, but also to know if things are different in the UK.
So this mail is full of questions.

The people from the RCAHMS gave you an archaeological
digital data set. Let's call this dataset x.
You will conduct a research. During this research you will
apply a some quantitative methods, approaches etc. Let's
imagine your research can be synthesized as a function: f().
So we may hope that at the end of your research you will
publish a volume, that if I have understood will be available
to everyone through the RCAHMS library. Let's call the outcome
of your research V.
So basically we may say that:

f(x) -> V

Once V is published how can I validate or refuse V? Will
other people around the world be entitled to have access
to x? I mean the exact data set you have used. But also f()
is important. What are the exact algorithm's you have used?
Are they open or closed source?
So basically my point is that for a real scientific approach
it is important to the public have access to V (obviously)
but also to x and eventually to f().

Sebastian, my original post (AGAIN: sharing the data) wants
to address this point. I want you and everybody else around
the globe to be entitled to challenge my models. To do that
I need do guarantee you access to my data.

giancarlo

---------------------------
Dorothy Graves
Postgraduate Researcher
Department of Archaeology
School of Arts, Culture and Environment
Old High School, Infirmary Street
Edinburgh EH1 1LT, Scotland, UK

---

Get your own web address for just $1.99/1st yr. We'll help. Yahoo! Small Business.

--
Mailing list info: http://lists.linux.it/listinfo/archaeology

[Sebastian Rahtz]

Dorothy Graves wrote:
> I agree that this is a problem.  Any mathematical algorithms used must be published in my thesis (V), and my database (x) would be available in digital format.  But of course, the algorithms used by software I use may not be accessible because I use ArcGIS, Imagine, and Idrisi.  I have tried more than once to learn how to use GRASS, and found the process extremely frustrating.  I am not using it to complete my research, but I did give it a fair go more than once.  I do not have the programming skills to use it even remotely effectively, and at this point in my research, I cannot sacrifice the time to learn.  You could arguably take my database and put it into the different closed-source packages I have used, but there's always the inherent danger of incongruent GISes.  I find that to be extremely worrying, more so than the question of access to data.
>
>  
You are implying that closed source software may always involve proprietary
analytical algorithms. In much software, it really does not matter; yes,
Oracle
no doubt have secret code for making big fast databases, while MySQL's
can be
examined. But for the purposes of this argument we don't care.

GIS may be a different matter. But I don't believe that what ArcGIS or
Idrisi
do for you is entirely opaque - you wouldn't use a function which
just said "give me all your data and I'll show you The Answer". You
use functions there which correspond to some known technique, surely?
It may be that I cannot immediately reproduce your results, because only
Idrisi has implemented the technique, and it would be nicer if we had
an open source implementation too; but its not the end of the world.
It is in the interests of GIS vendors to stay honest and publish their
algorithms, surely?

However, one of the best arguments of OSS is that it allows
us to have a public implementation of a standard or a technique,
to test the commercial vendors against, and keep them clean.
So I share the concern.



--
Sebastian Rahtz      

Information Manager, Oxford University Computing Services
13 Banbury Road, Oxford OX2 6NN. Phone [phone]

OSS Watch: JISC Open Source Advisory Service
http://www.oss-watch.ac.uk


--
Mailing list info: http://lists.linux.it/listinfo/archaeology

[Dorothy Graves]

"You are implying that closed source software may always involve proprietary
    analytical algorithms. In much software, it really does not matter; yes,
    Oracle no doubt have secret code for making big fast databases, while MySQL's can be examined. But for the purposes of this argument we don't care."

Right, I'm not very sophisticated when it comes to computer programmes; I think I made that pretty clear.   I'm a typical archaeologist attempting to use quantitative methods to understand the past.  At the QMDAA summer school, some talk was made about giving data to different GISes to see whether they perform analyses in the same way and get the same results.  Apparently the results can vary, and that may be a problem.  However, the main point I was trying to make is that while everyone can agree that OSS is best for archaeology for a number of reasons, we still struggle to implement it.
  
    "GIS may be a different matter. But I don't believe that what ArcGIS or
    Idris
    do for you is entirely opaque - you wouldn't use a function which
    just said "give me all your data and I'll show you The Answer". You
    use functions there which correspond to some known technique, surely?"

That's what I said - re: publishing the mathematical algorithms (meant in the broadest sense) in my thesis.

I think you're missing the point I was trying to make. If you don't have a background in computer science, how can you implement a strategy designed to leave data as open as possible for others?  Do you hire a specialist who may be able to sort out your database problems and tailor-create programmes for your individual site/problem?  This is expensive, and doesn't even begin to address the long-term problems of getting archaeologists to learn and use OSS as a matter of course. It would be far better for archaeology as a discipline to have more archaeologists skilled in quantitative methods and computer science, capable of using a variety of different programmes as the data/problem warrants, and capable of using OSS.  Training, training, training, is what I honestly believe we need.  Some universities and groups are valiantly trying to do this, and I applaud their efforts.  Certainly waiting on archaeologists to pick it up on their own in their spare time is not working.

   "However, one of the best arguments of OSS is that it allows
    us to have a public implementation of a standard or a technique,
    to test the commercial vendors against, and keep them clean.
    So I share the concern."

I'd say the best argument of OSS is that it's free, and can be tailored to specifically address the problems in archaeological research.  It is not easy to pick up though, and this is what causes some to avoid it in the end, like me.  


---------------------------
Dorothy Graves
Postgraduate Researcher
Department of Archaeology
School of Arts, Culture and Environment
Old High School, Infirmary Street
Edinburgh EH1 1LT, Scotland, UK

---

Get your email and more, right on the new Yahoo.com

--
Mailing list info: http://lists.linux.it/listinfo/archaeology

[Sebastian Rahtz]

Dorothy Graves wrote:
> ourse. It would be far better for archaeology as a discipline to have more archaeologists skilled in quantitative methods and computer science, capable of using a variety of different programmes as the data/problem warrants, and capable of using OSS.
This seems a bit of a red herring. Archaeologists need training in
quantitative methods, obviously (well, to most people....).
But do they need to be computer scientists? and I would argue strongly
that OSS has no special
characteristics which make people treat it differently. OSS is a
business model, and a software development
model - it does not produce different types of software. OK, so GRASS is
not as easy to use as Idrisi;
but similarly the IBM backup software we use here has an interface from
the dark ages.
> I'd say the best argument of OSS is that it's free,
being British, do you mean it costs nothing? if so, that implies that
archaeologists
are by definition poor, which is simply not true! If you mean "libre",
than yes
>  and can be tailored to specifically address the problems in archaeological research.
which is good.
>  It is not easy to pick up though, and this is what causes some to avoid it in the end, like me.    
>
>  
you are lumping all OSS software together as clunky and hard to use. That
is NOT a prerequisite of being OSS!

--
Sebastian Rahtz      

Information Manager, Oxford University Computing Services
13 Banbury Road, Oxford OX2 6NN. Phone [phone]

OSS Watch: JISC Open Source Advisory Service
http://www.oss-watch.ac.uk


--
Mailing list info: http://lists.linux.it/listinfo/archaeology

[Wood]

Sharing Data is an extremely important issue. In my opinion the value gained far outways the complications which arise when trying to standardise data, methods etc. If you are wanting to move to a more practical level you should have a look at www.nabonidus.org. It is an evolving website for the storage, management, manipulation and publication of excavation data. It is totally free and we think far advanced of any data recording system in use on any academic or commercial excavation today.

The reason for offering Nabonidus to the archaeological community is that we believe a system such as this could change archaeological excavation and research across the world. It is currently used by the Pompeii Archaeological Research Project: Porta Stabia run by the University of Michigan and Stanford University and has the capacity to be used by 1000’s of excavations. All data is secure and private, accessible and malleable and the system can be adapted to any excavation’s needs. Nabonidus saves enormous amounts of work and time for archaeologists both onsite and during post excavation research. Even if you are currently using a database for your excavation, all information can be easily imported from this into the Nabonidus system. Please if you have the time; take a look at the website and at the demonstration excavations and feel free to experiment. There is a PDF attached with more information.

Nabonidus is produced on an entirely voluntary basis. It has been built by archaeologists for archaeologists so if you have any questions or contributions please get in touch at [e-mail] – your suggestions will be incorporated into the next build.

This is the blurb from our website - after reading these threads I am glad to see that others are considering and wanting to address the issue of archaoelogical data sharing. Nabonidus gives you the choice of making your data public or private so prior to publication you could keep it private if wished then expose it all for anyone to look at.

Please have a look, we would be happy to hear from anyone. thanks

Sam Wood

Giancarlo Macchi wrote:

Well since Vittorio's last words things got extremely quiet in
here:

 > Let's try to get more
 > practical in the next posts; if anyone agrees at all on the several
points
 > that have been raised, let's try to formulate some proposals on how to go
 > about it.
 >  

Are we stuck at this point? Yeah, I definitely think so. :-)

In fact we are able to talk about a lot on nice theoretical
things but when we try to move to a practical level then ...

Well, I think that I will still insist on the
"data sharing" issue. Which I think is actually the ISSUE here.
Just think for a second, if the data of my archaeological models
is available on the public domain then I am forced to be formal
in the quantification and definition processes.  Therefore
whoever would like to follow this path should be formal also in
the definition of the data itself.  And finally one may also be
forced to save his/her files encoded with common open formats.
Roberto has always insisted on this point: "the real problem is
the data". "Where is data?" "Do I have access to your data?"

So, before we start to be practical let us measure if it is worth it.

a) How many of you are willing to share their own data?
b) How many of you are able to share their own data?
c) What kind of data would you eventually share with the community?
   everything? a part of it? only the part related with my published
   models or theories?

Once we know exactly the amount and type of information that
eventually we will be available then probably we will decide if
it is worth to move to a more practical level.

Have a nice W.E.

   giancarlo




--
Mailing list info: http://lists.linux.it/listinfo/archaeology