data import question & 008 use in Koha

[Christopher Curry-2]

Hello all,

We're in the middle of migrating MARC data to Koha.  We tried importing serials data into Koha and a record that had 1900+ items attached did not import.  I found the record and tested importing it, removing items until it would import and found out that 1238 was the maximum number of items I could import with one MARC record.  Can anyone tell me if there is a way to configure the import to accept more items?

I used bulkmarcimport.pl on Koha 3.0.1, running on Debian Lenny.

Also, our old ILS has a database issue that results in incomplete export of records created from original cataloging.  These records are missing 008 fields.  We're working out a method of exporting the data from the back end and splicing it back into the MARC before import, but the data imports into Koha without announced or noticeable errors.  I wonder if we're wasting our time.  Does Koha use the 008 field for anything?


_______________________________________________
Koha-devel mailing list
Koha-devel@...
http://lists.koha.org/mailman/listinfo/koha-devel

[tajoli-2]

Hi,

>Also, our old ILS has a database issue that results in incomplete
>export of records created from original cataloging.  These records
>are missing 008 fields.  We're working out a method of exporting the
>data from the back end and splicing it back into the MARC before
>import, but the data imports into Koha without announced or
>noticeable errors.  I wonder if we're wasting our time.  Does Koha
>use the 008 field for anything?

I answer on the question about missing 008.
I think you can use import without 008 but you loose many indexes
based on 008 values.
In fact 008 is an important point in a good Marc21 record.

If you use Zebra, the definition of indexes are in the file
etc/zebradb/marc_defs/marc21/biblios/record.abs
The indexes connect with 008 are:

date-entered-on-file:n:range(data,0,5),date-entered-on-file:s:range(data,0,5),
pubdate:w:range(data,7,4),pubdate:n:range(data,7,4),pubdate:y:range(data,7,4),pubdate:s:range(data,7,4),
pl:w:range(data,15,3),
ta:w:range(data,22,1),
ff8-23:w:range(data,23,1),ff8-29:w:range(data,29,1),
lf:w:range(data,33,1),
bio:w:range(data,34,1),
ln:n:range(data,35,3),
ctype:w:range(data,24,4),
Record-source:w:range(data,39,0)

So I strongly suggest you to insert records with 008.
But for test you can insert also data without.

Bye


Zeno Tajoli
CILEA - Segrate (MI)
tajoliAT_SPAM_no_prendiATcilea.it
(Indirizzo mascherato anti-spam; sostituisci quanto tra AT con @)

_______________________________________________
Koha-devel mailing list
Koha-devel@...
http://lists.koha.org/mailman/listinfo/koha-devel

[Magnus Enger]

2009/11/3 Christopher Curry <ccurry@...>:

> Hello all,
>
> We're in the middle of migrating MARC data to Koha.  We tried importing
> serials data into Koha and a record that had 1900+ items attached did not
> import.  I found the record and tested importing it, removing items until it
> would import and found out that 1238 was the maximum number of items I could
> import with one MARC record.  Can anyone tell me if there is a way to
> configure the import to accept more items?
>
> I used bulkmarcimport.pl on Koha 3.0.1, running on Debian Lenny.

I think you are running into one of the limitations that the MARC
format inhertited from the days when magnetic tape was the cool new
thing...

"Record length  (character positions 00-04 [in the leader]), contains
a five-character ASCII numeric string equal to the length of the
entire record, including itself and the record terminator. The
five-character numeric string is right justified and unused positions
contain zeroes (zero fill). ***The maximum length of a record is 99999
octets***." (My attempt at emphasis.)
http://www.loc.gov/marc/specifications/specrecstruc.html#genrec

And if you have enough items you reach that limit in size.

I have this same problem with one of the records i tried to import for
my client #1 - my plan is to try and transform the record to MARCXML,
which does not impose arbitrary limits on the size of a record, and
import that with the "-m MARCXML" option for bulkmarcimport.pl.

Regards,
Magnus Enger
libriotech.no
_______________________________________________
Koha-devel mailing list
Koha-devel@...
http://lists.koha.org/mailman/listinfo/koha-devel

[Christopher Curry-2]

Thanks, Magnus & Zeno, for your very useful replies.


Magnus Enger wrote:

2009/11/3 Christopher Curry ccurry@...:
  

Hello all,

We're in the middle of migrating MARC data to Koha.  We tried importing
serials data into Koha and a record that had 1900+ items attached did not
import.  I found the record and tested importing it, removing items until it
would import and found out that 1238 was the maximum number of items I could
import with one MARC record.  Can anyone tell me if there is a way to
configure the import to accept more items?

I used bulkmarcimport.pl on Koha 3.0.1, running on Debian Lenny.
    

I think you are running into one of the limitations that the MARC
format inhertited from the days when magnetic tape was the cool new
thing...

"Record length  (character positions 00-04 [in the leader]), contains
a five-character ASCII numeric string equal to the length of the
entire record, including itself and the record terminator. The
five-character numeric string is right justified and unused positions
contain zeroes (zero fill). ***The maximum length of a record is 99999
octets***." (My attempt at emphasis.)
http://www.loc.gov/marc/specifications/specrecstruc.html#genrec

And if you have enough items you reach that limit in size.

I have this same problem with one of the records i tried to import for
my client #1 - my plan is to try and transform the record to MARCXML,
which does not impose arbitrary limits on the size of a record, and
import that with the "-m MARCXML" option for bulkmarcimport.pl.

Regards,
Magnus Enger
libriotech.no
  

_______________________________________________
Koha-devel mailing list
Koha-devel@...
http://lists.koha.org/mailman/listinfo/koha-devel