Saving MARC records very slow

[Ondrej Mlecka]

I'd like to ask one question. When I am saving MARC records in Staff Client, after clicking the butto "Save", normally it takes 10-30 seconds in which the web browser is frozen and seems tdoing nothing. Afterwards it saves the record.

When I do the same operation directly on the server, it is much faster - it takes only 3-4 seconds. But I can't let cataloguers work on server.

Would anybody know what could be the cause and how could I fasten it?

Thank you

Ondrej Mlecka

_______________________________________________
Koha mailing list
Koha@...
http://lists.katipo.co.nz/mailman/listinfo/koha

[Cab Vinton]

This was covered on the list a while back -- in Firefox it was taking
40-50 s on our staff clients to save records.

We've switched to Google Chrome & the time is now much more manageable.

I don't believe a diagnosis or fix were ever found for the Firefox issue.

Cab Vinton, Director
Sanbornton Public Library
Sanbornton, NH


2009/6/29 Ondrej Mlecka <omlecka@...>:

> I'd like to ask one question. When I am saving MARC records in Staff Client,
> after clicking the butto "Save", normally it takes 10-30 seconds in which
> the web browser is frozen and seems tdoing nothing. Afterwards it saves the
> record.
>
> When I do the same operation directly on the server, it is much faster - it
> takes only 3-4 seconds. But I can't let cataloguers work on server.
>
> Would anybody know what could be the cause and how could I fasten it?
>
> Thank you
>
> Ondrej Mlecka
>
> _______________________________________________
> Koha mailing list
> Koha@...
> http://lists.katipo.co.nz/mailman/listinfo/koha
>
>

_______________________________________________
Koha mailing list
Koha@...
http://lists.katipo.co.nz/mailman/listinfo/koha

[Owen Leonard-4]

> This was covered on the list a while back -- in Firefox it was taking
> 40-50 s on our staff clients to save records.
>
> I don't believe a diagnosis or fix were ever found for the Firefox issue.

It's not a Firefox issue per se, it's a JavaScript issue. When you
save a MARC record Koha uses JavaScript to validate all the fields in
the entry form--usually a lot of them! That takes time, and makes the
browser appear unresponsive while it's processing. Chrome will be
faster than Firefox 3 because its JavaScript parser is faster. But
Firefox will be faster than Internet Explorer.

So the solution isn't a browser-specific fix, but a general one of optimization.

  -- Owen

--
Web Developer
Athens County Public Libraries
http://www.myacpl.org
_______________________________________________
Koha mailing list
Koha@...
http://lists.katipo.co.nz/mailman/listinfo/koha

[Nicole Engard]

Has anyone tried Firefox 3.5?  It's still in beta, but is supposed to
have a much better response time.  Just curious :)

---

Nicole C. Engard
Open Source Evangelist, LibLime
(888) Koha ILS (564-2457) ext. 714
nce@...
AIM/Y!/Skype: nengard

http://liblime.com
http://blogs.liblime.com/open-sesame/



On Mon, Jun 29, 2009 at 7:50 AM, Owen Leonard<oleonard@...> wrote:

>> This was covered on the list a while back -- in Firefox it was taking
>> 40-50 s on our staff clients to save records.
>>
>> I don't believe a diagnosis or fix were ever found for the Firefox issue.
>
> It's not a Firefox issue per se, it's a JavaScript issue. When you
> save a MARC record Koha uses JavaScript to validate all the fields in
> the entry form--usually a lot of them! That takes time, and makes the
> browser appear unresponsive while it's processing. Chrome will be
> faster than Firefox 3 because its JavaScript parser is faster. But
> Firefox will be faster than Internet Explorer.
>
> So the solution isn't a browser-specific fix, but a general one of optimization.
>
>  -- Owen
>
> --
> Web Developer
> Athens County Public Libraries
> http://www.myacpl.org
> _______________________________________________
> Koha mailing list
> Koha@...
> http://lists.katipo.co.nz/mailman/listinfo/koha
>

_______________________________________________
Koha mailing list
Koha@...
http://lists.katipo.co.nz/mailman/listinfo/koha

[Owen Leonard-4]

> Has anyone tried Firefox 3.5?  It's still in beta, but is supposed to
> have a much better response time.  Just curious :)

It would be interesting to find out, from a technical point of view.
But as much as I'd like to try, mandating a particular browser--or
suggesting its use as a bug-fix--just isn't possible.

  -- Owen

--
Web Developer
Athens County Public Libraries
http://www.myacpl.org
_______________________________________________
Koha mailing list
Koha@...
http://lists.katipo.co.nz/mailman/listinfo/koha

[Rick Welykochy]

Cab Vinton wrote:

> This was covered on the list a while back -- in Firefox it was taking
> 40-50 s on our staff clients to save records.
>
> We've switched to Google Chrome & the time is now much more manageable.
>
> I don't believe a diagnosis or fix were ever found for the Firefox issue.

As pointed out, this is because of Javascript.

Question: can Koha operate properly with JS disabled? If not, why not?

Observation: JS is quite slow compared the server itself. Perhaps the
*HUGE* task of validating a MARC save could be (optionally) left to
the server to validate.


cheers
rickw



--
_________________________________
Rick Welykochy || Praxis Services

Debra Jackson says she likes shopping at the Dollar Palace because it's
convenient and casual. "I don't have to get dressed up like I'm going
to Wal-mart or something," she said.      -- spotted in a newspaper
_______________________________________________
Koha mailing list
Koha@...
http://lists.katipo.co.nz/mailman/listinfo/koha

[Joe Atzberger]

I find it totally undesirable to make or maintain any changes in the cataloging module for the purpose of going HTML-only.   Your sense of server-side processing advantage may or may not translate to an advantage in actual environment constraints.   Consider that adding a repeatable field would have to constitute a new request and response for each element, parsing the marcxl each time, aggregating form state changes in some yet undefined table.  The server might validate the end results faster, but getting there just became 30 times as expensive and you still need to round trip to the server to get the results.

I'd wager that exactly zero of our clients would prefer scriptless cataloguing, and would likewise be upset for Koha to sacrifice scripted features for it.

Cataloging is already a huge amount of logic. Duplicating the plugins to operate cleanly as server-side validators would be a major undertaking in itself, and be more likely to introduce runtime warnings and fatals.

Sent from my phone,b/c my power is out!
--joe

On Jun 29, 2009 7:53 PM, "Rick Welykochy" <rick@...> wrote:

Cab Vinton wrote: > This was covered on the list a while back -- in Firefox it was taking > 40-50 s...

As pointed out, this is because of Javascript.

Question: can Koha operate properly with JS disabled? If not, why not?

Observation: JS is quite slow compared the server itself. Perhaps the
*HUGE* task of validating a MARC save could be (optionally) left to
the server to validate.


cheers
rickw



--
_________________________________
Rick Welykochy || Praxis Services

Debra Jackson says she likes shopping at the Dollar Palace because it's
convenient and casual. "I don't have to get dressed up like I'm going
to Wal-mart or something," she said.      -- spotted in a newspaper

_______________________________________________ Koha mailing list Koha@... http://lis...

_______________________________________________
Koha mailing list
Koha@...
http://lists.katipo.co.nz/mailman/listinfo/koha

[Cab Vinton]

I know nothing about the programming side, but if Chrome has figured
out how to speed up JavaScript, then surely it won't be long before
the other browsers are forced to catch up.

Firefox 3.5 is being advertised along these lines, so it will be
interesting to see if they've been successful in catching up. Explorer
8 unfortunately still seems to lag behind.

Cab Vinton, Director
Sanbornton Public Library
Sanbornton, NH
_______________________________________________
Koha mailing list
Koha@...
http://lists.katipo.co.nz/mailman/listinfo/koha

[Stefano Bargioni]

Another browser that runs Javascript very fast is Safari 4 (Mac and  
Win). Chrome and Safari use the same Javascript interpreter, if I'm  
not wrong.
Anyway, IMHO the best solution is to detect bottlenecks in opensource  
and multiplatform browsers.
Firefox with Firebug extension can help to test and trace.
sb

On Jun 30, 2009, at 11:57 , Cab Vinton wrote:

> I know nothing about the programming side, but if Chrome has figured
> out how to speed up JavaScript, then surely it won't be long before
> the other browsers are forced to catch up.
>
> Firefox 3.5 is being advertised along these lines, so it will be
> interesting to see if they've been successful in catching up. Explorer
> 8 unfortunately still seems to lag behind.
>
> Cab Vinton, Director
> Sanbornton Public Library
> Sanbornton, NH
> _______________________________________________
> Koha mailing list
> Koha@...
> http://lists.katipo.co.nz/mailman/listinfo/koha
>

_______________________________________________
Koha mailing list
Koha@...
http://lists.katipo.co.nz/mailman/listinfo/koha

[BWS Johnson]

Salvete!


>I find it totally undesirable to make or maintain any changes in the cataloging module for the purpose of
>going HTML-only.


Forgive me if I'm wrong, but my sense was that a change or changes would be made with paucity of code and processing time in mind. It was an optimisation suggestion.


>
Your sense of server-side processing advantage may or may not translate to an
>advantage in actual environment constraints. Consider that adding a repeatable field would have to
>constitute a new request and response for each element, parsing the marcxl each time, aggregating form
>state changes in some yet undefined table. The server might validate the end results faster, but getting
>there just became 30 times as expensive and you still need to round trip to the server to get the results.
>

Is that 30 times rooted in fact? Also, from what is being described, it seems as though the long wait for the record's finalisation would be traded for a few shorter waits upon edit. Do I have that right Rick?


>I'd wager that exactly zero of our clients would prefer scriptless cataloguing, and would likewise be upset
>for Koha to sacrifice scripted features for it.


I'd reckon none of your clients would care if the job were done, and a whole bunch would be happier were the job done quicker. I don't think anyone was calling for a full sack of JS, just suggesting a well intended tinker to see if we can slim down an ugly just under a minute wait. Have you tried this your way Rick? I'd love to see side by side optimised versions to see what the pitfalls of both were.


>Cataloging is already a huge amount of logic. Duplicating the plugins to operate cleanly as server-side
>validators would be a major undertaking in itself, and be more likely to introduce runtime warnings and
>fatals.


True, but I'm always all ears about making cataloguing better. What we have now is rather undesirable. I don't think anyone hinted that tinkering with or improving cataloguing was easy. :)
For pros, (and it's been a while since I looked, so maybe this is implemented, but I don't think so) would it save querying time if there were a simple text box where one edited an entire MARC compliant or single original record? I didn't have a text box for every field with III, just a big old text box with a record. It seems to me as though this would shift the processing burden to the server, as well, and probably save time in the end. I realise the initial outlay of coding time would be substantial, but the friendliness to experienced users would most likely be worth it. Usability would be oh so much better, too. (I still need to find the right star to wish on for guided cataloguing for non pros :) )

Cheers,
Brooke


_______________________________________________
Koha mailing list
Koha@...
http://lists.katipo.co.nz/mailman/listinfo/koha

[Emrys Minnig]

I had similar problems with saving MARC records with FF, though it started out fast and then progressively slowed down as I catalogued more and more.  I found that if I cleared all private data (tools menu), things sped up again.
 
Emrys

________________________________

From: koha-bounces@... on behalf of Ondrej Mlecka
Sent: Mon 29/06/2009 10:04
To: koha@...
Subject: [Koha] Saving MARC records very slow


I'd like to ask one question. When I am saving MARC records in Staff Client, after clicking the butto "Save", normally it takes 10-30 seconds in which the web browser is frozen and seems tdoing nothing. Afterwards it saves the record.

When I do the same operation directly on the server, it is much faster - it takes only 3-4 seconds. But I can't let cataloguers work on server.

Would anybody know what could be the cause and how could I fasten it?

Thank you

Ondrej Mlecka

_______________________________________________
Koha mailing list
Koha@...
http://lists.katipo.co.nz/mailman/listinfo/koha

[Beda Szukics-2]

Hi

Cab Vinton schrieb:
> This was covered on the list a while back -- in Firefox it was taking
> 40-50 s on our staff clients to save records.
>
> We've switched to Google Chrome & the time is now much more manageable.
>
> I don't believe a diagnosis or fix were ever found for the Firefox issue.

Update to Firefox 3.0.11!

The times are much better :-) (I have no idea why :-()

It is strange that only the windows-version of firefox is affected (I
tested version 2.0.0.14 on my eeepc and it worked fine, i.e. 7-8 seconds
for the saving).


Greetings           Beda
_______________________________________________
Koha mailing list
Koha@...
http://lists.katipo.co.nz/mailman/listinfo/koha

[Owen Leonard-4]

> Forgive me if I'm wrong, but my sense was that a change or changes would be
> made with paucity of code and processing time in mind. It was an
> optimisation suggestion.

The actual question was "Question: can Koha operate properly with JS
disabled? If not, why not?" Depending on the poser, that question can
either be genuine and neutral or it can be leading and accusatory.
Posers, you know who you are ;)

> Is that 30 times rooted in fact? Also, from what is being described, it
> seems as though the long wait for the record's finalisation would be traded
> for a few shorter waits upon edit. Do I have that right Rick?

I think "30 times" should be considered an estimate. It depends on how
much you're editing your record. How many times might one:

- add a tag
- delete a tag
- add a subfield
- delete a subfield
- use any plugin

What am I leaving out? Instead of having one longer wait at the end,
you're having 30 (or whatever) waits while you're performing any one
of these actions.

> I'd reckon none of your clients would care if the job were done

And unfortunately that means none of them would sponsor it.

> I don't think anyone was
> calling for a full sack of JS

I know some do!

 -- Owen

--
Web Developer
Athens County Public Libraries
http://www.myacpl.org
_______________________________________________
Koha mailing list
Koha@...
http://lists.katipo.co.nz/mailman/listinfo/koha

[Lenora Oftedahl]

UM, I use Firefox and I don't have a problem with time for saving records (a few seconds at most).  And I do a lot of them as I'm still cleaning up our switch from InMagic.  

Yes, I'm on 3.00 (not sure exactly which)

Lenora

Lenora A. Oftedahl
StreamNet Regional Librarian
Columbia River Inter-Tribal Fish Commission
http://www.fishlib.org
 
 


_______________________________________________
Koha mailing list
Koha@...
http://lists.katipo.co.nz/mailman/listinfo/koha

[Rick Welykochy]

BWS Johnson wrote:

>  >there just became 30 times as expensive and you still need to round
> trip to the server to get the results.
>  >
>
> Is that 30 times rooted in fact? Also, from what is being described, it
> seems as though the long wait for the record's finalisation would be
> traded for a few shorter waits upon edit. Do I have that right Rick?

Correct.

It is disingenuous to whine about return trips to the server when
Koha is already littered with such in the form of Ajax calls, which
sometimes for each keystroke invoke a return trip to the server.

Example: some of the Koha searches update a textbox / dropdown
control with new search possibilities each time you hit a key on
the keyboard.


cheers
rickw


--
_________________________________
Rick Welykochy || Praxis Services

Debra Jackson says she likes shopping at the Dollar Palace because it's
convenient and casual. "I don't have to get dressed up like I'm going
to Wal-mart or something," she said.      -- spotted in a newspaper
_______________________________________________
Koha mailing list
Koha@...
http://lists.katipo.co.nz/mailman/listinfo/koha

[Joe Atzberger]

>> there just became 30 times as expensive and you still need to round
>> trip to the server to get the results.
>
> Is that 30 times rooted in fact?

It isn't implemented, so it is hypothetical, but there isn't any way around *every* operation requiring a round trip to the server w/o js.  Do some original cataloging, count how many times you change tabs, search for an authorized value, and add/remove fields. 
 

It is disingenuous to whine about return trips to the server when

Koha is already littered with such in the form of Ajax calls, which
sometimes for each keystroke invoke a return trip to the server.

Uh, no.  It's legit.  Especially on the staff side, Koha would benefit from *more* ajax, not less.  When the logic of a webapp can be broken into ajax-able atomic pieces, then each of those can execute and reply with very small updates without having to reparse everything else on the page that isn't changing. 

As a simple (hypothetical) example, building a list gets really expensive if you have to read out the growing list AND all the other elements of the page to the client for every item loaded.  With ajax you can get confirmation from server as terse as one line.  The non-ajax method doesn't scale.  It gets slower as the list gets bigger until the point where it is too slow to use.  The ajax method can maintain the same performance at the 1000th request as the first, because it doesn't have to care about the last 999. 

Example: some of the Koha searches update a textbox / dropdown control with new search possibilities each time you hit a key on the keyboard.

Those YUI autocomplete calls aren't parsing MARCXML into MARC::Record objects like the cataloging editor must, and they can already be disabled by syspref.  YUI autocomplete performs reasonably well for us and the thousands of other sites that use the package.  And, lastly, ajax has nothing to do with the cataloging editor anyway.  It doesn't use it. 

You are comparing two clearly different entities.  HTML-only cataloging with anything approaching the current features would require repeated expensive MARC parsing, aggregating the cataloger's manipulations in an unspecified location, all the existing permissions checking for toolbars and HTML::Template::Pro building, etc.  That looks nothing like optional industry standard AJAX autocomplete that hits ysearch.pl, querying one table and running in 60 lines.

--
Joe Atzberger
LibLime - Open Source Library Solutions


_______________________________________________
Koha mailing list
Koha@...
http://lists.katipo.co.nz/mailman/listinfo/koha