Performance regression loading from fixtures in 1.1 & trunk

[David Turner-7]

I have created a test that does nothing but read in a fixture of 10000
items.  
http://novalis.org/bugs/perf.zip

It goes significantly slower on Django 1.1-beta and trunk than it does
in Django 1.0.

1.0: Ran 1 test in 16.904s
1.1: Ran 1 test in 21.541s
trunk: Ran 1 test in 21.228s

My rejected patch in #11181 reduces each of these times to about 11
seconds.

If deserializer.base used force_insert (without my patch), it would cut
loading times down to about 6 seconds.  




--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-developers@...
To unsubscribe from this group, send email to django-developers+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---

[Russell Keith-Magee-2]

On Fri, Jul 10, 2009 at 5:00 AM, David Turner<novalis@...> wrote:
>
> I have created a test that does nothing but read in a fixture of 10000
> items.
> http://novalis.org/bugs/perf.zip

I'd love to help out here, but it is 142 bytes in size, and when I
unpack it, all I get is an empty directory.

> It goes significantly slower on Django 1.1-beta and trunk than it does
> in Django 1.0.
>
> 1.0: Ran 1 test in 16.904s
> 1.1: Ran 1 test in 21.541s
> trunk: Ran 1 test in 21.228s
>
> My rejected patch in #11181 reduces each of these times to about 11
> seconds.

My surprise matches that of Honza and Alex in the ticket comments. I'd
need to see proof of this.

> If deserializer.base used force_insert (without my patch), it would cut
> loading times down to about 6 seconds.

This sounds like it could be a workable solution.

On a procedural note - I get the impression that you're frustrated
with the fact that we're not accepting your patches, but my response
would be that you're not making this easy for us.

 * You have presented a patch for #11181 that doesn't obviously speed
up anything (as two long term contributors have commented). However,
you haven't provided any evidence - such as a test case - that proves
the conditions under which your patch provides a speed improvement.
 * You have opened up #11449, but haven't responded to the request on
the ticket for a test case.
 * I'm guessing the project you mention in this email is the test case
that should be attached to #11449 - but I can't download the test case
(ok - this is easily a technical glitch - but Django's ticket tracker
allows for file uploads for exactly this reason).
 * You haven't mentioned the potential force_insert patch in the
discussion for #11449.
 * You haven't mentioned the existence of #11449 in this email.

Obviously, we would love to fix any serious performance regression.
However, you have to help us help you. Splitting discussions over
multiple sources doesn't help. Not providing all the details at your
disposal doesn't help. Not responding to comments on tickets doesn't
help.

If you can provide evidence of your claims, w'd love to fix them, but
we're not going to run around trying to put together the pieces of a
puzzle that you have scattered in multiple locations.

Yours,
Russ Magee %-)

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-developers@...
To unsubscribe from this group, send email to django-developers+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---

[David Turner-7]

On Sat, 2009-07-11 at 23:00 +0800, Russell Keith-Magee wrote:
> On Fri, Jul 10, 2009 at 5:00 AM, David Turner<novalis@...> wrote:
> >
> > I have created a test that does nothing but read in a fixture of 10000
> > items.
> > http://novalis.org/bugs/perf.zip
>
> I'd love to help out here, but it is 142 bytes in size, and when I
> unpack it, all I get is an empty directory.

Sorry -- I didn't notice that zip is nonrecursive by default.   I have
uploaded a fixed version.  The README explains how to generate a fixture
and then run the test.  My timings are for the test alone.

> > If deserializer.base used force_insert (without my patch), it would cut
> > loading times down to about 6 seconds.
>
> This sounds like it could be a workable solution.
>
> On a procedural note - I get the impression that you're frustrated
> with the fact that we're not accepting your patches, but my response
> would be that you're not making this easy for us.

Sorry I screwed this submission up.  I had initially intended to submit
a ticket, then thought a mailing list thread would be a better idea.  I
was closing out my tabs in preparation for going out of town, and I must
have thought I meant to submit the ticket.  Please disregard that ticket.

Also, sorry for the long delay between submitting 11181 and sending this
email -- it took a while to get enough time together to create a valid
test case.

Let's leave the tickets aside for now, and we'll see if you guys can
reproduce my performance issue.  If so, we can talk about what (if
anything) to do about it.

Again, sorry for sending the same thing to multiple places -- I know
that's a pain in the butt.


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-developers@...
To unsubscribe from this group, send email to django-developers+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---