[jira] Created: (HADOOP-2284) BasicTypeSorterBase.compare calls progress on each compare

    [ https://issues.apache.org/jira/browse/HADOOP-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12560665#action_12560665 ]

Amar Kamat commented on HADOOP-2284:
------------------------------------

The tests that have failed now are {{TestCrcCorruption}}, {{TestFsck}} and {{TestLogRolling}} but passed in the last run.

> BasicTypeSorterBase.compare calls progress on each compare
> ----------------------------------------------------------
>
>                 Key: HADOOP-2284
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2284
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Owen O'Malley
>            Assignee: Amar Kamat
>             Fix For: 0.16.0
>
>         Attachments: HADOOP-2284.patch
>
>
> The inner loop of the sort is calling progress on each compare. I think it would make more sense to call progress in the sort rather than the compare or at most every 10000 compares. In the performance numbers, the call to progress as part of the sort are consuming 12% of the total cpu time when running word count under the local runner.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/HADOOP-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Devaraj Das updated HADOOP-2284:
--------------------------------

    Resolution: Fixed
        Status: Resolved  (was: Patch Available)

I just committed this. Thanks, Amar!

> BasicTypeSorterBase.compare calls progress on each compare
> ----------------------------------------------------------
>
>                 Key: HADOOP-2284
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2284
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Owen O'Malley
>            Assignee: Amar Kamat
>             Fix For: 0.16.0
>
>         Attachments: HADOOP-2284.patch
>
>
> The inner loop of the sort is calling progress on each compare. I think it would make more sense to call progress in the sort rather than the compare or at most every 10000 compares. In the performance numbers, the call to progress as part of the sort are consuming 12% of the total cpu time when running word count under the local runner.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12561711#action_12561711 ]

Hudson commented on HADOOP-2284:
--------------------------------

Integrated in Hadoop-Nightly #374 (See [http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/374/])

> BasicTypeSorterBase.compare calls progress on each compare
> ----------------------------------------------------------
>
>                 Key: HADOOP-2284
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2284
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Owen O'Malley
>            Assignee: Amar Kamat
>             Fix For: 0.16.0
>
>         Attachments: HADOOP-2284.patch
>
>
> The inner loop of the sort is calling progress on each compare. I think it would make more sense to call progress in the sort rather than the compare or at most every 10000 compares. In the performance numbers, the call to progress as part of the sort are consuming 12% of the total cpu time when running word count under the local runner.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.