[jira] Created: (HADOOP-2189) Incrementing user counters should count as progress

Incrementing user counters should count as progress
---------------------------------------------------

                 Key: HADOOP-2189
                 URL: https://issues.apache.org/jira/browse/HADOOP-2189
             Project: Hadoop
          Issue Type: Bug
          Components: mapred
            Reporter: Owen O'Malley
             Fix For: 0.16.0


An application that is incrementing the counters is making progress and should get credit for it and not be killed after 10 minutes.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12541606 ]

Doug Cutting commented on HADOOP-2189:
--------------------------------------

Did it work this way in 0.14?  It was implied on the user list that things used to work this way and then were changed in 0,14.  If that's the case, this is a regression and should be included in 0.15.1, no?

> Incrementing user counters should count as progress
> ---------------------------------------------------
>
>                 Key: HADOOP-2189
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2189
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Owen O'Malley
>             Fix For: 0.16.0
>
>
> An application that is incrementing the counters is making progress and should get credit for it and not be killed after 10 minutes.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12541705 ]

Arun C Murthy commented on HADOOP-2189:
---------------------------------------

I don't this is a regression.

 HADOOP-1462 fixed some bugs in the reporting framework, that might have hidden this bug then...

> Incrementing user counters should count as progress
> ---------------------------------------------------
>
>                 Key: HADOOP-2189
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2189
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Owen O'Malley
>             Fix For: 0.16.0
>
>
> An application that is incrementing the counters is making progress and should get credit for it and not be killed after 10 minutes.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/HADOOP-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Devaraj Das updated HADOOP-2189:
--------------------------------


Raising the priority of this since it is a regression for some apps that (correctly) assumes incrCounter to be an indicator of progress made.

> Incrementing user counters should count as progress
> ---------------------------------------------------
>
>                 Key: HADOOP-2189
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2189
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Owen O'Malley
>             Fix For: 0.16.0
>
>
> An application that is incrementing the counters is making progress and should get credit for it and not be killed after 10 minutes.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/HADOOP-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Devaraj Das reassigned HADOOP-2189:
-----------------------------------

    Assignee: Amar Kamat

> Incrementing user counters should count as progress
> ---------------------------------------------------
>
>                 Key: HADOOP-2189
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2189
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Owen O'Malley
>            Assignee: Amar Kamat
>            Priority: Blocker
>             Fix For: 0.16.0
>
>
> An application that is incrementing the counters is making progress and should get credit for it and not be killed after 10 minutes.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/HADOOP-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Devaraj Das updated HADOOP-2189:
--------------------------------

    Priority: Blocker  (was: Major)

> Incrementing user counters should count as progress
> ---------------------------------------------------
>
>                 Key: HADOOP-2189
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2189
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Owen O'Malley
>            Priority: Blocker
>             Fix For: 0.16.0
>
>
> An application that is incrementing the counters is making progress and should get credit for it and not be killed after 10 minutes.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12560495#action_12560495 ]

Nigel Daley commented on HADOOP-2189:
-------------------------------------

I don't see why this is a blocker.  It was filed over 2 months ago and never got fixed in a 0.15.x release.  Arun commented in Nov that it wasn't a regression.  Please justify or downgrade.  I've added a 0.16.1 release if you want to assign it there or get a patch in today.

> Incrementing user counters should count as progress
> ---------------------------------------------------
>
>                 Key: HADOOP-2189
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2189
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Owen O'Malley
>            Assignee: Amar Kamat
>            Priority: Blocker
>             Fix For: 0.16.0
>
>
> An application that is incrementing the counters is making progress and should get credit for it and not be killed after 10 minutes.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/HADOOP-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Devaraj Das updated HADOOP-2189:
--------------------------------

    Attachment: 2189.patch

Ok, here is the patch.

> Incrementing user counters should count as progress
> ---------------------------------------------------
>
>                 Key: HADOOP-2189
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2189
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Owen O'Malley
>            Assignee: Amar Kamat
>            Priority: Blocker
>             Fix For: 0.16.0
>
>         Attachments: 2189.patch
>
>
> An application that is incrementing the counters is making progress and should get credit for it and not be killed after 10 minutes.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/HADOOP-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Devaraj Das updated HADOOP-2189:
--------------------------------

    Status: Patch Available  (was: Open)

> Incrementing user counters should count as progress
> ---------------------------------------------------
>
>                 Key: HADOOP-2189
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2189
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Owen O'Malley
>            Assignee: Devaraj Das
>            Priority: Blocker
>             Fix For: 0.16.0
>
>         Attachments: 2189.patch
>
>
> An application that is incrementing the counters is making progress and should get credit for it and not be killed after 10 minutes.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/HADOOP-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Devaraj Das reassigned HADOOP-2189:
-----------------------------------

    Assignee: Devaraj Das  (was: Amar Kamat)

> Incrementing user counters should count as progress
> ---------------------------------------------------
>
>                 Key: HADOOP-2189
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2189
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Owen O'Malley
>            Assignee: Devaraj Das
>            Priority: Blocker
>             Fix For: 0.16.0
>
>         Attachments: 2189.patch
>
>
> An application that is incrementing the counters is making progress and should get credit for it and not be killed after 10 minutes.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12560723#action_12560723 ]

Hadoop QA commented on HADOOP-2189:
-----------------------------------

+1 overall.  Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12373548/2189.patch
against trunk revision r613359.

    @author +1.  The patch does not contain any @author tags.

    javadoc +1.  The javadoc tool did not generate any warning messages.

    javac +1.  The applied patch does not generate any new compiler warnings.

    findbugs +1.  The patch does not introduce any new Findbugs warnings.

    core tests +1.  The patch passed core unit tests.

    contrib tests +1.  The patch passed contrib unit tests.

Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1653/testReport/
Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1653/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1653/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1653/console

This message is automatically generated.

> Incrementing user counters should count as progress
> ---------------------------------------------------
>
>                 Key: HADOOP-2189
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2189
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Owen O'Malley
>            Assignee: Devaraj Das
>            Priority: Blocker
>             Fix For: 0.16.0
>
>         Attachments: 2189.patch
>
>
> An application that is incrementing the counters is making progress and should get credit for it and not be killed after 10 minutes.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/HADOOP-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Devaraj Das updated HADOOP-2189:
--------------------------------

    Resolution: Fixed
        Status: Resolved  (was: Patch Available)

I just committed this.

> Incrementing user counters should count as progress
> ---------------------------------------------------
>
>                 Key: HADOOP-2189
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2189
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Owen O'Malley
>            Assignee: Devaraj Das
>            Priority: Blocker
>             Fix For: 0.16.0
>
>         Attachments: 2189.patch
>
>
> An application that is incrementing the counters is making progress and should get credit for it and not be killed after 10 minutes.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12561708#action_12561708 ]

Hudson commented on HADOOP-2189:
--------------------------------

Integrated in Hadoop-Nightly #374 (See [http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/374/])

> Incrementing user counters should count as progress
> ---------------------------------------------------
>
>                 Key: HADOOP-2189
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2189
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Owen O'Malley
>            Assignee: Devaraj Das
>            Priority: Blocker
>             Fix For: 0.16.0
>
>         Attachments: 2189.patch
>
>
> An application that is incrementing the counters is making progress and should get credit for it and not be killed after 10 minutes.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.