[jira] Created: (HADOOP-2178) Job history on HDFS

Job history on HDFS
-------------------

                 Key: HADOOP-2178
                 URL: https://issues.apache.org/jira/browse/HADOOP-2178
             Project: Hadoop
          Issue Type: Improvement
            Reporter: Amareshwari Sri Ramadasu
            Assignee: Amareshwari Sri Ramadasu
             Fix For: 0.16.0


This issue addresses the following items :

1.  Check for accuracy of job tracker history logs.

2.  After completion of the job, copy the JobHistory.log(Master index file) and the job history files to the DFS.

3. User can load the history with commands
bin/hadoop job -history <directory>
or
bin/hadoop job -history <jobid>
This will start a stand-alone jetty and load jsps

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/HADOOP-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Devaraj Das updated HADOOP-2178:
--------------------------------

    Component/s: mapred

> Job history on HDFS
> -------------------
>
>                 Key: HADOOP-2178
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2178
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Amareshwari Sri Ramadasu
>            Assignee: Amareshwari Sri Ramadasu
>             Fix For: 0.16.0
>
>
> This issue addresses the following items :
> 1.  Check for accuracy of job tracker history logs.
> 2.  After completion of the job, copy the JobHistory.log(Master index file) and the job history files to the DFS.
> 3. User can load the history with commands
> bin/hadoop job -history <directory>
> or
> bin/hadoop job -history <jobid>
> This will start a stand-alone jetty and load jsps

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12555417#action_12555417 ]

eric baldeschwieler commented on HADOOP-2178:
---------------------------------------------

Why copy files once a job is complete?  Why not just always write them directly to the HDFS (or any other URL the user configures)?

I'm not a real fan of hidden directories like these.  The user will not know of them and potentially will fill a lot of disk/name space with never viewed material.  I'd be much happier if job history were considered part of the output of a job, unless configured otherwise.  IE put it in the map-reduce output directory in a file or directories prefixed with an underscore.  So <output>/_jobHistory or perhaps <output>/_logs/history.

We added the convention that map-reduce ignores underscore prefixed files specifically to allow this use case...

This also reduces jobid/name confusion, since the history is directly associated with the job's output.

We could then provide an option to put it in another location if the user desires.

thoughts?

> Job history on HDFS
> -------------------
>
>                 Key: HADOOP-2178
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2178
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Amareshwari Sri Ramadasu
>            Assignee: Amareshwari Sri Ramadasu
>             Fix For: 0.16.0
>
>
> This issue addresses the following items :
> 1.  Check for accuracy of job tracker history logs.
> 2.  After completion of the job, copy the JobHistory.log(Master index file) and the job history files to the DFS.
> 3. User can load the history with commands
> bin/hadoop job -history <directory>
> or
> bin/hadoop job -history <jobid>
> This will start a stand-alone jetty and load jsps

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12555439#action_12555439 ]

Runping Qi commented on HADOOP-2178:
------------------------------------


The output data may be deleted anytime when it is no longer needed. The log data may be needed long after the output data is deleted.

> Job history on HDFS
> -------------------
>
>                 Key: HADOOP-2178
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2178
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Amareshwari Sri Ramadasu
>            Assignee: Amareshwari Sri Ramadasu
>             Fix For: 0.16.0
>
>
> This issue addresses the following items :
> 1.  Check for accuracy of job tracker history logs.
> 2.  After completion of the job, copy the JobHistory.log(Master index file) and the job history files to the DFS.
> 3. User can load the history with commands
> bin/hadoop job -history <directory>
> or
> bin/hadoop job -history <jobid>
> This will start a stand-alone jetty and load jsps

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12556625#action_12556625 ]

Devaraj Das commented on HADOOP-2178:
-------------------------------------

+1 on Eric's suggestion. The jobhistory viewer (web server with the job history related JSPs) can take the output directory as the input and populate the history datastructures. This can be on a per-user basis for now (e.g., bin/hadoop jobhistoryview -output <dir> .. ), and, in the future, we could make the viewer a centralized web-enabled server that anyone can use. Thoughts?

bq. The output data may be deleted anytime when it is no longer needed. The log data may be needed long after the output data is deleted.

As Eric suggested, this can be solved by allowing the user to move the files to some persistent location...

> Job history on HDFS
> -------------------
>
>                 Key: HADOOP-2178
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2178
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Amareshwari Sri Ramadasu
>            Assignee: Amareshwari Sri Ramadasu
>             Fix For: 0.16.0
>
>
> This issue addresses the following items :
> 1.  Check for accuracy of job tracker history logs.
> 2.  After completion of the job, copy the JobHistory.log(Master index file) and the job history files to the DFS.
> 3. User can load the history with commands
> bin/hadoop job -history <directory>
> or
> bin/hadoop job -history <jobid>
> This will start a stand-alone jetty and load jsps

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12556630#action_12556630 ]

Runping Qi commented on HADOOP-2178:
------------------------------------


BTW, Is this issue the same as H-1876
(https://issues.apache.org/jira/browse/HADOOP-1876) ?

> Job history on HDFS
> -------------------
>
>                 Key: HADOOP-2178
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2178
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Amareshwari Sri Ramadasu
>            Assignee: Amareshwari Sri Ramadasu
>             Fix For: 0.16.0
>
>
> This issue addresses the following items :
> 1.  Check for accuracy of job tracker history logs.
> 2.  After completion of the job, copy the JobHistory.log(Master index file) and the job history files to the DFS.
> 3. User can load the history with commands
> bin/hadoop job -history <directory>
> or
> bin/hadoop job -history <jobid>
> This will start a stand-alone jetty and load jsps

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12556876#action_12556876 ]

Amareshwari Sri Ramadasu commented on HADOOP-2178:
--------------------------------------------------

To address both the issues (HADOOP-1876 and this), we will store job history files in a centralized location like _{/logs/history/<jobtracker.hostname>_<user.name>_<jobid>}_ on DFS. This location will avoid name conficts among users and hostnames.
The job history viewer(centralized) can display available history files and user can select the job he wants to view from there.
For 1876, job status can be queried giving the jobid and hostname.

> Job history on HDFS
> -------------------
>
>                 Key: HADOOP-2178
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2178
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Amareshwari Sri Ramadasu
>            Assignee: Amareshwari Sri Ramadasu
>             Fix For: 0.16.0
>
>
> This issue addresses the following items :
> 1.  Check for accuracy of job tracker history logs.
> 2.  After completion of the job, copy the JobHistory.log(Master index file) and the job history files to the DFS.
> 3. User can load the history with commands
> bin/hadoop job -history <directory>
> or
> bin/hadoop job -history <jobid>
> This will start a stand-alone jetty and load jsps

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12556876#action_12556876 ]

amareshwari edited comment on HADOOP-2178 at 1/8/08 4:30 AM:
--------------------------------------------------------------------------

To address both the issues (HADOOP-1876 and this), we will store job history files in a centralized location like _{/logs/history/<jobtracker.hostname>_<user.name>_<jobid>}_ on DFS. This location will avoid name conficts among users and hostnames.
The job history viewer(centralized) can display available history files sorted based on the creation time on the dfs and user can select the job he wants to view from there.

For 1876, job status can be queried giving the jobid, hostname and username. The hostname can be inferred and the username if not specified will be ignored and the first occurence of the jobid string taken.


      was (Author: amareshwari):
    To address both the issues (HADOOP-1876 and this), we will store job history files in a centralized location like _{/logs/history/<jobtracker.hostname>_<user.name>_<jobid>}_ on DFS. This location will avoid name conficts among users and hostnames.
The job history viewer(centralized) can display available history files and user can select the job he wants to view from there.
For 1876, job status can be queried giving the jobid and hostname.
 

> Job history on HDFS
> -------------------
>
>                 Key: HADOOP-2178
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2178
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Amareshwari Sri Ramadasu
>            Assignee: Amareshwari Sri Ramadasu
>             Fix For: 0.16.0
>
>
> This issue addresses the following items :
> 1.  Check for accuracy of job tracker history logs.
> 2.  After completion of the job, copy the JobHistory.log(Master index file) and the job history files to the DFS.
> 3. User can load the history with commands
> bin/hadoop job -history <directory>
> or
> bin/hadoop job -history <jobid>
> This will start a stand-alone jetty and load jsps

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12556907#action_12556907 ]

eric baldeschwieler commented on HADOOP-2178:
---------------------------------------------

I don't think it makes sense to address HADOOP-1876 by storing history on HDFS by default.

I think it is better to keep the server logs on the local machine by default and allow a configuration to store them in a well specified place on HDFS.  This will keep a HOD user from creating leavings all over the HDFS, but allow for alternate uses like HADOOP-1876.

We have 3 use cases we need to think about separately:

1) A cluster with one permanent JT.  This should have a single well know place it logs.

2) A HOD JT, this should send logs to a user specified directory

3) Central logging of job summary data that works in either of the above cases.  We should create a distinct JIRA to discuss this.

> Job history on HDFS
> -------------------
>
>                 Key: HADOOP-2178
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2178
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Amareshwari Sri Ramadasu
>            Assignee: Amareshwari Sri Ramadasu
>             Fix For: 0.16.0
>
>
> This issue addresses the following items :
> 1.  Check for accuracy of job tracker history logs.
> 2.  After completion of the job, copy the JobHistory.log(Master index file) and the job history files to the DFS.
> 3. User can load the history with commands
> bin/hadoop job -history <directory>
> or
> bin/hadoop job -history <jobid>
> This will start a stand-alone jetty and load jsps

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12556908#action_12556908 ]

Devaraj Das commented on HADOOP-2178:
-------------------------------------

Makes sense i think. Keeps things simple..

> Job history on HDFS
> -------------------
>
>                 Key: HADOOP-2178
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2178
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Amareshwari Sri Ramadasu
>            Assignee: Amareshwari Sri Ramadasu
>             Fix For: 0.16.0
>
>
> This issue addresses the following items :
> 1.  Check for accuracy of job tracker history logs.
> 2.  After completion of the job, copy the JobHistory.log(Master index file) and the job history files to the DFS.
> 3. User can load the history with commands
> bin/hadoop job -history <directory>
> or
> bin/hadoop job -history <jobid>
> This will start a stand-alone jetty and load jsps

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12556914#action_12556914 ]

Runping Qi commented on HADOOP-2178:
------------------------------------

+1.

This is much clearer.

Need coordination in implementing this and HADOOP-1876.

> Job history on HDFS
> -------------------
>
>                 Key: HADOOP-2178
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2178
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Amareshwari Sri Ramadasu
>            Assignee: Amareshwari Sri Ramadasu
>             Fix For: 0.16.0
>
>
> This issue addresses the following items :
> 1.  Check for accuracy of job tracker history logs.
> 2.  After completion of the job, copy the JobHistory.log(Master index file) and the job history files to the DFS.
> 3. User can load the history with commands
> bin/hadoop job -history <directory>
> or
> bin/hadoop job -history <jobid>
> This will start a stand-alone jetty and load jsps

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12558010#action_12558010 ]

Amareshwari Sri Ramadasu commented on HADOOP-2178:
--------------------------------------------------

To address use cases 1 and 2 suggested by Eric,  I propose the following approach.

If the job tracker is static, we will store history logs in a location specified by hadoop.job.history.location, by default it is local file system.
If the job tracker is not static (like HOD JT) we will store log files in user specified location, by default it is job output directory.

We will not have index file any more, because appending becomes an issue in DFS. And we dont need one in case of non-static JT.
For static JT, We can do listing of files in the log directory to show the first page.

> Job history on HDFS
> -------------------
>
>                 Key: HADOOP-2178
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2178
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Amareshwari Sri Ramadasu
>            Assignee: Amareshwari Sri Ramadasu
>             Fix For: 0.16.0
>
>
> This issue addresses the following items :
> 1.  Check for accuracy of job tracker history logs.
> 2.  After completion of the job, copy the JobHistory.log(Master index file) and the job history files to the DFS.
> 3. User can load the history with commands
> bin/hadoop job -history <directory>
> or
> bin/hadoop job -history <jobid>
> This will start a stand-alone jetty and load jsps

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12558029#action_12558029 ]

Runping Qi commented on HADOOP-2178:
------------------------------------


Even with hod JT, we still need to address case 3. That is,
we need a central place to store the job history files so that they can be analyzed offline.
It can be the same place in the local file system as it is now, or some common directory in DFS.
This is in addition to the one in the output directory.

> Job history on HDFS
> -------------------
>
>                 Key: HADOOP-2178
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2178
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Amareshwari Sri Ramadasu
>            Assignee: Amareshwari Sri Ramadasu
>             Fix For: 0.16.0
>
>
> This issue addresses the following items :
> 1.  Check for accuracy of job tracker history logs.
> 2.  After completion of the job, copy the JobHistory.log(Master index file) and the job history files to the DFS.
> 3. User can load the history with commands
> bin/hadoop job -history <directory>
> or
> bin/hadoop job -history <jobid>
> This will start a stand-alone jetty and load jsps

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12558712#action_12558712 ]

eric baldeschwieler commented on HADOOP-2178:
---------------------------------------------

The user vs admin dimension is not really the same as the static vs HOD dimension.

Even in a static JT, the job history is probably usefully part of the output directory (for a user).  I think this should just be part of the output API, no matter how the cluster is configured.  This will be much easier to document and use.

Then we could handle static job trackers as Amareshwari describes.  In the case of HOD deployed JTs, I think we can then either set hadoop.job.history.location to NULL or to a HOD specified output directory, probably on HDFS or another shared FS.  This would be useful if the user is running a lot jobs through a single HOD instance.  NULL is probably a fine default.

---

I agree with runping that we need to define an API for collecting central stats from HOD deployed JTs.  I think a configured URL is ok as an API, but we need to be clear that this output will be for central collection, not user diagnostics and as such the layout should be optimized to simplify that (probably time sorted, not user sorted for example).  Ideally this could be a single file per JT instance.

> Job history on HDFS
> -------------------
>
>                 Key: HADOOP-2178
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2178
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Amareshwari Sri Ramadasu
>            Assignee: Amareshwari Sri Ramadasu
>             Fix For: 0.16.0
>
>
> This issue addresses the following items :
> 1.  Check for accuracy of job tracker history logs.
> 2.  After completion of the job, copy the JobHistory.log(Master index file) and the job history files to the DFS.
> 3. User can load the history with commands
> bin/hadoop job -history <directory>
> or
> bin/hadoop job -history <jobid>
> This will start a stand-alone jetty and load jsps

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12559356#action_12559356 ]

Amareshwari Sri Ramadasu commented on HADOOP-2178:
--------------------------------------------------

bq.In the case of HOD deployed JTs, I think we can then either set hadoop.job.history.location to NULL or to a HOD specified output directory, probably on HDFS or another shared FS. This would be useful if the user is running a lot jobs through a single HOD instance. NULL is probably a fine default.

In case of HOD, hadoop.job.history.location default value can be local file system, sothat HOD can collect logs at the shutdown of cluster as is today.

> Job history on HDFS
> -------------------
>
>                 Key: HADOOP-2178
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2178
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Amareshwari Sri Ramadasu
>            Assignee: Amareshwari Sri Ramadasu
>             Fix For: 0.16.0
>
>
> This issue addresses the following items :
> 1.  Check for accuracy of job tracker history logs.
> 2.  After completion of the job, copy the JobHistory.log(Master index file) and the job history files to the DFS.
> 3. User can load the history with commands
> bin/hadoop job -history <directory>
> or
> bin/hadoop job -history <jobid>
> This will start a stand-alone jetty and load jsps

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/HADOOP-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Amareshwari Sri Ramadasu updated HADOOP-2178:
---------------------------------------------

    Attachment: patch-2178.txt

> Job history on HDFS
> -------------------
>
>                 Key: HADOOP-2178
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2178
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Amareshwari Sri Ramadasu
>            Assignee: Amareshwari Sri Ramadasu
>             Fix For: 0.16.0
>
>         Attachments: patch-2178.txt
>
>
> This issue addresses the following items :
> 1.  Check for accuracy of job tracker history logs.
> 2.  After completion of the job, copy the JobHistory.log(Master index file) and the job history files to the DFS.
> 3. User can load the history with commands
> bin/hadoop job -history <directory>
> or
> bin/hadoop job -history <jobid>
> This will start a stand-alone jetty and load jsps

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/HADOOP-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Amareshwari Sri Ramadasu updated HADOOP-2178:
---------------------------------------------

    Status: Patch Available  (was: Open)

> Job history on HDFS
> -------------------
>
>                 Key: HADOOP-2178
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2178
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Amareshwari Sri Ramadasu
>            Assignee: Amareshwari Sri Ramadasu
>             Fix For: 0.16.0
>
>         Attachments: patch-2178.txt
>
>
> This issue addresses the following items :
> 1.  Check for accuracy of job tracker history logs.
> 2.  After completion of the job, copy the JobHistory.log(Master index file) and the job history files to the DFS.
> 3. User can load the history with commands
> bin/hadoop job -history <directory>
> or
> bin/hadoop job -history <jobid>
> This will start a stand-alone jetty and load jsps

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

    [ https://issues.apache.org/jira/browse/HADOOP-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12559867#action_12559867 ]

Amareshwari Sri Ramadasu commented on HADOOP-2178:
--------------------------------------------------

This patch logs history in hadoop.job.history.location whose default value is local file system.
In addition to that it logs at hadoop.job.history.user.location whose default value isuser output directory.

And now the history jsp files are in seperate webapps i.e src/webapps/history, ratherthan in src/webapps/job .
We start a statushttpserver from job tracker with hadoop.job.history.location as history log directory and view history files. This page is linked from job tracker.
User can view history files in output directory by giving the following command
bin/hadoop job -history <outputdir>
 This will start a jetty on that machine and user can view the jsps of the history directory he has given.

Now we have centralised location of history as "hadoop.job.history.location" .

> Job history on HDFS
> -------------------
>
>                 Key: HADOOP-2178
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2178
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Amareshwari Sri Ramadasu
>            Assignee: Amareshwari Sri Ramadasu
>             Fix For: 0.16.0
>
>         Attachments: patch-2178.txt
>
>
> This issue addresses the following items :
> 1.  Check for accuracy of job tracker history logs.
> 2.  After completion of the job, copy the JobHistory.log(Master index file) and the job history files to the DFS.
> 3. User can load the history with commands
> bin/hadoop job -history <directory>
> or
> bin/hadoop job -history <jobid>
> This will start a stand-alone jetty and load jsps

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/HADOOP-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Amareshwari Sri Ramadasu updated HADOOP-2178:
---------------------------------------------

    Attachment: patch-2178.txt

> Job history on HDFS
> -------------------
>
>                 Key: HADOOP-2178
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2178
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Amareshwari Sri Ramadasu
>            Assignee: Amareshwari Sri Ramadasu
>             Fix For: 0.16.0
>
>         Attachments: patch-2178.txt
>
>
> This issue addresses the following items :
> 1.  Check for accuracy of job tracker history logs.
> 2.  After completion of the job, copy the JobHistory.log(Master index file) and the job history files to the DFS.
> 3. User can load the history with commands
> bin/hadoop job -history <directory>
> or
> bin/hadoop job -history <jobid>
> This will start a stand-alone jetty and load jsps

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

     [ https://issues.apache.org/jira/browse/HADOOP-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Amareshwari Sri Ramadasu updated HADOOP-2178:
---------------------------------------------

    Status: Patch Available  (was: Open)

try hudson with the correct patch.

> Job history on HDFS
> -------------------
>
>                 Key: HADOOP-2178
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2178
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Amareshwari Sri Ramadasu
>            Assignee: Amareshwari Sri Ramadasu
>             Fix For: 0.16.0
>
>         Attachments: patch-2178.txt
>
>
> This issue addresses the following items :
> 1.  Check for accuracy of job tracker history logs.
> 2.  After completion of the job, copy the JobHistory.log(Master index file) and the job history files to the DFS.
> 3. User can load the history with commands
> bin/hadoop job -history <directory>
> or
> bin/hadoop job -history <jobid>
> This will start a stand-alone jetty and load jsps

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.