Hadoop Streaming - running a jar file

[Amit_Gupta]

Hi

I have a jar file which takes input from stdin and writes something on stdout. i.e. When I run

java -jar A.jar < input

It prints the required output.

However, when I run it as a mapper in hadoop streaming using the command

$HADOOP_HOME/bin/hadoop jar ....streaming.jar -input .. -output ...  -mapper 'java -jar A.jar'  -reducer NONE

i get the broken pipe exception.


the error message is

additionalConfSpec_:null
null=@@@userJobConfProps_.get(stream.shipped.hadoopstreaming
packageJobJar: [/mnt/hadoop/HADOOP/hadoop-0.16.3/tmp/dir/hadoop-hadoop/hadoop-unjar45410/] [] /tmp/streamjob45411.jar tmpDir=null
08/11/11 23:20:14 INFO mapred.FileInputFormat: Total input paths to process : 1
08/11/11 23:20:14 INFO streaming.StreamJob: getLocalDirs(): [/mnt/hadoop/HADOOP/hadoop-0.16.3/tmp/mapred]
08/11/11 23:20:14 INFO streaming.StreamJob: Running job: job_200811111724_0014
08/11/11 23:20:14 INFO streaming.StreamJob: To kill this job, run:
08/11/11 23:20:14 INFO streaming.StreamJob: /mnt/hadoop/HADOOP/hadoop-0.16.3/bin/../bin/hadoop job  -Dmapred.job.tracker=10.105.41.25:54311 -kill job_200811111724_0014
08/11/11 23:20:15 INFO streaming.StreamJob: Tracking URL: http://sayali:50030/jobdetails.jsp?jobid=job_200811111724_0014
08/11/11 23:20:16 INFO streaming.StreamJob:  map 0%  reduce 0%
08/11/11 23:21:00 INFO streaming.StreamJob:  map 100%  reduce 100%
08/11/11 23:21:00 INFO streaming.StreamJob: To kill this job, run:
08/11/11 23:21:00 INFO streaming.StreamJob: /mnt/hadoop/HADOOP/hadoop-0.16.3/bin/../bin/hadoop job  -Dmapred.job.tracker=10.105.41.25:54311 -kill job_200811111724_0014
08/11/11 23:21:00 INFO streaming.StreamJob: Tracking URL: http://sayali:50030/jobdetails.jsp?jobid=job_200811111724_0014
08/11/11 23:21:00 ERROR streaming.StreamJob: Job not Successful!
08/11/11 23:21:00 INFO streaming.StreamJob: killJob...
Streaming Job Failed!

Could some one please help me with any ideas or pointers.

regards
Amit