« Return to Thread: hadoop and local files

hadoop and local files

by jerrro :: Rate this Message:

Reply to Author | View in Thread

Hello,

When launching a map-reduce job, I am interested in copying a certain file to the datanodes, but not HDFS - the local file system, so I can access that file from my job on the datanode. (The file is around 500KB, so I don't think there will be much overhead). Is there a way to tell hadoop to do that (I heard it is possible, but not sure how)? Also, how do I know where the file is copied to? (I understood it can be copied to /tmp or something of that sort of the datanode).

Thanks.



Jerr.

 « Return to Thread: hadoop and local files