Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

I am a just data analyst hails from non cs background (not a hardcore system programmer) working on linux OS. While exploiting analysis using hadoop, a question was coming to my mind that ... / is a root under which all files of your system exists in a hierarchical manner. In hadoop envirorment, a special file system called as HDFS is there which is actually meant to store huge files to be processed by hadoop programming frameworks.

hadoop -fs put localfile.txt 

Although, such files should be accessible under /. so where can i see such files by using cat, less or more linux commands without prefixing hadoop -fs

If, unfortunately i get some error in hadoop/HDFS environment, then how can i acess my data which is still residing on my linux machine.

share|improve this question

2 Answers 2

Hadoop stores it data locally in forms of block on each datanode and that property is configurable in hdfs-site.xml file under dfs.data.dir property

In most of the case it is

$HADOOP_HOME/data/dfs/data/hadoop-${user.name}/current
share|improve this answer

You cannot directly browse HDFS from terminal using cat or similar commands. HDFS is a logical file system and does not directly map to Unix file system. You should have an HDFS client and your Hadoop cluster should be running. When you browse HDFS, you are getting your directory structure from namenode and actual data from datanodes.

Although you cannot browse, data is there stored by datanode daemon. Its path is specified by dfs.data.dir property in hdfs-site.xml.

Directory structure is stored by namenode daemon and its path is specified by dfs.name.dir property in hdfs-site.xml

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.