Questions tagged [hadoop]

Apache Hadoop is a software framework that supports data-intensive distributed applications under a free license. It enables applications to work with thousands of computational independent computers and petabytes of data. Hadoop was derived from Google's MapReduce and Google File System (GFS) papers.

Filter by
Sorted by
Tagged with
0
votes
0answers
19 views

hdfs normally working but with sudo it shows command not found

I am trying to install hadoop3.1.1 and i have set ./bashrc file and all the other necessary files. while applying hdfs i am facing this situation shown in the picture. this image shows commands I have ...
0
votes
0answers
78 views

Error: Could not find or load main class org.apache.sqoop.Sqoop

Could not find or load main class org.apache.sqoop.Sqoop While installing sqoop-1.4.7.bin__hadoop-2.6.0 pseudo mode my bashrc is : export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 #export JAVA_HOME=/...
0
votes
0answers
15 views

running a jar file(java) on Hadoop yarn but it got some error “failing over to rm1 ” or “failing over to rm2 ”

I'm running a jar file on Hadoop yarn but it got some error "failing over to rm1 " or "failing over to rm2 " , what is the reason? and my application is stuck on accepted status. ...
0
votes
0answers
1k views

GVfs Metadata is not supported while Map Reduce Configuration

while executing this commond: sudo gedit /etc/profile.d/hadoop.sh Getting this error: (gedit:5376): Tepl-WARNING **: 16:32:59.581: GVfs metadata is not supported. Fallback to TeplMetadataManager. ...
0
votes
1answer
813 views

Error while running jar file

I used Ubuntu 18.04.4 in my IBM virtual machine and I got an error while running my jar file in Ubuntu terminal. I've already installed Java. Exception in thread "main" java.lang.NoClassDefFoundError:...
0
votes
0answers
21 views

Want to connect two virtual machine (host windows) running on different laptops (using different networks)

I want to build a hadoop cluster with one master node and 2 slave nodes. The master node will be running on a virtual box of pc-1. Slave nodes will be running on virtual box of pc-2. Slaves on same ...
0
votes
1answer
231 views

Unable to start single node cluster of Hadoop

I am trying to install hadoop 2.9.1 on ubuntu 19.10 and I did all steps explained in this video https://www.youtube.com/watch?v=Y6oit3rCsZo The problem is that when I triyed to start single node ...
2
votes
1answer
52 views

What is the conventional directory to install Hadoop into?

Sooo... Hadoop doesn't have an installer, so can't use apt for it... Still pretty new at ubuntu stuff... I realize I can install it anywhere, but is there an established convention to be followed for ...
1
vote
0answers
94 views

Connect Multiple nodes for hadoop via ssh on ubuntu

We are trying to set-up a two-nodes hadoop clusters on two different machines. SSH to localhost without password have been setted up on both machines. And hadoop can run well as single node on each ...
-3
votes
2answers
198 views

Error during ssh server installation: connect to host localhost port 22: Connection refused

I am trying to install openssh-server. Then at the end I am getting the error: ssh.service - OpenBSD Secure Shell server Loaded: loaded (/lib/systemd/system/ssh.service; enabled; vendor preset: ...
0
votes
1answer
72 views

dealing low memory in hadoop 3.1.3 in vmware virtual machine ubuntu 19.10

I installed Hadoop on a Ubuntu virtual machine. I downloaded the source file that means among the releases available on this page https://hadoop.apache.org/releases.html I downloaded 3.1.3 file https:...
1
vote
1answer
112 views

hadoop installation errors in ubuntu 19.10

I was installing Hadoop on Ubuntu 19.10 I got following errors I am comipiling https://hadoop.apache.org/releases.html file version https://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-3.1.3/...
1
vote
0answers
150 views

Unable to connect to remote host: Connection refused

I try run hadoop: hadoop jar hadoop-examples.jar pi 4 100 and get error: java.net.ConnectionException: ... to localhost:9000 failed on connection exception... Connection refused. When I run: ...
0
votes
1answer
53 views

Give permission to hdf to remove directory

When I run hdf namenode -format then it wants to remove a directory /home/hadoop/hadooptmpdata/dfs/name/current but it has no permission to do it. How to give it a permission?
2
votes
2answers
853 views

I am unable to edit gedit ~/.bashrc in ubuntu please help how to do

I am installing hadoop in ubuntu 19 and I am trying to edit bashrc file in gedit but unable to edit gedit bashrc file please help me how to do. This my hduser dir: hdusr@suri-VirtualBox when I ...
-1
votes
1answer
2k views

ssh: Could not resolve hostname

i try to input this command: start-dfs.sh and the output like this: Starting namenodes on [HadoopMaster] HadoopMaster: ssh: Could not resolve hostname hadoopmaster: Name or service not known what ...
1
vote
0answers
288 views

su {user} not working inside the ssh shell on the server

I am trying to connect to the server as root using ssh command in Ubuntu bash. I use ssh -p port root@ip to connect to the server and the config of ssh is a password-less connection ssh with the ...
1
vote
1answer
63 views

Jdk or headless version ? Which should I use for Hadoop installation

Which version of java should I use for Hadoop installation ? Jdk or headless version ?
1
vote
0answers
103 views

Installed Hadoop in fully distributed mode and unable to run datanode daemon on one of the datanodes

I created 3 datanodes in hadoop , fully distributed mode. and accidentally used 'hdfs namenode -format' in one of the data nodes and i get this error: 172.31.17.100: ERROR: Cannot set priority of ...
0
votes
2answers
4k views

Hadoop giving error "ls: Failed on local exception: org.apache.hadoop.ipc.RpcException: RPC response exceeds maximum data length"

Folks, I've literally searched everywhere, spent ounces and ounces of time. Can you please help me here? I have Java 8 and Hadoop 2.9.1 in my Ubuntu 14.04. After I type start-all.sh, services do get ...
1
vote
1answer
2k views

SSH without password only works one time

I want to configure SSH to use it without the need of writing the password. I am using Ubuntu 18.04 LTS on Windows 10. I need it to run Hadoop 3.1.1 (https://hadoop.apache.org/docs/r3.1.1/hadoop-...
3
votes
1answer
32k views

FAILED: HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

I'm trying to run Hive 3.1 with Hadoop 3.0. Following is my system configuration: Ubuntu 18.04.1 LTS Hadoop version 3.0.3 Hive 3.1.0 Derby 10.14.2 When I execute the show tables; query I get the ...
1
vote
1answer
6k views

Could not resolve hostname datanode1: Name or service not known

I am building an SSH connection with my namenode and two datanodes on AWS. When I try to copy the config & my .pem file from namenode to datanodes, I get the following error: Could not resolve ...
1
vote
1answer
3k views

Configuring JAVA HOME for Hadoop 3.0.3 when having multiple Java versions

I'm working on a Hadoop installation done by someone else. Ubuntu 18.04.1 LTS Hadoop version 3.0.3 Java open jdk 11 (auto mode) and 8 (manual mode) Currently Hadoop runs with Java 11 but I want to ...
0
votes
1answer
2k views

Copy File from HDFS to Local File System based on timestamp Using Shell Script

I want to move n number of file residing in the Hadoop directory to Local File system based on timestamp. Scenario: suppose I am doing the first time transition from HDFS to the Local file system, so ...
0
votes
1answer
1k views

Sqoop import from MySQL to Hive throwing HIVE_CONF_DIR error

I am trying to load my MySQL tableLab5 to Hive using Sqoop. Sqoop ran successfully and has taken the data from mysql into hdfs. But throwing the below bolded error about HIVE_CONF_DIR not set. I have ...
0
votes
1answer
424 views

A JNI error has occurred, in HIVE 3.0.0

I installed Hadoop successfully and it's working fine, but when I installed Hive and ran the hive command in terminal, I got this error Error: A JNI error has occurred, please check your installation ...
1
vote
0answers
36 views

Hadoop stopped working after installing bdsgames package

I have hadoop 2.x installed. I typed this command and entered password: sudo apt-get bdsgames Since then my hadoop commands have not been working. How can I resolve this?
1
vote
0answers
1k views

HDF5 Installation Error in the latest Ubuntu 18.04LTS with Intel Compilers

While I used HDF5 library without any problem with Ubuntu version 16.04 LTS, after I upgraded Ubuntu to Ubuntu 18.04 LTS, I have trouble in installing HDF5 with Intel Fortran and C Compilers. I ...
0
votes
1answer
1k views

Cannot start hadoop "localhost: nice: cannot set niceness"

Starting namenodes on [localhost] localhost: starting namenode, logging to /opt/hadoop-2.9.0/logs/hadoop-ha-namenode-root.out localhost: nice: cannot set niceness: Permission denied localhost: ...
0
votes
1answer
2k views

hdfs: split ls output

on hdfs, the output of ls command looks like this: drwxrwxr-x 5 matteorr matteorr 4096 Jan 10 17:37 /data/Cluster drwxr-xr-x 2 matteorr matteorr 4096 Jan 19 10:43 /data/Desktop drwxrwxr-x 9 ...
0
votes
1answer
3k views

Cannot open display error when I run sudo gedit on Ubuntu 14.04.5

solitude@Solitude:~/Desktop$ sudo gedit ~/.bashrc error: XDG_RUNTIME_DIR not set in the environment. (gedit:12631): Gtk-WARNING **: cannot open display:
0
votes
0answers
93 views

Entire file system permissions were changed on Ubuntu 16.04 LTS

I'm running hadoop in my machine with hduser. When I was trying to user 'put' and 'get' commands it was throwing that "permission denied". hduser@salmon-son-of-god:/usr/local/hadoop-2.6.5/bin$ hadoop ...
3
votes
1answer
218 views

What is a good way to create a low cost Ubuntu based Cassandra or Hadoop Cluster in my house?

I have a series of "old" data center servers (that I bought when my company upgraded) that I use to run Cassandra and Hadoop clusters at home. They are loud, power hungry and take up a lot of space ...
1
vote
1answer
4k views

Error while copying file to HDFS in hadoop ecosystem

While issuing a command to copy file from Local File system to HDFS on terminal in Hadoop 3.0, it shows error hadoop-3.0.0/hadoop2_data/hdfs/datanode': No such file or directory: `hdfs://localhost:...
1
vote
1answer
716 views

hadoop namenode -format showing error

I am trying to setup Hadoop 3.0.0 on my Ubuntu desktop. I have done all the required setups, but when running the command ./hadoop namenode -format to format name it shows error: root@DESKTOP-JIUFBOR:...
0
votes
1answer
473 views

Which version of Hadoop is compatible with Ubuntu 17.10?

Which version of Hadoop is compatible with Ubuntu 17.10 ?
1
vote
1answer
804 views

Unexpected Error: Error renaming temporary file, Text File Busy

I've been working on a Ubuntu and Hadoop, I'm at installing and configuring stage. For this I'm following a link: Hadoop Beginner When I am configuring the Hadoop so when I reach to gedit core-site....
0
votes
2answers
2k views

mapred-site.xml.template is missing in Hadoop 2.9.0

I'm a brand new user of this forum, Ubuntu, and Hadoop. I'm running Ubuntu on VirtualBox, and I've been following a tutorial for installing and configuring Ubuntu and Hadoop 2.7.3 (though I'm trying ...
2
votes
1answer
370 views

Installing Hadoop in Ubuntu 16.04

I am pretty new to Ubuntu/linux and totally new to Hadoop. I'm following the guide here: https://www.digitalocean.com/community/tutorials/how-to-install-hadoop-in-stand-alone-mode-on-ubuntu-16-04 I ...
0
votes
0answers
254 views

pig -version Shows error

deepak@deepak-Linux:~$ pig -version /usr/local/hadoop/bin/hadoop: line 166: /usr//bin/java: No such file or directory deepak@deepak-Linux:~$ echo $JAVA_HOME /usr/java/jdk-9.0.1 deepak@deepak-Linux:~$...
0
votes
2answers
2k views

Hbase doesn't recognize my java version

I'm trying to install Hbase with this tutorial and I stepped upon something curious : hbase doesn't recognize my java version whereas Ubuntu 16.04 does. mike@mike-thinks:~/hbase-1.2.6$ ./bin/start-...
0
votes
1answer
542 views

Is my Hadoop Data File System okay with an Incorrect configuration message?

I am trying to install hadoop on my own laptop for study purposes with the tutorial from tutorialspoint. I launched start-dfs.sh: The expected output was: 10/24/14 21:37:56 Starting namenodes on [...
0
votes
1answer
4k views

Starting Hadoop file system with `start-dfs.sh` permission denied

I try to install hbase one a standalone node but I have some issue to launch Hadoop file system with start-dfs.sh : mike@mike-thinks:/usr/local/hadoop/sbin$ ./start-dfs.sh Incorrect configuration: ...
0
votes
1answer
3k views

hdfs: command not found with Hadoop2.9.0

I am trying to install hadoop with the tutorial from tutorialspoint. I just sucessfully installed hadoop2.9.0. I then tried to set up the namenode using the command “hdfs namenode -format” as follows. ...
0
votes
1answer
670 views

Why don't I have a /usr/local on Ubuntu 16.04?

I followed a tutorial to install hadoop. I tried to move the folder to /usr/local/: sudo mv hadoop-2.9* /usr/local/hadoop but it answered me : mv: target '/usr/local/hadoop' is not a directory Isn'...
0
votes
1answer
214 views

Installing Spark on Hadoop 2.5

I'm running Hadoop on Ubuntu on a Oracle VM on Windows 7. I'm trying to install Spark on an existing Hadoop installation. For demonstration purposes I do not want to change the existing Hadoop version....
0
votes
1answer
597 views

Hadoop Name Node Formatting fail

I am using following tutorial to install Hadoop on my system and have followed all the steps uptil formatting the name node when the command hdfs namenode -format is giving me error: NFO util....
1
vote
1answer
2k views

hadoop command is not working

I am trying to install single node hadoop on my ubuntu 14.04 as per the documentation available at bogotobogo.com. after following all the steps when I tried to format the hadoop file system it shows ...
0
votes
1answer
174 views

Error Installing Bigtop - E: Sub-process /usr/bin/dpkg returned an error code (1)

I'm facing the following error when trying to install the Bigtop distribution of Hadoop. Errors were encountered while processing: /var/cache/apt/archives/oozie_4.3.0-1_all.deb E: Sub-process /usr/...

1
2 3 4 5