Questions tagged [hadoop]

Apache Hadoop is a software framework that supports data-intensive distributed applications under a free license. It enables applications to work with thousands of computational independent computers and petabytes of data. Hadoop was derived from Google's MapReduce and Google File System (GFS) papers.

Filter by
Sorted by
Tagged with
0
votes
1answer
74 views

Error while running jar file

I used Ubuntu 18.04.4 in my IBM virtual machine and I got an error while running my jar file in Ubuntu terminal. I've already installed Java. Exception in thread "main" java.lang.NoClassDefFoundError:...
0
votes
0answers
18 views

Want to connect two virtual machine (host windows) running on different laptops (using different networks)

I want to build a hadoop cluster with one master node and 2 slave nodes. The master node will be running on a virtual box of pc-1. Slave nodes will be running on virtual box of pc-2. Slaves on same ...
0
votes
1answer
21 views

Unable to start single node cluster of Hadoop

I am trying to install hadoop 2.9.1 on ubuntu 19.10 and I did all steps explained in this video https://www.youtube.com/watch?v=Y6oit3rCsZo The problem is that when I triyed to start single node ...
2
votes
1answer
46 views

What is the conventional directory to install Hadoop into?

Sooo... Hadoop doesn't have an installer, so can't use apt for it... Still pretty new at ubuntu stuff... I realize I can install it anywhere, but is there an established convention to be followed for ...
0
votes
0answers
14 views

Hadoop input folder not found?

I am new to hadoop and trying installing hadoop-3.2.1. I can start the following process and can see dfshealth.html#tab-datanode. But I am stuck at " copy the input files into the distributed file ...
0
votes
0answers
36 views

NoNode for HBase master pseudodistributed mode

To me it seems like my hadoop and HBase are not configured correctly to interact. When I through the HBase shell try to create a table it yields me with following error ERROR: KeeperErrorCode = ...
0
votes
0answers
15 views

hadoop related storage problem in Ubuntu 19.10

I had installed hadoop and I am facing low memory error on Hadoop installation. From past 2-3 months I am trying to solve this problem. I had installed hadoop on a Ubuntu virtual machine but I am ...
0
votes
0answers
46 views

Connect Multiple nodes for hadoop via ssh on ubuntu

We are trying to set-up a two-nodes hadoop clusters on two different machines. SSH to localhost without password have been setted up on both machines. And hadoop can run well as single node on each ...
0
votes
0answers
25 views

$ hdfs envvars is giving error

I've installed Hadoop and default openjdk and set the path succesfully. Running the command: $ hdfs envvars is giving the error: "Could not find or load main class envvars. Caused by: java.lang....
-2
votes
2answers
110 views

Error during ssh server installation: connect to host localhost port 22: Connection refused

I am trying to install openssh-server. Then at the end I am getting the error: ssh.service - OpenBSD Secure Shell server Loaded: loaded (/lib/systemd/system/ssh.service; enabled; vendor preset: ...
0
votes
0answers
40 views

hadoop Installation Fails on Local Repository

Ambari cant able to find packages in the local repository for hadoop installation. I have tried everything but it fails at 4%. I browsed for the packages inside the local repositories using the ...
0
votes
1answer
39 views

dealing low memory in hadoop 3.1.3 in vmware virtual machine ubuntu 19.10

I installed Hadoop on a Ubuntu virtual machine. I downloaded the source file that means among the releases available on this page https://hadoop.apache.org/releases.html I downloaded 3.1.3 file https:...
0
votes
1answer
87 views

hadoop installation errors in ubuntu 19.10

I was installing Hadoop on Ubuntu 19.10 I got following errors I am comipiling https://hadoop.apache.org/releases.html file version https://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-3.1.3/...
0
votes
0answers
112 views

curl: (35) gnutls_handshake() failed: The TLS connection was non-properly terminated

I had some problem installing Hadoop so connection broke in between and so there were some errors I am pasting them please see them and suggest me what should I do Step 10/36 : RUN update-java-...
0
votes
0answers
67 views

gpg: hadoop-3.1.3.tar.gz.asc: encryption failed: No public key

I downloaded Hadoop-3.1.3 from http://mirrors.estointernet.in/apache/hadoop/common/hadoop-3.1.3/hadoop-3.1.3.tar.gz as mentioned here https://hadoop.apache.org/releases.html as per instructions on ...
0
votes
0answers
171 views

how to install java version 1.8 and hadoop and scala in Ubuntu 19.10

How can I install java in Ubuntu 19.10 On my windows 10 machine I have following java -version java version "1.8.0_161" Java(TM) SE Runtime Environment (build 1.8.0_161-b12) Java HotSpot(TM) 64-Bit ...
1
vote
0answers
74 views

Unable to connect to remote host: Connection refused

I try run hadoop: hadoop jar hadoop-examples.jar pi 4 100 and get error: java.net.ConnectionException: ... to localhost:9000 failed on connection exception... Connection refused. When I run: ...
0
votes
1answer
32 views

Give permission to hdf to remove directory

When I run hdf namenode -format then it wants to remove a directory /home/hadoop/hadooptmpdata/dfs/name/current but it has no permission to do it. How to give it a permission?
1
vote
2answers
290 views

I am unable to edit gedit ~/.bashrc in ubuntu please help how to do

I am installing hadoop in ubuntu 19 and I am trying to edit bashrc file in gedit but unable to edit gedit bashrc file please help me how to do. This my hduser dir: hdusr@suri-VirtualBox when I ...
-1
votes
1answer
519 views

ssh: Could not resolve hostname

i try to input this command: start-dfs.sh and the output like this: Starting namenodes on [HadoopMaster] HadoopMaster: ssh: Could not resolve hostname hadoopmaster: Name or service not known what ...
1
vote
0answers
195 views

su {user} not working inside the ssh shell on the server

I am trying to connect to the server as root using ssh command in Ubuntu bash. I use ssh -p port root@ip to connect to the server and the config of ssh is a password-less connection ssh with the ...
0
votes
1answer
48 views

Jdk or headless version ? Which should I use for Hadoop installation

Which version of java should I use for Hadoop installation ? Jdk or headless version ?
1
vote
0answers
96 views

Installed Hadoop in fully distributed mode and unable to run datanode daemon on one of the datanodes

I created 3 datanodes in hadoop , fully distributed mode. and accidentally used 'hdfs namenode -format' in one of the data nodes and i get this error: 172.31.17.100: ERROR: Cannot set priority of ...
0
votes
2answers
3k views

Hadoop giving error “ls: Failed on local exception: org.apache.hadoop.ipc.RpcException: RPC response exceeds maximum data length”

Folks, I've literally searched everywhere, spent ounces and ounces of time. Can you please help me here? I have Java 8 and Hadoop 2.9.1 in my Ubuntu 14.04. After I type start-all.sh, services do get ...
1
vote
1answer
1k views

SSH without password only works one time

I want to configure SSH to use it without the need of writing the password. I am using Ubuntu 18.04 LTS on Windows 10. I need it to run Hadoop 3.1.1 (https://hadoop.apache.org/docs/r3.1.1/hadoop-...
2
votes
1answer
22k views

FAILED: HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

I'm trying to run Hive 3.1 with Hadoop 3.0. Following is my system configuration: Ubuntu 18.04.1 LTS Hadoop version 3.0.3 Hive 3.1.0 Derby 10.14.2 When I execute the show tables; query I get the ...
1
vote
1answer
3k views

Could not resolve hostname datanode1: Name or service not known

I am building an SSH connection with my namenode and two datanodes on AWS. When I try to copy the config & my .pem file from namenode to datanodes, I get the following error: Could not resolve ...
1
vote
1answer
2k views

Configuring JAVA HOME for Hadoop 3.0.3 when having multiple Java versions

I'm working on a Hadoop installation done by someone else. Ubuntu 18.04.1 LTS Hadoop version 3.0.3 Java open jdk 11 (auto mode) and 8 (manual mode) Currently Hadoop runs with Java 11 but I want to ...
0
votes
1answer
2k views

Copy File from HDFS to Local File System based on timestamp Using Shell Script

I want to move n number of file residing in the Hadoop directory to Local File system based on timestamp. Scenario: suppose I am doing the first time transition from HDFS to the Local file system, so ...
0
votes
1answer
842 views

Sqoop import from MySQL to Hive throwing HIVE_CONF_DIR error

I am trying to load my MySQL tableLab5 to Hive using Sqoop. Sqoop ran successfully and has taken the data from mysql into hdfs. But throwing the below bolded error about HIVE_CONF_DIR not set. I have ...
0
votes
1answer
352 views

A JNI error has occurred, in HIVE 3.0.0

I installed Hadoop successfully and it's working fine, but when I installed Hive and ran the hive command in terminal, I got this error Error: A JNI error has occurred, please check your installation ...
1
vote
0answers
31 views

Hadoop stopped working after installing bdsgames package

I have hadoop 2.x installed. I typed this command and entered password: sudo apt-get bdsgames Since then my hadoop commands have not been working. How can I resolve this?
1
vote
0answers
940 views

HDF5 Installation Error in the latest Ubuntu 18.04LTS with Intel Compilers

While I used HDF5 library without any problem with Ubuntu version 16.04 LTS, after I upgraded Ubuntu to Ubuntu 18.04 LTS, I have trouble in installing HDF5 with Intel Fortran and C Compilers. I ...
0
votes
1answer
1k views

Cannot start hadoop “localhost: nice: cannot set niceness”

Starting namenodes on [localhost] localhost: starting namenode, logging to /opt/hadoop-2.9.0/logs/hadoop-ha-namenode-root.out localhost: nice: cannot set niceness: Permission denied localhost: ...
0
votes
1answer
1k views

hdfs: split ls output

on hdfs, the output of ls command looks like this: drwxrwxr-x 5 matteorr matteorr 4096 Jan 10 17:37 /data/Cluster drwxr-xr-x 2 matteorr matteorr 4096 Jan 19 10:43 /data/Desktop drwxrwxr-x 9 ...
0
votes
1answer
2k views

Cannot open display error when I run sudo gedit on Ubuntu 14.04.5

solitude@Solitude:~/Desktop$ sudo gedit ~/.bashrc error: XDG_RUNTIME_DIR not set in the environment. (gedit:12631): Gtk-WARNING **: cannot open display:
0
votes
0answers
86 views

Entire file system permissions were changed on Ubuntu 16.04 LTS

I'm running hadoop in my machine with hduser. When I was trying to user 'put' and 'get' commands it was throwing that "permission denied". hduser@salmon-son-of-god:/usr/local/hadoop-2.6.5/bin$ hadoop ...
2
votes
1answer
196 views

What is a good way to create a low cost Ubuntu based Cassandra or Hadoop Cluster in my house?

I have a series of "old" data center servers (that I bought when my company upgraded) that I use to run Cassandra and Hadoop clusters at home. They are loud, power hungry and take up a lot of space ...
1
vote
1answer
3k views

Error while copying file to HDFS in hadoop ecosystem

While issuing a command to copy file from Local File system to HDFS on terminal in Hadoop 3.0, it shows error hadoop-3.0.0/hadoop2_data/hdfs/datanode': No such file or directory: `hdfs://localhost:...
1
vote
1answer
578 views

hadoop namenode -format showing error

I am trying to setup Hadoop 3.0.0 on my Ubuntu desktop. I have done all the required setups, but when running the command ./hadoop namenode -format to format name it shows error: root@DESKTOP-JIUFBOR:...
0
votes
1answer
428 views

Which version of Hadoop is compatible with Ubuntu 17.10?

Which version of Hadoop is compatible with Ubuntu 17.10 ?
0
votes
0answers
861 views

Ubuntu Hadoop-3.0.0 Can't set priority of all the nodes . Details in the picture

Both Master and Slave1 can't set priority of any nodes process ..... It's my last few steps to accomplish this cluster , thanks in advanced.
0
votes
0answers
127 views

Hadoop 3.0.0 in Ubuntu : Master node can't log in by command “ssh Master” details in the picture

I've been changed the set of /etc/ssh/sshd_config/, but it still goes: Permission Denied, please try again. And the command ssh localhost does work! I just can't figure out what's wrong with it!
1
vote
1answer
545 views

Unexpected Error: Error renaming temporary file, Text File Busy

I've been working on a Ubuntu and Hadoop, I'm at installing and configuring stage. For this I'm following a link: Hadoop Beginner When I am configuring the Hadoop so when I reach to gedit core-site....
0
votes
2answers
2k views

mapred-site.xml.template is missing in Hadoop 2.9.0

I'm a brand new user of this forum, Ubuntu, and Hadoop. I'm running Ubuntu on VirtualBox, and I've been following a tutorial for installing and configuring Ubuntu and Hadoop 2.7.3 (though I'm trying ...
2
votes
1answer
356 views

Installing Hadoop in Ubuntu 16.04

I am pretty new to Ubuntu/linux and totally new to Hadoop. I'm following the guide here: https://www.digitalocean.com/community/tutorials/how-to-install-hadoop-in-stand-alone-mode-on-ubuntu-16-04 I ...
0
votes
0answers
201 views

pig -version Shows error

deepak@deepak-Linux:~$ pig -version /usr/local/hadoop/bin/hadoop: line 166: /usr//bin/java: No such file or directory deepak@deepak-Linux:~$ echo $JAVA_HOME /usr/java/jdk-9.0.1 deepak@deepak-Linux:~$...
0
votes
2answers
1k views

Hbase doesn't recognize my java version

I'm trying to install Hbase with this tutorial and I stepped upon something curious : hbase doesn't recognize my java version whereas Ubuntu 16.04 does. mike@mike-thinks:~/hbase-1.2.6$ ./bin/start-...
0
votes
1answer
470 views

Is my Hadoop Data File System okay with an Incorrect configuration message?

I am trying to install hadoop on my own laptop for study purposes with the tutorial from tutorialspoint. I launched start-dfs.sh: The expected output was: 10/24/14 21:37:56 Starting namenodes on [...
0
votes
1answer
3k views

Starting Hadoop file system with `start-dfs.sh` permission denied

I try to install hbase one a standalone node but I have some issue to launch Hadoop file system with start-dfs.sh : mike@mike-thinks:/usr/local/hadoop/sbin$ ./start-dfs.sh Incorrect configuration: ...

1
2 3 4 5