Questions tagged [hadoop]

Apache Hadoop is a software framework that supports data-intensive distributed applications under a free license. It enables applications to work with thousands of computational independent computers and petabytes of data. Hadoop was derived from Google's MapReduce and Google File System (GFS) papers.

Filter by
Sorted by
Tagged with
-1 votes
0 answers
10 views

bigtop hadoop start and stop scripts used to restart entire cluster

I don't find these below mentioned start and stop script files in bigtop hadoop-3.2.4 package distribution. start-dfs.sh start-yarn.sh stop-dfs.sh stop-yarn.sh start-all.sh stop-all.sh please let me ...
0 votes
1 answer
41 views

usage of mount -o remount,exec /tmp Hbase hadoop file system and solr

Can you please tell me what is main work of this command mount -o remount,exec /tmp I am using it when my hbase giving some errors but i don't know what is happening using this command. and i want ...
0 votes
0 answers
24 views

Namenode format error Ubuntu 20.04 WSL2

Hi i finished the hadoop installation using Ubuntu 20.04 when i finally put the hdfs namenode -format command i am getting the following error command 'hdfs' not found, did you mean: command 'hdfls' ...
  • 1
0 votes
0 answers
175 views

I just install the kafka using command line on Ubuntu 18.04 and trying to start the zookeeper server but unfortunately I am getting two errors given

bin/zookeeper-server-start.sh config/zookeeper/properties Could not rename log file '/home/hafiz/kafka/bin/../logs/zookeeper-gc.log' to '/home/hafiz/kafka/bin/../logs/zookeeper-gc.log.0' (Permission ...
0 votes
1 answer
3k views

mkdir: cannot create directory permission denied when launching hadoop

I get this error whenever I try to launch Hadoop. I have already configured the files needed to launch it but I get: $ hdfs namenode -format WARNING: /home/jeunesseafricaine/hadoop-3.3.1/logs does ...
-1 votes
1 answer
93 views

Why are Hadoop and Spark not in the official Ubuntu repositories?

UPDATE (2021-11-13 22:12 GMT+8): regarding the Snap packages, @karel suggested that this question is a duplicate of Why don't the Ubuntu repositories have the latest versions of software? I ...
0 votes
1 answer
112 views

hdfs normally working but with sudo it shows command not found

I am trying to install hadoop3.1.1 and i have set ./bashrc file and all the other necessary files. while applying hdfs i am facing this situation shown in the picture. this image shows commands I have ...
0 votes
0 answers
2k views

GVfs Metadata is not supported while Map Reduce Configuration

while executing this commond: sudo gedit /etc/profile.d/hadoop.sh Getting this error: (gedit:5376): Tepl-WARNING **: 16:32:59.581: GVfs metadata is not supported. Fallback to TeplMetadataManager. ...
0 votes
1 answer
1k views

Error while running jar file

I used Ubuntu 18.04.4 in my IBM virtual machine and I got an error while running my jar file in Ubuntu terminal. I've already installed Java. Exception in thread "main" java.lang.NoClassDefFoundError:...
0 votes
0 answers
27 views

Want to connect two virtual machine (host windows) running on different laptops (using different networks)

I want to build a hadoop cluster with one master node and 2 slave nodes. The master node will be running on a virtual box of pc-1. Slave nodes will be running on virtual box of pc-2. Slaves on same ...
0 votes
1 answer
492 views

Unable to start single node cluster of Hadoop

I am trying to install hadoop 2.9.1 on ubuntu 19.10 and I did all steps explained in this video https://www.youtube.com/watch?v=Y6oit3rCsZo The problem is that when I triyed to start single node ...
2 votes
2 answers
133 views

What is the conventional directory to install Hadoop into?

Sooo... Hadoop doesn't have an installer, so can't use apt for it... Still pretty new at ubuntu stuff... I realize I can install it anywhere, but is there an established convention to be followed for ...
  • 21
1 vote
0 answers
120 views

Connect Multiple nodes for hadoop via ssh on ubuntu

We are trying to set-up a two-nodes hadoop clusters on two different machines. SSH to localhost without password have been setted up on both machines. And hadoop can run well as single node on each ...
-3 votes
2 answers
348 views

Error during ssh server installation: connect to host localhost port 22: Connection refused

I am trying to install openssh-server. Then at the end I am getting the error: ssh.service - OpenBSD Secure Shell server Loaded: loaded (/lib/systemd/system/ssh.service; enabled; vendor preset: ...
  • 5
0 votes
1 answer
96 views

dealing low memory in hadoop 3.1.3 in vmware virtual machine ubuntu 19.10

I installed Hadoop on a Ubuntu virtual machine. I downloaded the source file that means among the releases available on this page https://hadoop.apache.org/releases.html I downloaded 3.1.3 file https:...
1 vote
1 answer
152 views

hadoop installation errors in ubuntu 19.10

I was installing Hadoop on Ubuntu 19.10 I got following errors I am comipiling https://hadoop.apache.org/releases.html file version https://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-3.1.3/...
1 vote
0 answers
232 views

Unable to connect to remote host: Connection refused

I try run hadoop: hadoop jar hadoop-examples.jar pi 4 100 and get error: java.net.ConnectionException: ... to localhost:9000 failed on connection exception... Connection refused. When I run: ...
0 votes
1 answer
65 views

Give permission to hdf to remove directory

When I run hdf namenode -format then it wants to remove a directory /home/hadoop/hadooptmpdata/dfs/name/current but it has no permission to do it. How to give it a permission?
2 votes
2 answers
2k views

I am unable to edit gedit ~/.bashrc in ubuntu please help how to do

I am installing hadoop in ubuntu 19 and I am trying to edit bashrc file in gedit but unable to edit gedit bashrc file please help me how to do. This my hduser dir: hdusr@suri-VirtualBox when I ...
-1 votes
1 answer
4k views

ssh: Could not resolve hostname

i try to input this command: start-dfs.sh and the output like this: Starting namenodes on [HadoopMaster] HadoopMaster: ssh: Could not resolve hostname hadoopmaster: Name or service not known what ...
1 vote
0 answers
325 views

su {user} not working inside the ssh shell on the server

I am trying to connect to the server as root using ssh command in Ubuntu bash. I use ssh -p port root@ip to connect to the server and the config of ssh is a password-less connection ssh with the ...
1 vote
1 answer
81 views

Jdk or headless version ? Which should I use for Hadoop installation

Which version of java should I use for Hadoop installation ? Jdk or headless version ?
1 vote
0 answers
112 views

Installed Hadoop in fully distributed mode and unable to run datanode daemon on one of the datanodes

I created 3 datanodes in hadoop , fully distributed mode. and accidentally used 'hdfs namenode -format' in one of the data nodes and i get this error: 172.31.17.100: ERROR: Cannot set priority of ...
0 votes
2 answers
4k views

Hadoop giving error "ls: Failed on local exception: org.apache.hadoop.ipc.RpcException: RPC response exceeds maximum data length"

Folks, I've literally searched everywhere, spent ounces and ounces of time. Can you please help me here? I have Java 8 and Hadoop 2.9.1 in my Ubuntu 14.04. After I type start-all.sh, services do get ...
1 vote
1 answer
2k views

SSH without password only works one time

I want to configure SSH to use it without the need of writing the password. I am using Ubuntu 18.04 LTS on Windows 10. I need it to run Hadoop 3.1.1 (https://hadoop.apache.org/docs/r3.1.1/hadoop-...
  • 111
4 votes
1 answer
37k views

FAILED: HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

I'm trying to run Hive 3.1 with Hadoop 3.0. Following is my system configuration: Ubuntu 18.04.1 LTS Hadoop version 3.0.3 Hive 3.1.0 Derby 10.14.2 When I execute the show tables; query I get the ...
1 vote
1 answer
10k views

Could not resolve hostname datanode1: Name or service not known

I am building an SSH connection with my namenode and two datanodes on AWS. When I try to copy the config & my .pem file from namenode to datanodes, I get the following error: Could not resolve ...
2 votes
1 answer
4k views

Configuring JAVA HOME for Hadoop 3.0.3 when having multiple Java versions

I'm working on a Hadoop installation done by someone else. Ubuntu 18.04.1 LTS Hadoop version 3.0.3 Java open jdk 11 (auto mode) and 8 (manual mode) Currently Hadoop runs with Java 11 but I want to ...
0 votes
1 answer
2k views

Copy File from HDFS to Local File System based on timestamp Using Shell Script

I want to move n number of file residing in the Hadoop directory to Local File system based on timestamp. Scenario: suppose I am doing the first time transition from HDFS to the Local file system, so ...
's user avatar
0 votes
1 answer
1k views

Sqoop import from MySQL to Hive throwing HIVE_CONF_DIR error

I am trying to load my MySQL tableLab5 to Hive using Sqoop. Sqoop ran successfully and has taken the data from mysql into hdfs. But throwing the below bolded error about HIVE_CONF_DIR not set. I have ...
0 votes
1 answer
480 views

A JNI error has occurred, in HIVE 3.0.0

I installed Hadoop successfully and it's working fine, but when I installed Hive and ran the hive command in terminal, I got this error Error: A JNI error has occurred, please check your installation ...
1 vote
0 answers
38 views

Hadoop stopped working after installing bdsgames package

I have hadoop 2.x installed. I typed this command and entered password: sudo apt-get bdsgames Since then my hadoop commands have not been working. How can I resolve this?
1 vote
0 answers
1k views

HDF5 Installation Error in the latest Ubuntu 18.04LTS with Intel Compilers

While I used HDF5 library without any problem with Ubuntu version 16.04 LTS, after I upgraded Ubuntu to Ubuntu 18.04 LTS, I have trouble in installing HDF5 with Intel Fortran and C Compilers. I ...
0 votes
1 answer
1k views

Cannot start hadoop "localhost: nice: cannot set niceness"

Starting namenodes on [localhost] localhost: starting namenode, logging to /opt/hadoop-2.9.0/logs/hadoop-ha-namenode-root.out localhost: nice: cannot set niceness: Permission denied localhost: ...
  • 1
0 votes
1 answer
2k views

hdfs: split ls output

on hdfs, the output of ls command looks like this: drwxrwxr-x 5 matteorr matteorr 4096 Jan 10 17:37 /data/Cluster drwxr-xr-x 2 matteorr matteorr 4096 Jan 19 10:43 /data/Desktop drwxrwxr-x 9 ...
  • 231
0 votes
1 answer
4k views

Cannot open display error when I run sudo gedit on Ubuntu 14.04.5

solitude@Solitude:~/Desktop$ sudo gedit ~/.bashrc error: XDG_RUNTIME_DIR not set in the environment. (gedit:12631): Gtk-WARNING **: cannot open display:
0 votes
0 answers
96 views

Entire file system permissions were changed on Ubuntu 16.04 LTS

I'm running hadoop in my machine with hduser. When I was trying to user 'put' and 'get' commands it was throwing that "permission denied". hduser@salmon-son-of-god:/usr/local/hadoop-2.6.5/bin$ hadoop ...
3 votes
1 answer
226 views

What is a good way to create a low cost Ubuntu based Cassandra or Hadoop Cluster in my house?

I have a series of "old" data center servers (that I bought when my company upgraded) that I use to run Cassandra and Hadoop clusters at home. They are loud, power hungry and take up a lot of space ...
1 vote
1 answer
6k views

Error while copying file to HDFS in hadoop ecosystem

While issuing a command to copy file from Local File system to HDFS on terminal in Hadoop 3.0, it shows error hadoop-3.0.0/hadoop2_data/hdfs/datanode': No such file or directory: `hdfs://localhost:...
1 vote
1 answer
855 views

hadoop namenode -format showing error

I am trying to setup Hadoop 3.0.0 on my Ubuntu desktop. I have done all the required setups, but when running the command ./hadoop namenode -format to format name it shows error: root@DESKTOP-JIUFBOR:...
0 votes
1 answer
504 views

Which version of Hadoop is compatible with Ubuntu 17.10?

Which version of Hadoop is compatible with Ubuntu 17.10 ?
1 vote
1 answer
998 views

Unexpected Error: Error renaming temporary file, Text File Busy

I've been working on a Ubuntu and Hadoop, I'm at installing and configuring stage. For this I'm following a link: Hadoop Beginner When I am configuring the Hadoop so when I reach to gedit core-site....
  • 11
0 votes
2 answers
2k views

mapred-site.xml.template is missing in Hadoop 2.9.0

I'm a brand new user of this forum, Ubuntu, and Hadoop. I'm running Ubuntu on VirtualBox, and I've been following a tutorial for installing and configuring Ubuntu and Hadoop 2.7.3 (though I'm trying ...
  • 11
2 votes
1 answer
389 views

Installing Hadoop in Ubuntu 16.04

I am pretty new to Ubuntu/linux and totally new to Hadoop. I'm following the guide here: https://www.digitalocean.com/community/tutorials/how-to-install-hadoop-in-stand-alone-mode-on-ubuntu-16-04 I ...
  • 395
0 votes
0 answers
277 views

pig -version Shows error

deepak@deepak-Linux:~$ pig -version /usr/local/hadoop/bin/hadoop: line 166: /usr//bin/java: No such file or directory deepak@deepak-Linux:~$ echo $JAVA_HOME /usr/java/jdk-9.0.1 deepak@deepak-Linux:~$...
0 votes
2 answers
2k views

Hbase doesn't recognize my java version

I'm trying to install Hbase with this tutorial and I stepped upon something curious : hbase doesn't recognize my java version whereas Ubuntu 16.04 does. mike@mike-thinks:~/hbase-1.2.6$ ./bin/start-...
0 votes
1 answer
611 views

Is my Hadoop Data File System okay with an Incorrect configuration message?

I am trying to install hadoop on my own laptop for study purposes with the tutorial from tutorialspoint. I launched start-dfs.sh: The expected output was: 10/24/14 21:37:56 Starting namenodes on [...
0 votes
1 answer
5k views

Starting Hadoop file system with `start-dfs.sh` permission denied

I try to install hbase one a standalone node but I have some issue to launch Hadoop file system with start-dfs.sh : mike@mike-thinks:/usr/local/hadoop/sbin$ ./start-dfs.sh Incorrect configuration: ...
0 votes
1 answer
5k views

hdfs: command not found with Hadoop2.9.0

I am trying to install hadoop with the tutorial from tutorialspoint. I just sucessfully installed hadoop2.9.0. I then tried to set up the namenode using the command “hdfs namenode -format” as follows. ...
0 votes
1 answer
743 views

Why don't I have a /usr/local on Ubuntu 16.04?

I followed a tutorial to install hadoop. I tried to move the folder to /usr/local/: sudo mv hadoop-2.9* /usr/local/hadoop but it answered me : mv: target '/usr/local/hadoop' is not a directory Isn'...

1
2 3 4 5