Questions tagged [hadoop]
Apache Hadoop is a software framework that supports data-intensive distributed applications under a free license. It enables applications to work with thousands of computational independent computers and petabytes of data. Hadoop was derived from Google's MapReduce and Google File System (GFS) papers.
207
questions
0
votes
0
answers
32
views
Error running hadoop: hadoop-mapreduce-examples-3.3.6.jar: Permission denied
Whenever I am running hadoop on my VM ubuntu I am getting this error:
command
/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.3.6.jar grep /input /output 'dfs[a-z.]+'
error:
bash:...
0
votes
0
answers
91
views
404 not found error when trying to download Hadoop
I entered the command to download hadoop, but it tells me that there's no connection and 404 not found.
HTTP request transmitted, waiting for response… 404 not found
-1
votes
1
answer
952
views
usage of mount -o remount,exec /tmp Hbase hadoop file system and solr
Can you please tell me what is main work of this command
mount -o remount,exec /tmp
I am using it when my hbase giving some errors but i don't know what is happening using this command.
and i want ...
0
votes
1
answer
9k
views
mkdir: cannot create directory permission denied when launching hadoop
I get this error whenever I try to launch Hadoop. I have already configured the files needed to launch it but I get:
$ hdfs namenode -format
WARNING: /home/jeunesseafricaine/hadoop-3.3.1/logs does ...
-1
votes
1
answer
130
views
Why are Hadoop and Spark not in the official Ubuntu repositories?
UPDATE (2021-11-13 22:12 GMT+8): regarding the Snap packages, @karel suggested that this question is a duplicate of Why don't the Ubuntu repositories have the latest versions of software? I ...
-1
votes
1
answer
263
views
hdfs normally working but with sudo it shows command not found
I am trying to install hadoop3.1.1 and i have set ./bashrc file and all the other necessary files. while applying hdfs i am facing this situation shown in the picture.
this image shows commands I have ...
0
votes
0
answers
2k
views
GVfs Metadata is not supported while Map Reduce Configuration
while executing this commond:
sudo gedit /etc/profile.d/hadoop.sh
Getting this error:
(gedit:5376): Tepl-WARNING **: 16:32:59.581: GVfs metadata is not supported. Fallback to TeplMetadataManager. ...
0
votes
1
answer
1k
views
Error while running jar file
I used Ubuntu 18.04.4 in my IBM virtual machine and I got an error while running my jar file in Ubuntu terminal. I've already installed Java.
Exception in thread "main" java.lang.NoClassDefFoundError:...
0
votes
0
answers
33
views
Want to connect two virtual machine (host windows) running on different laptops (using different networks)
I want to build a hadoop cluster with one master node and 2 slave nodes. The master node will be running on a virtual box of pc-1. Slave nodes will be running on virtual box of pc-2.
Slaves on same ...
0
votes
1
answer
889
views
Unable to start single node cluster of Hadoop
I am trying to install hadoop 2.9.1 on ubuntu 19.10 and I did all steps explained in this video https://www.youtube.com/watch?v=Y6oit3rCsZo
The problem is that when I triyed to start single node ...
- 1
2
votes
2
answers
339
views
What is the conventional directory to install Hadoop into?
Sooo... Hadoop doesn't have an installer, so can't use apt for it... Still pretty new at ubuntu stuff... I realize I can install it anywhere, but is there an established convention to be followed for ...
- 21
1
vote
0
answers
181
views
Connect Multiple nodes for hadoop via ssh on ubuntu
We are trying to set-up a two-nodes hadoop clusters on two different machines.
SSH to localhost without password have been setted up on both machines. And hadoop can run well as single node on each ...
- 11
-3
votes
2
answers
504
views
Error during ssh server installation: connect to host localhost port 22: Connection refused
I am trying to install openssh-server. Then at the end I am getting the error:
ssh.service - OpenBSD Secure Shell server
Loaded: loaded (/lib/systemd/system/ssh.service; enabled; vendor preset: ...
- 5
0
votes
1
answer
163
views
dealing low memory in hadoop 3.1.3 in vmware virtual machine ubuntu 19.10
I installed Hadoop on a Ubuntu virtual machine.
I downloaded the source file that means among the releases available on this page https://hadoop.apache.org/releases.html I downloaded 3.1.3 file
https:...
1
vote
1
answer
211
views
hadoop installation errors in ubuntu 19.10
I was installing Hadoop on Ubuntu 19.10 I got following errors
I am comipiling https://hadoop.apache.org/releases.html
file version https://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-3.1.3/...
1
vote
0
answers
304
views
Unable to connect to remote host: Connection refused
I try run hadoop:
hadoop jar hadoop-examples.jar pi 4 100
and get error:
java.net.ConnectionException: ... to localhost:9000 failed on connection exception... Connection refused.
When I run:
...
- 11
0
votes
1
answer
99
views
Give permission to hdf to remove directory
When I run hdf namenode -format then it wants to remove a directory /home/hadoop/hadooptmpdata/dfs/name/current but it has no permission to do it.
How to give it a permission?
- 11
2
votes
2
answers
3k
views
I am unable to edit gedit ~/.bashrc in ubuntu please help how to do
I am installing hadoop in ubuntu 19 and I am trying to edit bashrc file in gedit but unable to edit gedit bashrc file please help me how to do.
This my hduser dir:
hdusr@suri-VirtualBox
when I ...
-1
votes
1
answer
5k
views
ssh: Could not resolve hostname
i try to input this command:
start-dfs.sh
and the output like this:
Starting namenodes on [HadoopMaster]
HadoopMaster: ssh: Could not resolve hostname hadoopmaster: Name or service not known
what ...
1
vote
0
answers
418
views
su {user} not working inside the ssh shell on the server
I am trying to connect to the server as root using ssh command in Ubuntu bash. I use
ssh -p port root@ip
to connect to the server and the config of ssh is a password-less connection ssh with the ...
- 111
1
vote
1
answer
103
views
Jdk or headless version ? Which should I use for Hadoop installation
Which version of java should I use for Hadoop installation ? Jdk or headless version ?
0
votes
2
answers
5k
views
Hadoop giving error "ls: Failed on local exception: org.apache.hadoop.ipc.RpcException: RPC response exceeds maximum data length"
Folks,
I've literally searched everywhere, spent ounces and ounces of time. Can you please help me here? I have Java 8 and Hadoop 2.9.1 in my Ubuntu 14.04. After I type start-all.sh, services do get ...
1
vote
1
answer
2k
views
SSH without password only works one time
I want to configure SSH to use it without the need of writing the password. I am using Ubuntu 18.04 LTS on Windows 10. I need it to run Hadoop 3.1.1 (https://hadoop.apache.org/docs/r3.1.1/hadoop-...
- 111
5
votes
1
answer
43k
views
FAILED: HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
I'm trying to run Hive 3.1 with Hadoop 3.0. Following is my system configuration:
Ubuntu 18.04.1 LTS
Hadoop version 3.0.3
Hive 3.1.0
Derby 10.14.2
When I execute the show tables; query I get the ...
- 183
1
vote
1
answer
14k
views
Could not resolve hostname datanode1: Name or service not known
I am building an SSH connection with my namenode and two datanodes on AWS. When I try to copy the config & my .pem file from namenode to datanodes, I get the following error:
Could not resolve ...
- 111
2
votes
1
answer
5k
views
Configuring JAVA HOME for Hadoop 3.0.3 when having multiple Java versions
I'm working on a Hadoop installation done by someone else.
Ubuntu 18.04.1 LTS
Hadoop version 3.0.3
Java open jdk 11 (auto mode) and 8 (manual mode)
Currently Hadoop runs with Java 11 but I want to ...
- 183
0
votes
1
answer
3k
views
Copy File from HDFS to Local File System based on timestamp Using Shell Script
I want to move n number of file residing in the Hadoop directory to Local File system based on timestamp. Scenario:
suppose I am doing the first time transition from HDFS to the Local file system, so ...
user851104
0
votes
1
answer
1k
views
Sqoop import from MySQL to Hive throwing HIVE_CONF_DIR error
I am trying to load my MySQL tableLab5 to Hive using Sqoop. Sqoop ran successfully and has taken the data from mysql into hdfs. But throwing the below bolded error about HIVE_CONF_DIR not set. I have ...
0
votes
1
answer
549
views
A JNI error has occurred, in HIVE 3.0.0
I installed Hadoop successfully and it's working fine, but when I installed Hive and ran the hive command in terminal, I got this error
Error: A JNI error has occurred, please check your installation ...
1
vote
0
answers
39
views
Hadoop stopped working after installing bdsgames package
I have hadoop 2.x installed. I typed this command and entered password:
sudo apt-get bdsgames
Since then my hadoop commands have not been working.
How can I resolve this?
1
vote
0
answers
1k
views
HDF5 Installation Error in the latest Ubuntu 18.04LTS with Intel Compilers
While I used HDF5 library without any problem with Ubuntu version 16.04 LTS, after I upgraded Ubuntu to Ubuntu 18.04 LTS, I have trouble in installing HDF5 with Intel Fortran and C Compilers.
I ...
0
votes
1
answer
2k
views
Cannot start hadoop "localhost: nice: cannot set niceness"
Starting namenodes on [localhost]
localhost: starting namenode, logging to /opt/hadoop-2.9.0/logs/hadoop-ha-namenode-root.out
localhost: nice: cannot set niceness: Permission denied
localhost: ...
- 1
0
votes
1
answer
2k
views
hdfs: split ls output
on hdfs, the output of ls command looks like this:
drwxrwxr-x 5 matteorr matteorr 4096 Jan 10 17:37 /data/Cluster
drwxr-xr-x 2 matteorr matteorr 4096 Jan 19 10:43 /data/Desktop
drwxrwxr-x 9 ...
- 231
0
votes
1
answer
4k
views
Cannot open display error when I run sudo gedit on Ubuntu 14.04.5
solitude@Solitude:~/Desktop$ sudo gedit ~/.bashrc
error: XDG_RUNTIME_DIR not set in the environment.
(gedit:12631): Gtk-WARNING **: cannot open display:
- 21
0
votes
0
answers
110
views
Entire file system permissions were changed on Ubuntu 16.04 LTS
I'm running hadoop in my machine with hduser. When I was trying to user 'put' and 'get' commands it was throwing that "permission denied".
hduser@salmon-son-of-god:/usr/local/hadoop-2.6.5/bin$ hadoop ...
- 11
3
votes
1
answer
235
views
What is a good way to create a low cost Ubuntu based Cassandra or Hadoop Cluster in my house?
I have a series of "old" data center servers (that I bought when my company upgraded) that I use to run Cassandra and Hadoop clusters at home. They are loud, power hungry and take up a lot of space ...
- 593
1
vote
1
answer
8k
views
Error while copying file to HDFS in hadoop ecosystem
While issuing a command to copy file from Local File system to HDFS on terminal in Hadoop 3.0, it shows error
hadoop-3.0.0/hadoop2_data/hdfs/datanode': No such file or directory:
`hdfs://localhost:...
- 25
1
vote
1
answer
964
views
hadoop namenode -format showing error
I am trying to setup Hadoop 3.0.0 on my Ubuntu desktop. I have done all the required setups, but when running the command ./hadoop namenode -format to format name it shows error:
root@DESKTOP-JIUFBOR:...
- 25
0
votes
1
answer
533
views
Which version of Hadoop is compatible with Ubuntu 17.10?
Which version of Hadoop is compatible with Ubuntu 17.10 ?
1
vote
1
answer
1k
views
Unexpected Error: Error renaming temporary file, Text File Busy
I've been working on a Ubuntu and Hadoop, I'm at installing and configuring stage.
For this I'm following a link:
Hadoop Beginner
When I am configuring the Hadoop so when I reach to
gedit core-site....
- 11
0
votes
2
answers
3k
views
mapred-site.xml.template is missing in Hadoop 2.9.0
I'm a brand new user of this forum, Ubuntu, and Hadoop. I'm running Ubuntu on VirtualBox, and I've been following a tutorial for installing and configuring Ubuntu and Hadoop 2.7.3 (though I'm trying ...
- 11
2
votes
1
answer
401
views
Installing Hadoop in Ubuntu 16.04
I am pretty new to Ubuntu/linux and totally new to Hadoop.
I'm following the guide here:
https://www.digitalocean.com/community/tutorials/how-to-install-hadoop-in-stand-alone-mode-on-ubuntu-16-04
I ...
- 415
0
votes
0
answers
309
views
pig -version Shows error
deepak@deepak-Linux:~$ pig -version
/usr/local/hadoop/bin/hadoop: line 166: /usr//bin/java: No such file or directory
deepak@deepak-Linux:~$ echo $JAVA_HOME
/usr/java/jdk-9.0.1
deepak@deepak-Linux:~$...
- 1
0
votes
2
answers
3k
views
Hbase doesn't recognize my java version
I'm trying to install Hbase with this tutorial and I stepped upon something curious : hbase doesn't recognize my java version whereas Ubuntu 16.04 does.
mike@mike-thinks:~/hbase-1.2.6$ ./bin/start-...
- 1,173
0
votes
1
answer
690
views
Is my Hadoop Data File System okay with an Incorrect configuration message?
I am trying to install hadoop on my own laptop for study purposes with the tutorial from tutorialspoint. I launched start-dfs.sh:
The expected output was:
10/24/14 21:37:56
Starting namenodes on [...
- 1,173
0
votes
1
answer
5k
views
Starting Hadoop file system with `start-dfs.sh` permission denied
I try to install hbase one a standalone node but I have some issue to launch Hadoop file system with start-dfs.sh :
mike@mike-thinks:/usr/local/hadoop/sbin$ ./start-dfs.sh
Incorrect configuration: ...
- 1,173
0
votes
1
answer
8k
views
hdfs: command not found with Hadoop2.9.0
I am trying to install hadoop with the tutorial from tutorialspoint.
I just sucessfully installed hadoop2.9.0. I then tried to set up the namenode using the command “hdfs namenode -format” as follows.
...
- 1,173
0
votes
1
answer
1k
views
Why don't I have a /usr/local on Ubuntu 16.04?
I followed a tutorial to install hadoop. I tried to move the folder to /usr/local/:
sudo mv hadoop-2.9* /usr/local/hadoop
but it answered me :
mv: target '/usr/local/hadoop' is not a directory
Isn'...
- 1,173
1
vote
1
answer
241
views
Installing Spark on Hadoop 2.5
I'm running Hadoop on Ubuntu on a Oracle VM on Windows 7. I'm trying to install Spark on an existing Hadoop installation. For demonstration purposes I do not want to change the existing Hadoop version....
- 183
0
votes
1
answer
701
views
Hadoop Name Node Formatting fail
I am using following tutorial to install Hadoop on my system and have followed all the steps uptil formatting the name node when the command
hdfs namenode -format
is giving me error:
NFO util....
- 101