Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [hadoop]

Apache Hadoop is a software framework that supports data-intensive distributed applications under a free license. It enables applications to work with thousands of computational independent computers and petabytes of data. Hadoop was derived from Google's MapReduce and Google File System (GFS) papers.

0
votes
0answers
73 views

Issues installing Hadoop in Ubuntu

I am getting to this point and getting this error. Not sure what to do. I followed these two step by step guidelines and got the same error for both. attempt 1: https://www.digitalvidya.com/blog/...
0
votes
0answers
26 views

NullPointerException at MRClientService.getHttpPort while running Hadoop example

I am new to Ubuntu and Hadoop. I have configure Hadoop 3.1.2 on Ubuntu 18.04. All daemons are running fine however, i get exception when running builtin PI examples of hadoop. The log shows me the ...
0
votes
0answers
53 views

Problem with Sahara and Devstack on Ubuntu 16.04

I am trying to configure Devstack with Sahara enabled on Ubuntu Server 16.04. Now, I am able to load Devstack properly and I can see the dashboard. However if I add the enable sahara option in the ...
0
votes
0answers
20 views

How to connect to remote web-server through specific port on destination node

A web-server (HDFS web UI) is running on a remote host node and I want to launch it in my home computer browser. The problem is the host node is reachable just through port 2100, for example, if the ...
0
votes
0answers
12 views

How to set-up a local cluster within a external network

I have 3 nodes that are connected to the external network and can see each other through the network, namely master, slave1 and slave2. I want to create a local cluster between them such that: 192....
0
votes
0answers
27 views

Ubuntu 12.04 to 14.04 LTS upgrade fails [duplicate]

It fails with - "could not calculate the upgrade" and shows failure possibilities like pre-release upgrade and "unofficial software packages not provided by Ubuntu" /var/lib/dpkg/status shows ...
1
vote
0answers
35 views

Installed Hadoop in fully distributed mode and unable to run datanode daemon on one of the datanodes

I created 3 datanodes in hadoop , fully distributed mode. and accidentally used 'hdfs namenode -format' in one of the data nodes and i get this error: 172.31.17.100: ERROR: Cannot set priority of ...
0
votes
0answers
36 views

How to “kdestroy” user's ticket in FreeIPA (LDAP 389 Directory Server)

We are trying out FreeIPA (LDAP 389 Directory Server) and integrated the same to our Ambari Hadoop cluster (HDP v3.0.1). We are able to add users and provide them access to Hadoop with help of Kinit ...
0
votes
0answers
28 views

How can I use Ubuntu hadoop3.1.1 in win C:\hadoop-3.1.1

I've already downloaded hadoop3.1.1 in Windows to C:\hadoop-3.1.1 Today I started using Ubuntu. Is it possible to use hadoop which I already downloaded to C:\hadoop-3.1.1, or do I have to download it ...
0
votes
2answers
548 views

Hadoop giving error “ls: Failed on local exception: org.apache.hadoop.ipc.RpcException: RPC response exceeds maximum data length”

Folks, I've literally searched everywhere, spent ounces and ounces of time. Can you please help me here? I have Java 8 and Hadoop 2.9.1 in my Ubuntu 14.04. After I type start-all.sh, services do get ...
1
vote
1answer
539 views

SSH without password only works one time

I want to configure SSH to use it without the need of writing the password. I am using Ubuntu 18.04 LTS on Windows 10. I need it to run Hadoop 3.1.1 (https://hadoop.apache.org/docs/r3.1.1/hadoop-...
0
votes
1answer
5k views

FAILED: HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

I'm trying to run Hive 3.1 with Hadoop 3.0. Following is my system configuration: Ubuntu 18.04.1 LTS Hadoop version 3.0.3 Hive 3.1.0 Derby 10.14.2 When I execute the show tables; query I get the ...
1
vote
1answer
637 views

Could not resolve hostname datanode1: Name or service not known

I am building an SSH connection with my namenode and two datanodes on AWS. When I try to copy the config & my .pem file from namenode to datanodes, I get the following error: Could not resolve ...
0
votes
1answer
768 views

Configuring JAVA HOME for Hadoop 3.0.3 when having multiple Java versions

I'm working on a Hadoop installation done by someone else. Ubuntu 18.04.1 LTS Hadoop version 3.0.3 Java open jdk 11 (auto mode) and 8 (manual mode) Currently Hadoop runs with Java 11 but I want to ...
0
votes
1answer
755 views

Copy File from HDFS to Local File System based on timestamp Using Shell Script

I want to move n number of file residing in the Hadoop directory to Local File system based on timestamp. Scenario: suppose I am doing the first time transition from HDFS to the Local file system, so ...
0
votes
1answer
421 views

Sqoop import from MySQL to Hive throwing HIVE_CONF_DIR error

I am trying to load my MySQL tableLab5 to Hive using Sqoop. Sqoop ran successfully and has taken the data from mysql into hdfs. But throwing the below bolded error about HIVE_CONF_DIR not set. I have ...
0
votes
0answers
182 views

A JNI error has occurred, in HIVE 3.0.0

I installed Hadoop successfully and it's working fine, but when I installed Hive and ran the hive command in terminal, I got this error Error: A JNI error has occurred, please check your installation ...
0
votes
0answers
29 views

Unable to login from master to slave while try to start hadoop datanode from master

Actully my master and slave have different username,when it try to logging slave it's user name come as master, for example my master user name is testmaster and IP in ****192.168.56.100** and my ...
0
votes
0answers
129 views

Trouble installing Cloudera CDH 5 with YARN on Ubuntu 16.04

I have been following this installation guide to try and setup Cloudera CDH 5 with Yarn on Ubuntu 16.04. I have successfully implemented all the steps up until step 3, where, when I try to install ...
0
votes
0answers
58 views

I'm trying to install Hadoop in VirtualBox. Where the command “hdfs namenode -format” throws me an error

[vitcc@localhost ~]$ hdfs namenode -format /usr/bin/hdfs: line 304: /usr/lib/jvm/jre/bin/java: No such file or directory
0
votes
0answers
33 views

YARN in EC2 cluster cannot create /logs directory

I have created a 3 instance cluster for Hadoop 2.9.1 and configured all the properties as mentioned here. When I start-all.sh, the DFS starts without any issue however for YARN I get the following ...
0
votes
0answers
21 views

Hadoop stopped working after installing bdsgames package

I have hadoop 2.x installed. I typed this command and entered password: sudo apt-get bdsgames Since then my hadoop commands have not been working. How can I resolve this?
0
votes
0answers
42 views

How to tranfer Influxdb data into hadoop HDFS for storage?

Time series data are stored in the influxdb, but how to further connect influxdb with hadoop hdfs to store all influxdb data into HDFS
1
vote
0answers
637 views

HDF5 Installation Error in the latest Ubuntu 18.04LTS with Intel Compilers

While I used HDF5 library without any problem with Ubuntu version 16.04 LTS, after I upgraded Ubuntu to Ubuntu 18.04 LTS, I have trouble in installing HDF5 with Intel Fortran and C Compilers. I ...
0
votes
0answers
31 views

Ambari Confirm Hosts been too long

I am in Confirm Hosts step in Ambari configuration for install hadoop on agents. Now It's past several hours since the registration. Note: I created containers (lxc) as hosts.
0
votes
0answers
132 views

Problem with hadoop: only 2 nodes appearing in jps

I'm new to Ubuntu and hadoop also. I've installed hadoop for my university project but I'm facing difficulties. When I'm running jps in terminal only 2 of the nodes are appearing: Jps and DataNode .....
0
votes
0answers
77 views

Does Ubuntu 14.04 support Apache-Ambari?

I'm new to Hadoop and Apache Ambari. I see major of tutorials about installing Ambari are on Linux os except for Ubuntu. Does Ambari even last version support ubuntu 14.04? Both ambari-server and ...
0
votes
0answers
76 views

Unable to locate package hadoop-yarn-resourcemanager

I am using Ubuntu 16.04 LTS and I am trying to install cloudera. When I came to install YARN, the hadoop-yarn-resourcemanager, the terminal shows this output: Error: Unable to locate package hadoop-...
0
votes
1answer
761 views

Cannot start hadoop “localhost: nice: cannot set niceness”

Starting namenodes on [localhost] localhost: starting namenode, logging to /opt/hadoop-2.9.0/logs/hadoop-ha-namenode-root.out localhost: nice: cannot set niceness: Permission denied localhost: ...
0
votes
1answer
333 views

hdfs: split ls output

on hdfs, the output of ls command looks like this: drwxrwxr-x 5 matteorr matteorr 4096 Jan 10 17:37 /data/Cluster drwxr-xr-x 2 matteorr matteorr 4096 Jan 19 10:43 /data/Desktop drwxrwxr-x 9 ...
-1
votes
1answer
841 views

Cannot open display error when I run sudo gedit on Ubuntu 14.04.5

solitude@Solitude:~/Desktop$ sudo gedit ~/.bashrc error: XDG_RUNTIME_DIR not set in the environment. (gedit:12631): Gtk-WARNING **: cannot open display:
1
vote
0answers
75 views

Hadoop multinode cluster error in a p2p connection [closed]

We have a p2p connection using cross-wire, when I start the hadoop nodes using ./start-all.sh it asks us for password of the slave and denies access. When I use jps to check the status it shows the ...
0
votes
0answers
51 views

Entire file system permissions were changed on Ubuntu 16.04 LTS

I'm running hadoop in my machine with hduser. When I was trying to user 'put' and 'get' commands it was throwing that "permission denied". hduser@salmon-son-of-god:/usr/local/hadoop-2.6.5/bin$ hadoop ...
2
votes
1answer
124 views

What is a good way to create a low cost Ubuntu based Cassandra or Hadoop Cluster in my house?

I have a series of "old" data center servers (that I bought when my company upgraded) that I use to run Cassandra and Hadoop clusters at home. They are loud, power hungry and take up a lot of space ...
1
vote
1answer
1k views

Error while copying file to HDFS in hadoop ecosystem

While issuing a command to copy file from Local File system to HDFS on terminal in Hadoop 3.0, it shows error hadoop-3.0.0/hadoop2_data/hdfs/datanode': No such file or directory: `hdfs://localhost:...
1
vote
1answer
395 views

hadoop namenode -format showing error

I am trying to setup Hadoop 3.0.0 on my Ubuntu desktop. I have done all the required setups, but when running the command ./hadoop namenode -format to format name it shows error: root@DESKTOP-JIUFBOR:...
0
votes
1answer
314 views

Which version of Hadoop is compatible with Ubuntu 17.10?

Which version of Hadoop is compatible with Ubuntu 17.10 ?
0
votes
0answers
313 views

Ubuntu Hadoop-3.0.0 Can't set priority of all the nodes . Details in the picture

Both Master and Slave1 can't set priority of any nodes process ..... It's my last few steps to accomplish this cluster , thanks in advanced.
0
votes
0answers
98 views

Hadoop 3.0.0 in Ubuntu : Master node can't log in by command “ssh Master” details in the picture

I've been changed the set of /etc/ssh/sshd_config/, but it still goes: Permission Denied, please try again. And the command ssh localhost does work! I just can't figure out what's wrong with it!
1
vote
1answer
256 views

Unexpected Error: Error renaming temporary file, Text File Busy

I've been working on a Ubuntu and Hadoop, I'm at installing and configuring stage. For this I'm following a link: Hadoop Beginner When I am configuring the Hadoop so when I reach to gedit core-site....
0
votes
2answers
818 views

mapred-site.xml.template is missing in Hadoop 2.9.0

I'm a brand new user of this forum, Ubuntu, and Hadoop. I'm running Ubuntu on VirtualBox, and I've been following a tutorial for installing and configuring Ubuntu and Hadoop 2.7.3 (though I'm trying ...
3
votes
1answer
308 views

Installing Hadoop in Ubuntu 16.04

I am pretty new to Ubuntu/linux and totally new to Hadoop. I'm following the guide here: https://www.digitalocean.com/community/tutorials/how-to-install-hadoop-in-stand-alone-mode-on-ubuntu-16-04 I ...
0
votes
0answers
97 views

pig -version Shows error

deepak@deepak-Linux:~$ pig -version /usr/local/hadoop/bin/hadoop: line 166: /usr//bin/java: No such file or directory deepak@deepak-Linux:~$ echo $JAVA_HOME /usr/java/jdk-9.0.1 deepak@deepak-Linux:~$...
0
votes
2answers
639 views

Hbase doesn't recognize my java version

I'm trying to install Hbase with this tutorial and I stepped upon something curious : hbase doesn't recognize my java version whereas Ubuntu 16.04 does. mike@mike-thinks:~/hbase-1.2.6$ ./bin/start-...
0
votes
1answer
319 views

Is my Hadoop Data File System okay with an Incorrect configuration message?

I am trying to install hadoop on my own laptop for study purposes with the tutorial from tutorialspoint. I launched start-dfs.sh: The expected output was: 10/24/14 21:37:56 Starting namenodes on [...
0
votes
1answer
2k views

Starting Hadoop file system with `start-dfs.sh` permission denied

I try to install hbase one a standalone node but I have some issue to launch Hadoop file system with start-dfs.sh : mike@mike-thinks:/usr/local/hadoop/sbin$ ./start-dfs.sh Incorrect configuration: ...
0
votes
1answer
1k views

hdfs: command not found with Hadoop2.9.0

I am trying to install hadoop with the tutorial from tutorialspoint. I just sucessfully installed hadoop2.9.0. I then tried to set up the namenode using the command “hdfs namenode -format” as follows. ...
0
votes
1answer
349 views

Why don't I have a /usr/local on Ubuntu 16.04?

I followed a tutorial to install hadoop. I tried to move the folder to /usr/local/: sudo mv hadoop-2.9* /usr/local/hadoop but it answered me : mv: target '/usr/local/hadoop' is not a directory Isn'...
0
votes
1answer
169 views

Installing Spark on Hadoop 2.5

I'm running Hadoop on Ubuntu on a Oracle VM on Windows 7. I'm trying to install Spark on an existing Hadoop installation. For demonstration purposes I do not want to change the existing Hadoop version....
0
votes
1answer
298 views

Hadoop Name Node Formatting fail

I am using following tutorial to install Hadoop on my system and have followed all the steps uptil formatting the name node when the command hdfs namenode -format is giving me error: NFO util....