Questions tagged [hadoop]
Apache Hadoop is a software framework that supports data-intensive distributed applications under a free license. It enables applications to work with thousands of computational independent computers and petabytes of data. Hadoop was derived from Google's MapReduce and Google File System (GFS) papers.
207
questions
1
vote
1
answer
2k
views
hadoop command is not working
I am trying to install single node hadoop on my ubuntu 14.04 as per the documentation available at bogotobogo.com.
after following all the steps when I tried to format the hadoop file system it shows ...
0
votes
1
answer
278
views
Error Installing Bigtop - E: Sub-process /usr/bin/dpkg returned an error code (1)
I'm facing the following error when trying to install the Bigtop distribution of Hadoop.
Errors were encountered while processing:
/var/cache/apt/archives/oozie_4.3.0-1_all.deb
E: Sub-process /usr/...
0
votes
1
answer
4k
views
hadoop-datanode is not running
I am new to hadoop and did installation hadoop-2.7.3.Also completed all the steps for installation.however my datanode is not running after ran the command start-all.sh.
So my doubt is what action ...
1
vote
1
answer
2k
views
Shell script does not work on crontab but works otherwise
I have a script which merges all the small files present at hourly directories at hdfs into one single large file. The script works OK when executed through CLI. I then set the script to run at 01:30 ...
4
votes
1
answer
10k
views
"ssh localhost " command is showing 20 bad configuration options
hduser1@archit-HP-Notebook:~$ ssh localhost
/etc/ssh/ssh_config: line 11: Bad configuration option: hostkey
/etc/ssh/ssh_config: line 12: Bad configuration option: hostkey
/etc/ssh/ssh_config: line 13:...
0
votes
1
answer
149
views
hadoop fs -ls /usr/local is dealing my Hadoop installation folder as file
I am very much new to Hadoop. Recently I installed Hadoop-2.8.0 with the help of the blogs on internet.
I thought everything is running fine, but today when I tried running hadoop fs -ls /usr/local ...
2
votes
0
answers
2k
views
SLF4J: Class path contains multiple SLF4J bindings
I am trying to stream real time twitter data into HDFS using Apache flume. And when i run the command
./flume-ng agent -c /usr/local/apache-flume-1.4.0-bin/conf/ -f /usr/local/apache-flume-1.4.0-bin/...
1
vote
1
answer
1k
views
On Hadoop installation, bash: gives an 'is a directory error'
When I run the command:
$ $HADOOP_INSTALL/etc/hadoop fs -mkdir /tmp
it gives me the following error:
bash: /etc/hadoop-2.7.1/etc/hadoop: Is a directory
These are the contents of my .bashrc file:
...
1
vote
0
answers
2k
views
SSH asking password each time i run start-all.sh for hadoop
I have installed Hadoop on following path:
/opt/hadoop
Also installed openssh-server with following commands:
sudo apt-get install openssh-server openssh-client
After this I created RSA keys with ...
1
vote
1
answer
9k
views
root@localhost's password: localhost: Permission denied, please try again
I am trying to setup Hadoop in "Pseudo-distributed Mode". I have already performed the below steps:
I have installed Openssh-server.
I have generated the rsa type key pair.
Copied *rsa.pub to ...
1
vote
0
answers
161
views
Installing OpenTSDB on Ubuntu 14
Can you please guide me for how to install OpenTSDB on ubuntu 14.04? And is hadoop Prerequisite of OpenTSDB?
4
votes
3
answers
5k
views
Avoid entering password when launching Hadoop in Pseudo-distributed mode
So I am running Hadoop in pseudo distributed mode on my laptop, to learn about these things.
I am setting the thing up as explained in the manual.
I can start hadoop, no problem, but the launch ...
1
vote
1
answer
1k
views
How to access pyspark in anaconda installed in another user?
I have two user user1 and user2 with latter one having root privilege.I have installed spark and hadoop in user2.
Now i have installed Anaconda2 on user1 and when iam trying to run my job on jupyter ...
0
votes
1
answer
2k
views
Unable to upload directory to hdfs. `/usr/local/tmp/': No such file or directory
I'm trying to upload files to a directory i made on HDFS but I'm not able to do so as ubuntu is not able to locate the files on my machine. The directory 'states'(one i'm trying to upload) contains 50 ...
0
votes
0
answers
281
views
kylin.sh command
I have been trying to install Kylin and after I tried to execute the command Kylin.sh start, I get the following error.
I can successfully run hadoop, hbase and hive from the root.
Any kind of ...
1
vote
0
answers
317
views
Error: java.io.IOException: Communications link failure
I am doing a small project to achieve auto-completion on ubuntu.
I run the code in Docker with hadoop. I also use Ampps downloaded from http://ampps.com/download.
But I have a problem that I can't ...
3
votes
1
answer
9k
views
echo $HADOOP_HOME is blank
I installed hadoop but when i am using echo $HADOOP_HOME .its showing blank.please help how to check hadoop location.
1
vote
0
answers
253
views
How do I change the IP address of ubuntu(16.x) machine on oracle virtual box(5.0.20)
I'm trying to setup a hadoop multi node cluster on Ubuntu 16.x with oracle virtual box(5.0.20). As part of the setup I want to change the IP address of one Ubuntu machine.
But the problem is, there ...
1
vote
0
answers
711
views
Installation process of Apache Nutch in Ubuntu 16.04 , hadoop 2.6 already installed
What is the correct compatible format of Apache Nutch for Ubuntu 16.04 and hadoop 2.6 ? What is the installation process ?
0
votes
1
answer
14k
views
Editing the ~/.bashrc file for hadoop installation
I have been trying to install hadoop-2.6.0 in my Ubuntu 16.04 machine.
I have been logged into the hduser from another user krishna by:
sudo su hduser
and the hduser then gets into its use.
After ...
2
votes
0
answers
599
views
Ubuntu 16.04 /bin/kill bug?
I recently installed Ubuntu 16.04 on the virtual machine using VMware.
To test my MapReduce code, I also installed hadoop 2.7.3 and setup a single node cluster.
When I execute MapReduce algorithm, ...
2
votes
1
answer
2k
views
namenode not running... i have already tried sudo chown -R username /usr/local/hadoop/ please help
$ /usr/local/hadoop/sbin/start-all.sh
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
Starting namenodes on [localhost]
divyeshlad@localhost's password:
localhost: starting ...
1
vote
0
answers
504
views
Help needed for Hadoop to execute sbin/start-dfs.sh without being asked root login
I have followed some other answers but was unable to find something that works
When I do
sudo sbin/start-dfs.sh
I get prompted for the root passwd
I set in /etc/ssh/sshd_config
The following ...
0
votes
1
answer
535
views
Type '--2016-08-04' is not known on line 1 in source list /etc/apt/sources.list [duplicate]
I want to install hadoop in my Ubuntu 14.04 system. Following the instruction of a blog, I need to update the "source list".
But when I input the command: sudo apt-get update I get errors:
E:...
2
votes
1
answer
2k
views
start-all.sh giving error of not resolving hostname and name or servce not known
i am setting up hadoop-2.7.2 on ubuntu 14.04 through VMware.
After steps described in michael g. noll's tutorials , when i tried this,
/usr/local/hadoop/sbin/start-all.sh
i am getting this stuff,
...
1
vote
1
answer
476
views
Why my .bashrc has error?
I added this path to .bashrc file .
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
export HADOOP_INSTALL=/usr/local/hadoop
export $PATH=$PATH:$HADOOP_INSTALL/bin
export $PATH=$PATH:$...
2
votes
1
answer
2k
views
How to resolve UnsatisfiedLinkError installing Spark on Ubuntu 16.04?
I am trying to install Spark on Ubuntu 16.04 but it won't.
What I have done is I have tried to install Spark with ./Spark but it is showing me the following error:
Caused by: java.lang....
-1
votes
1
answer
168
views
I tried installing the ssh_package into my ubuntu 16.04 but it is giving me this error. I am running ubuntu on VMWare workstation pro [closed]
rmehra@ubuntu:~/Desktop/ssh_package$ sudo dpkg -i openssh-client_1%3a5.3p1-3ubuntu7_i386.deb
(Reading database ... 172254 files and directories currently installed.)
Preparing to unpack openssh-...
2
votes
3
answers
3k
views
move files in a hadoop dir with specified number of files
I wish to move files in a hadoop dir in a timely manner.
The hadoop dir contains 1000 files with the same extension. I wish to move 100 of them every 10 minutes. I can setup a cron job to move the ...
1
vote
1
answer
25k
views
-bash: export: not a valid identifier [duplicate]
Whenever I open a terminal, I always end up with this same error.
-bash: export: `/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin:/usr/...
4
votes
1
answer
9k
views
Ubuntu 16.04 kills session when resource usage is extremely high
I'm working on project in hadoop on my computer. When I start some job consuming most of the available resources (100% CPU, high RAM usage) 'something' kills my user's session and all its processes.
...
3
votes
0
answers
3k
views
Install Cloudera and Hadoop on ubuntu 16
I am trying to find online materials to find the steps or guideline on installing cloudera and Hadoop. I found this and this page but none of them helped.
If you know please help me on this as I am ...
0
votes
1
answer
2k
views
HADOOP commands not working any where in ubuntu 16.04
Hadoop commands not working any where
commands working hadoop/bin directory only once abserve first I have tried outside hadoop/bin second time I had done with in bin
0
votes
1
answer
3k
views
Hadoop and Eclipse - Server IPC version 9 cannot communicate with client version 4
I'm working on Ubuntu and using Eclipse to work with Hadoop. After setting the Run configuration, a message says Incomplete HDFS URI, no host
UPDATE: found the solution! but after running the program,...
0
votes
1
answer
1k
views
Problem when starting a single node of Hadoop cluster in Ubuntu
I am new to Hadoop and I was going through the installation process mentioned here. I have installed hadoop-1.2.1 on my system. After configuring everything when I tried to start a single node cluster ...
0
votes
1
answer
984
views
Hadoop - Error on binding to port 9000
I've been attempting to setup Hadoop on AWS for a few days now.
I am on fresh EC2 Ubuntu instances with no other software installed apart from Java.
I run these commands:
ubuntu@ip-172-30-0-117:/...
1
vote
2
answers
10k
views
Issues with disabling iPv6 on Ubuntu 14.0.4
To disable iPv6 on Ubuntu 14.0.4 (LTS) I added these lines in sysctl.conf file:
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
Since I ...
1
vote
1
answer
46
views
adduser fails when relating hadoop-plugin to hbase-master
I'm trying to set up Hbase in juju using the following charm: https://jujucharms.com/u/bigdata-dev/apache-hbase/trusty/19
I receive an error when I try to add a relation between the hadoop-plugin and ...
1
vote
2
answers
1k
views
Hadoop 2.6 start-dfs.sh errors on Centos 6.7 [closed]
I use this tutorial to install Hadoop 2.6 on Centos 6.7 with Java 1.8.0_72 and everything goes well before execution of start-dfs.sh from Hadoop-home/sbin/srart-dfs.sh. Below is the output:
[hadoop@10 ...
0
votes
1
answer
897
views
Jobtracker and tasktracker not showing
I am using hadoop 2.7.1 on Ubuntu 14.04, when I run hadoop
and use jps, the following processes are shown:
Resourcemanager
Namenode
Datanode
Jps
SecondaryNameNode
NodeManager
Here job tracker and ...
0
votes
1
answer
178
views
How to use amazon instances' public IPs in hadoop configuration?
I was trying to configure Hadoop by using the public IPs of amazon instances instead of their intra-network IPs as my aim is to create a hybrid cluster i.e. Cloud + Local machine cluster. Although all ...
1
vote
2
answers
15k
views
Permission denied on start-dfs.sh
I am trying to create a 2-node Hadoop cluster, following this guide. The one node is my Ubuntu laptop and the slave node, the 2nd one, is a virtualbox that also runs Ubuntu.
I am doing in the master ...
-1
votes
1
answer
37
views
Ubuntu and Windows [duplicate]
Currently I am working in Ubuntu. Big Data. I do not have Windows OS in my system. i am going to install hadoop in Ubuntu. But I don't know whether windows is compulsory to have along with ubuntu. Can ...
0
votes
1
answer
864
views
Problem with cluster installation with cloudera manager
I am installing Cloudera Manager in my system(14.04/64b).
While installing at the final step, before finish installation I got some ERRORs in validation as shown below,
errors in above page are,
...
0
votes
1
answer
869
views
Hadoop Installation Problem
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
15/11/07 16:01:44 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes ...
1
vote
1
answer
499
views
Hadoop & Spark - why no Ubuntu packages?
I want to install Hadoop & Spark using aptitude, not the lengthy and complicated process described in How to install Hadoop?.
hadoop-ubuntu appears to have been dead for almost 2 years.
I wonder ...
3
votes
1
answer
5k
views
gedit - cannot open display
I am using Ubuntu 14.04 LTS, and for installing Hadoop I created hadoop user as hduser. When I try to open ~/.bashrc using gedit am getting an error as follows,
hduser@arul-java ~ $ gedit ~/.bashrc
(...
2
votes
1
answer
580
views
Moving hadoop folder into /usr/local/hadoop
I want to move the Hadoop installation to the /usr/local/hadoop directory, but it says-
mv: cannot move ‘/home/pg/Desktop/hadoop-2.7.1’ to ‘/user/local/hadoop’: No such file or directory
0
votes
2
answers
2k
views
Which IP to use for localhost in Ubuntu virtual box- installing hadoop
I am trying to install hadoop single node on a Ubuntu 14 Virtual machine( on VMWare) on a windows 7 host.
One of the steps says :
Get IP address using:
ifconfig
You get xxx.xxx.xxx.xxx
Edit /...
1
vote
1
answer
4k
views
I forgot my localhost root password
I installed Hadoop on my machine.
To start it, I was logging in as the user named hduser.
I connected to the ssh port using ssh localhost command.
Then I went to the bin folder of hadoop to start ...