Apache Hadoop is a software framework that supports data-intensive distributed applications under a free license. It enables applications to work with thousands of computational independent computers and petabytes of data. Hadoop was derived from Google's MapReduce and Google File System (GFS) ...

learn more… | top users | synonyms

0
votes
0answers
23 views

yarn-env.sh : command not found

I have Hadoop 2.7.2 installed on Ubuntu 16.04. When I run the command: start-yarn.sh It gives the following output: starting yarn daemons /usr/local/hadoop-2.7.2/etc/hadoop/yarn-env.sh: line 122: ...
-1
votes
1answer
36 views

I tried installing the ssh_package into my ubuntu 16.04 but it is giving me this error. I am running ubuntu on VMWare workstation pro [closed]

rmehra@ubuntu:~/Desktop/ssh_package$ sudo dpkg -i openssh-client_1%3a5.3p1-3ubuntu7_i386.deb (Reading database ... 172254 files and directories currently installed.) Preparing to unpack openssh-...
1
vote
2answers
36 views

move files in a hadoop dir with specified number of files

I wish to move files in a hadoop dir in a timely manner. The hadoop dir contains 1000 files with the same extension. I wish to move 100 of them every 10 minutes. I can setup a cron job to move the ...
0
votes
1answer
97 views

-bash: export: not a valid identifier

Whenever opening the terminal, I always end up with this same error. -bash: export: `/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin:/usr/...
0
votes
0answers
36 views

Errors when running ./hive command

Hi all, When I'm starting my HIVE with ./hive in the bin directory I'm getting a lot of errors like below. Can anybody help me? Caused by: java.lang.RuntimeException: Unable to instantiate org....
0
votes
0answers
31 views

No passwd entry for hadoop [on hold]

I have install Hadoop 2.7.2 in /usr/local directory. When I run hadoop version it gives the output correctly as hadoop 2.7.2 Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r ...
1
vote
0answers
28 views

Unable to start-hbase.sh (throws error)

When I start-hbase.sh, I get the following error localhost: starting zookeeper, logging to /usr/lib/HBase/bin/../logs/hbase-hduser-zookeeper-nkhl.out localhost: java.io.FileNotFoundException: /home/...
0
votes
0answers
11 views

Hive NOt getting installed

I have my hadoop installed in the directory /usr/local/hadoop. Now I'm installing the hive in /usr/lib/hive and giving the below path in my .bashrc: export JAVA_HOME=/usr/local/java export ...
0
votes
0answers
107 views

Install Cloudera and Hadoop on ubuntu 16

I am trying to find online materials to find the steps or guideline on installing cloudera and Hadoop. I found this and this page but none of them helped. If you know please help me on this as I am ...
0
votes
0answers
22 views

Hadoop Namenode Format Problem

I tried installing Hadoop on my Ubuntu 14.04 OS.Here are the steps I followed for installation: Downloaded JDK,HADOOP and placed it in usr/local path. Followed the other steps and I edited the ...
0
votes
0answers
79 views

start-dfs.sh Error in Ubuntu 14.04 and HDFS 2.7.2

I am able to format HDFS nameformat -format and print hadoop version.. hduser@madhavan-Lenovo-G500:/usr$ hadoop version Hadoop 2.7.2 Subversion https://git-wip-us.apache.org/repos/asf/hadoop.git -r ...
0
votes
1answer
182 views

HADOOP commands not working any where in ubuntu 16.04

Hadoop commands not working any where commands working hadoop/bin directory only once abserve first I have tried outside hadoop/bin second time I had done with in bin
0
votes
1answer
153 views

Hadoop and Eclipse - Server IPC version 9 cannot communicate with client version 4

I'm working on Ubuntu and using Eclipse to work with Hadoop. After setting the Run configuration, a message says Incomplete HDFS URI, no host UPDATE: found the solution! but after running the program,...
0
votes
0answers
18 views

Hadoop- Jobtracker and Tasktracker not works

I use CDH5.4, I want to start the JobTracker and TaskTracker with this commands: sudo service hadoop-0.20-mapreduce-jobtracker start sudo service hadoop-0.20-mapreduce-tasktracker start I got this ...
0
votes
1answer
63 views

Problem when starting a single node of Hadoop cluster in Ubuntu

I am new to Hadoop and I was going through the installation process mentioned here. I have installed hadoop-1.2.1 on my system. After configuring everything when I tried to start a single node cluster ...
0
votes
1answer
60 views

Hadoop - Error on binding to port 9000

I've been attempting to setup Hadoop on AWS for a few days now. I am on fresh EC2 Ubuntu instances with no other software installed apart from Java. I run these commands: ubuntu@ip-172-30-0-117:/...
0
votes
2answers
140 views

Issues with disabling iPv6 on Ubuntu 14.0.4

To disable iPv6 on Ubuntu 14.0.4 (LTS) I added these lines in sysctl.conf file: net.ipv6.conf.all.disable_ipv6 = 1 net.ipv6.conf.default.disable_ipv6 = 1 net.ipv6.conf.lo.disable_ipv6 = 1 Since I ...
0
votes
0answers
19 views

Connect to Hadoop installed on Linux VM

I've done the following steps: Installed Hadoop on a Linux VM placed in my Windows 7 laptop. I am trying to connect the namenode of the above Linux VM through a web browser. The web browser through ...
0
votes
0answers
49 views

SSH permission denied from Linux Ubuntu to Windows using keys

I am a beginner in hadoop, and I am trying to set-up hadoop cluster in fully distribution mode. My master is windows and my slave machine is Linux running on VM. I have created SSH keys and in both ...
0
votes
0answers
23 views

Given dml mapping how to do pig transformation to perform filtering on raw data

Given dml mapping how to do pig transformation to perform filtering on raw data DML Mapping string('\x01')record_status= NULL(""); string('\x01')check_sum= ...
1
vote
1answer
23 views

adduser fails when relating hadoop-plugin to hbase-master

I'm trying to set up Hbase in juju using the following charm: https://jujucharms.com/u/bigdata-dev/apache-hbase/trusty/19 I receive an error when I try to add a relation between the hadoop-plugin and ...
0
votes
2answers
141 views

Hadoop 2.6 start-dfs.sh errors on Centos 6.7 [closed]

I use this tutorial to install Hadoop 2.6 on Centos 6.7 with Java 1.8.0_72 and everything goes well before execution of start-dfs.sh from Hadoop-home/sbin/srart-dfs.sh. Below is the output: [hadoop@...
0
votes
0answers
15 views

Warning while deleting/creating folder from/in hadoop

When i create or delete a directory of file following warning appears. hduser@anand-HP-Pavilion-15:/home/anand$ hadoop fs -mkdir /MRjobs 16/02/25 12:50:27 WARN util.NativeCodeLoader: Unable to load ...
0
votes
1answer
83 views

Jobtracker and tasktracker not showing

I am using hadoop 2.7.1 on Ubuntu 14.04, when I run hadoop and use jps, the following processes are shown: Resourcemanager Namenode Datanode Jps SecondaryNameNode NodeManager Here job tracker and ...
0
votes
0answers
182 views

I getting errors for this command /usr/local/hadoop/sbin$ start-all.sh

I getting errors for this command /usr/local/hadoop/sbin$ start-all.sh.. Errors are: localhost: /usr/local/hadoop/hadoop-2.7.1/sbin/yarn-daemon.sh: line 130: /usr/local/hadoop/hadoop-2.7.1/logs/yarn-...
0
votes
0answers
62 views

python mapreduce job failed - log4j not configured?

I have hadoop 2.7.1 installed on Ubuntu 15.10 as a single node cluster and I'm trying to run a python mapreduce wordcount fount in http://www.michael-noll.com/tutorials/writing-an-hadoop-mapreduce-...
0
votes
0answers
67 views

ssh local ipaddress

I am trying to setup a multi-node hadoop cluster on my local laptop. I have 2 VMs running ubuntu. I am able to successfully run hadoop on each one of these VMs as single-node. To configure multi-...
0
votes
1answer
23 views

How to use amazon instances' public IPs in hadoop configuration?

I was trying to configure Hadoop by using the public IPs of amazon instances instead of their intra-network IPs as my aim is to create a hybrid cluster i.e. Cloud + Local machine cluster. Although all ...
0
votes
0answers
29 views

Hadoop - Add new segment to Hawq cluster

I installed hawq on Hadoop using ambari. in the installation step , I chose the Hawq master host with 2 segments , and all was working fine. then later, I added a new hawq segment using ambari , ...
0
votes
1answer
320 views

Permission denied on start-dfs.sh

I am trying to create a 2-node Hadoop cluster, following this guide. The one node is my Ubuntu laptop and the slave node, the 2nd one, is a virtualbox that also runs Ubuntu. I am doing in the master ...
0
votes
0answers
93 views

can't run MapReduce jobs after re-installing Hadoop 2.7.1 on ubuntu 15.10 (single-node)

I have a single-node installation of Hadoop 2.7.1, which I had to reinstall recently. Installation was done with http://chaalpritam.blogspot.co.uk/2015/05/hadoop-270-single-node-cluster-setup-on.html. ...
0
votes
0answers
25 views

How do I write IP of the machine in hadoop datanode log file?

I want to write the IP and Hostname or Username of the machine along with all the other information in the hadoop datanode log file. I know I have to do this by modifying the log4j.properties file in ...
1
vote
0answers
283 views

Getting error while PIG installation - “The program 'pig' is currently not installed”. Please help

Error message says: The program 'pig' is currently not installed. You can install it by typing: sudo apt-get install bsdgames Hadoop single node cluster is already installed in my Ubuntu machine. Pig ...
-1
votes
1answer
31 views

Ubuntu and Windows [duplicate]

Currently I am working in Ubuntu. Big Data. I do not have Windows OS in my system. i am going to install hadoop in Ubuntu. But I don't know whether windows is compulsory to have along with ubuntu. Can ...
0
votes
0answers
220 views

issues while installing spark from source in linode machine

I downloaded the new spark1.5.2. copied it to the linode machine. Installed Java and scala. Finally sbt/sbt clean assembly -mem 512 Please find logs for the same. [warn] Run 'evicted' to see ...
0
votes
0answers
57 views

Permission issues in Ubuntu 15.10

Following this article I changed the permission for the user hduser and ever since then, I've not been able to log into my primary user profile, which is called nikhilken. When I put in the right ...
0
votes
1answer
101 views

Problem with cluster installation with cloudera manager

I am installing Cloudera Manager in my system(14.04/64b). While installing at the final step, before finish installation I got some ERRORs in validation as shown below, errors in above page are, ...
0
votes
1answer
152 views

Hadoop Installation Problem

This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh 15/11/07 16:01:44 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes ...
0
votes
0answers
289 views

How to completely uninstall hadoop after installing in pseudo distributed mode in Ubuntu 14.04 LTS?

I had installed CDH5 with Mvr1 in ubuntu 14.04 LTS (single node) in pseudo-distributed mode using this tutorial http://www.cloudera.com/content/www/en-us/documentation/enterprise/latest/topics/...
1
vote
1answer
147 views

Hadoop & Spark - why no Ubuntu packages?

I want to install Hadoop & Spark using aptitude, not the lengthy and complicated process described in How to install Hadoop?. hadoop-ubuntu appears to have been dead for almost 2 years. I wonder ...
1
vote
1answer
249 views

gedit - cannot open display

I am using Ubuntu 14.04 LTS, and for installing Hadoop I created hadoop user as hduser. When I try to open ~/.bashrc using gedit am getting an error as follows, hduser@arul-java ~ $ gedit ~/.bashrc (...
2
votes
1answer
97 views

Moving hadoop folder into /usr/local/hadoop

I want to move the Hadoop installation to the /usr/local/hadoop directory, but it says- mv: cannot move ‘/home/pg/Desktop/hadoop-2.7.1’ to ‘/user/local/hadoop’: No such file or directory
0
votes
2answers
106 views

Which IP to use for localhost in Ubuntu virtual box- installing hadoop

I am trying to install hadoop single node on a Ubuntu 14 Virtual machine( on VMWare) on a windows 7 host. One of the steps says : Get IP address using: ifconfig You get xxx.xxx.xxx.xxx Edit ...
0
votes
0answers
29 views

Change data placement strategy in Hadoop 1.2.1 multi-node cluster

I know that Hadoop distributes data blocks fairly on each node, for example 3 blocks for each node. By this, I do not mean the replicas of a block, I mean the number of blocks that Hadoop distributes ...
0
votes
0answers
60 views

Unable to access Ubuntu installed in vmplayer from windows10 desktop

I have installed Ubuntu 12.04 in a VMplayer and it is available in my Windows10 desktop. I'm practicing Hadoop big data and my purpose is to connect a ETL tool Pentaho with the Hadoop installed in ...
0
votes
0answers
64 views

JAVA_HOME not visible in Hadoop

I have trouble to understand what’s going on during Hadoop installation on Ubuntu. I’m following instructions from: Install Hadoop on Ubuntu Step by step got to the last step “Starting Hadoop” ...
0
votes
0answers
28 views

lost all files in home directory while creating a new user for hadoop

I tried to install Hadoop in Ubuntu 12.04 created a new user for hadoop and then I lost all the data that I had in my home folder but there is no change in the amount of memory(secondary storage) ...
0
votes
0answers
27 views

Slave DataNode not doing work despite looking up and healthy

I have a two node Haoop 2.7.1 installation on Ubuntu 12.04 LTS. All the demons are up and running after sbin/start-dfs.sh && sbin/start-yarn.sh as confirmed by jps. However, only the Master ...
1
vote
1answer
521 views

I forgot my localhost root password

I installed Hadoop on my machine. To start it, I was logging in as the user named hduser. I connected to the ssh port using ssh localhost command. Then I went to the bin folder of hadoop to start ...
0
votes
1answer
77 views

Authentication failure in Hadoop

I have already installed Hadoop and already added user as hduser to group, but I have forgotten the password of the user i.e. hduser. When running su hduser from the terminal I get the result: su: ...