9
votes
How to undo an ssh-copy-id?
If you have done a ssh-copy-id like:
remote='user@machine'
ssh-copy-id -i $remote
So you can access this remote machine without using a password:
ssh $remote
To undo it programmatically, you can ...
- 91
8
votes
"ssh localhost " command is showing 20 bad configuration options
Open the /etc/ssh/ssh_config and remove all the listed options. They are supposed to be part of the sshd_config (the server configuration), not the client one.
- 6,615
5
votes
Where in linux file system can i see files of Hadoop HDFS?
You can use hdfs fsck utility to locate the name of the block and then you can manually find it in the local filesystem:
$ echo "Hello world" >> test.txt
$ hdfs dfs -put test.txt /tmp/
$ hdfs ...
- 151
4
votes
"Command not found" message while executing "hadoop namenode -format"
In your ~/.bashrc, instead of writing
export HADOOP_INSTALL=/usr/local/hadoop
change it to
export HADOOP_INSTALL=/usr/local/hadoop/bin/hadoop
I was facing the same issue for a long time. This ...
4
votes
Accepted
What is a good way to create a low cost Ubuntu based Cassandra or Hadoop Cluster in my house?
A number of companies now produce small "laptop in a box" PCs that can be used as small scale internal servers running Ubuntu, etc. A quick check reveals that the Intel NUC, Gigabyte BRIX, ASUS Vivo, ...
- 593
3
votes
Unable to find hadoop-env.sh in hadoop folder
The usual location for the configuration and environment is in /etc/hadoop. Example: hadoop-2.5.1//etc/hadoop/hadoop-env.sh
- 131
3
votes
-bash: export: not a valid identifier
Probably your .bashrc or .profile is messedup.
Your gedit will not start if it is not in path and since your path is empty it is not in the path.
You should first establish a good enough path.
...
- 181
3
votes
Accepted
root@localhost's password: localhost: Permission denied, please try again
Source and destination users are different, and that should be the reason of this. Would suggest you trying to start Hadoop daemons without sudo. ssh is actually from one user@host to another user@...
- 146
3
votes
Accepted
Error while copying file to HDFS in hadoop ecosystem
Executive summary:
It seems like you are trying to copy the files in non-existed HDFS folder
Detailed Answer:
There is a huge difference between HDFS file-system and regular file-system.
HDFS file-...
- 13.2k
3
votes
SSH without password only works one time
You should take a look at this answer which describes configuring ssh for your user (editing ~/.ssh/config) and other details.
The steps are:
Generate your ssh key.
Add the Host to your ~/.ssh/...
- 11.3k
3
votes
Accepted
Why are Hadoop and Spark not in the official Ubuntu repositories?
Both Hadoop and Spark were dropped from Debian years ago, mostly due to a lack of volunteer interest in maintaining those packages. Ubuntu gets most of its deb packages from Debian, so they were ...
- 62.4k
2
votes
"Command not found" message while executing "hadoop namenode -format"
You can try
/usr/local/hadoop/bin/hadoop namenode -format
If you want to use hadoop command without specifying the path you can edit your ~/.bashrc file. If you have alredy done it, reopen your ...
- 235
2
votes
Accepted
Why my .bashrc has error?
As far I can see your lines are correct except two:
export $PATH=$PATH:$HADOOP_INSTALL/bin
export $PATH=$PATH:$HADOOP_INSTALL/sbin
should be:
export PATH=$PATH:$HADOOP_INSTALL/bin
export PATH=$PATH:...
- 33.4k
2
votes
Accepted
How to resolve UnsatisfiedLinkError installing Spark on Ubuntu 16.04?
Check your error log file in [Spark directory]/logs/error.log. It will have some more information.
looks like you are trying to install 32-bit software on a 64-bit system and do not have the ...
- 300k
2
votes
Permission denied on start-dfs.sh
I am able to resolve the issue using below commands.
The following commands are used for generating a key value pair using SSH. Copy the public keys form id_rsa.pub to authorized_keys, and provide ...
- 151
2
votes
moving hadoop installation into /usr/local/hadoop
You want to move the Hadoop installation to the /usr/local/hadoop directory. So, we should create the directory first:
hduser@Soumitra-PC:~$ sudo mkdir -p /usr/local/hadoop
[sudo] password for hduser:...
2
votes
Accepted
Editing the ~/.bashrc file for hadoop installation
To troubleshoot you should first run the command:
ls -l /home/hduser/.bashrc
It will display:
-rw-r--r-- 1 hduser 186 Jun 27 15:27 /home/hduser/.bashrc
The first column (-rw-r--r--) lists ...
- 3,316
2
votes
Accepted
namenode not running... i have already tried sudo chown -R username /usr/local/hadoop/ please help
Well, first the script tells you that it is deprecated and you should use the scripts start-dfs.sh and start-yarn.sh.
Second, you are starting the services as user divyeshlad, who does not have ...
- 6,223
2
votes
How to access pyspark in anaconda installed in another user?
Make sure user2 has SPARK_HOME environment variable configured if not, set it. If you have set SPARK_HOME for all users should, accessing SPARK_HOME should not be issue for user2.
Once SPARK_HOME is ...
- 21
2
votes
Accepted
Avoid entering password when launching Hadoop in Pseudo-distributed mode
Solved my problem using the steps described in this S.O. answer. Basically, do:
ssh-keygen -t rsa -P ""
cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys
ssh localhost
then
start-dfs....
- 14.4k
2
votes
Avoid entering password when launching Hadoop in Pseudo-distributed mode
I would also try the following (if you have an hdfs user):
sudo -u hdfs start-dfs.sh service-name
This way you would enter password once, but consequent calls should be cached.
- 219
2
votes
Accepted
Installing Hadoop in Ubuntu 16.04
When I type in whereis java it gives: java: /usr/local/java /usr/share/java /home/hugh/.sdkman/candidates/java/9.0.1-oracle/bin/java
The goal of Step 3 is to configure your Java home.
Based on your ...
- 4,898
2
votes
Cannot open display error when I run sudo gedit on Ubuntu 14.04.5
You should never use normal sudo to start graphical applications as root. For a complete explanation of this read the answers to this question: Why should users never use normal sudo to start ...
- 115k
2
votes
Accepted
FAILED: HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
I think Hive expect to find Derby database in current working directory. Hive will create the database, because create=true in javax.jdo.option.ConnectionURL property, but will not initialize schema ...
- 136
1
vote
Unable to upload directory to hdfs. `/usr/local/tmp/': No such file or directory
You say your local files are under (path: /home/shradha/Downloads) but you are trying to upload /shradha/Downloads.
check local directory existence:
ls -al /home/shradha/Downloads/states
if you can ...
- 219
1
vote
echo $HADOOP_HOME is blank
This may occur due to following reasons:
You are not logging into your user that you have created for Hadoop.
You have not specified the path for Hadoop home path all-together.
The Variable that you ...
- 33
1
vote
Accepted
Type '--2016-08-04' is not known on line 1 in source list /etc/apt/sources.list
Open /etc/apt/sources.list in an editor as a root (e.g. type in terminal gksudo gedit /etc/apt/sources.list), and replace all its content with the following
###########################################...
- 3,702
1
vote
"Command not found" message while executing "hadoop namenode -format"
If path is correct and still facing the error
then
after adding path variables in ~/.bashrc
run this command source ~/.bashrc
- 11
1
vote
I tried installing the ssh_package into my ubuntu 16.04 but it is giving me this error. I am running ubuntu on VMWare workstation pro
Instead of running that:
sudo dpkg -r openssh-client
sudo apt-get install ssh
- 3,831
1
vote
move files in a hadoop dir with specified number of files
What about using a script like this:
#!/bin/bash
Source="/where/they/are/now/*"
Destination="/where/they/will/go"
while true; do
count=0
for file in $Source; do
if [ $((count++)) -eq 100 ];...
- 1,022
Only top scored, non community-wiki answers of a minimum length are eligible