Ask Ubuntu
Ask Ubuntu

Questions tagged [hadoop]

Apache Hadoop is a software framework that supports data-intensive distributed applications under a free license. It enables applications to work with thousands of computational independent computers and petabytes of data. Hadoop was derived from Google's MapReduce and Google File System (GFS) papers.

Hadoop is a top-level Apache project being built and used by a global community of contributors, written in the Java programming language. Yahoo! has been the largest contributor to the project, and uses Hadoop extensively across its businesses.

Hadoop was created by Doug Cutting, who named it after his son's toy elephant. It was originally developed to support distribution for the Nutch search engine project.

IMG:

205 questions
29
votes
2 answers

How to undo an ssh-copy-id?

I have a 2 node hadoop cluster. I ran this command on the master: $ssh-copy-id -i /home/hadoop/.ssh/id_rsa.pub hadoop@192.168.1.1 How can I undo this? I would actually like to reassign the key. 192.168.1.1 is the slave.
Alex Gordon
  • 894
26
votes
5 answers

How to install Hadoop?

I am trying to install Hadoop in Ubuntu 12.04 version. Following the instructions from http://michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/, I installed java-6-openjdk from Ubuntu software-center. I have set…
Anitha
  • 321
14
votes
1 answer

How do I find out the version of Zookeeper I am running?

I have an Ubuntu-12.04 VM running on my laptop.I have installed zookeeper on it using the command sudo apt-get install zookeeper Now for traditional Ubuntu programs I check the version using --version command line option. For example gcc…
liv2hak
  • 267
11
votes
3 answers

Can't SSH localhost

I can't SSH localhost. I have installed OpenSSH server as well on my Ubuntu 12.04 LTS (32 bit). connected to net (ip : 192.168.1.2) installed hadoop, running hadoop for local machine. Here's what I get : root@kunal-Extensa-4620:/# ssh -vvv…
ASCIIbetical
  • 897
10
votes
2 answers

"Permission denied" errors whe starting a single node cluster in Hadoop

I'm working in Ubuntu 10.10 and am trying to start a single node cluster in Hadoop. hadoop@abraham-Dimension-3000:/usr/local/hadoop$ bin/start-all.sh mkdir: cannot create directory `/usr/local/hadoop/bin/../logs': Permission denied starting…
ATMathew
  • 1,266
9
votes
5 answers

Where in linux file system can i see files of Hadoop HDFS?

I am a just data analyst hails from non cs background (not a hardcore system programmer) working on linux OS. While exploiting analysis using hadoop, a question was coming to my mind that ... / is a root under which all files of your system exists…
bioinformatician
  • 211
8
votes
2 answers

should i install the .deb or .tar.gz?

Please pardon the elementary question. I'm going through this tutorial. I would like to install hadoop on vm ubuntu 64 bit. I would like to know whether I should be downloading the .deb file or the tar.gz file from here? What is the difference…
Alex Gordon
  • 894
7
votes
2 answers

how to allow VMs and host ping each other?

I'm trying to set up a few VMs of ubuntu on my windows 7 machine. I need the machines to be able to ping each other and I would like to assign them a FQDN. The purpose of this project is to set up a hadoop cluster. I'm following the instructions…
Alex Gordon
  • 894
6
votes
2 answers

hadoop : datanode not running?

I configured and deployed hadoop for single node setup via this tutorial. Everything deployed fine, but when I do jps for have a look on active processes, datanode is not shown. I manually tried to start datanode by going to $HADOOP_HOME/bin :…
ASCIIbetical
  • 897
5
votes
3 answers

I can't connect to Hadoop port 9000

So telnet actually is working, I mean telnet localhost 25 is connecting; but telnet localhost or telnet localhost 9000 got such result: Trying 127.0.0.1... telnet: Unable to connect to remote host: Connection refused nmap results: $ nmap…
DaunnC
  • 153
5
votes
4 answers

Unable to find hadoop-env.sh in hadoop folder

I am trying to install hadoop on Ubuntu 12.04. I followed the following article. http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/ I have installed hadoop as per the instruction given in this tutorial. I have…
KurioZ7
  • 201
5
votes
1 answer

FAILED: HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

I'm trying to run Hive 3.1 with Hadoop 3.0. Following is my system configuration: Ubuntu 18.04.1 LTS Hadoop version 3.0.3 Hive 3.1.0 Derby 10.14.2 When I execute the show tables; query I get the following error. FAILED: HiveException…
SriniShine
  • 183
4
votes
1 answer

"ssh localhost " command is showing 20 bad configuration options

hduser1@archit-HP-Notebook:~$ ssh localhost /etc/ssh/ssh_config: line 11: Bad configuration option: hostkey /etc/ssh/ssh_config: line 12: Bad configuration option: hostkey /etc/ssh/ssh_config: line 13: Bad configuration option:…
Archit Jain
  • 41
4
votes
3 answers

Avoid entering password when launching Hadoop in Pseudo-distributed mode

So I am running Hadoop in pseudo distributed mode on my laptop, to learn about these things. I am setting the thing up as explained in the manual. I can start hadoop, no problem, but the launch process is inconvenient. See, to launch the thing, one…
user2413
  • 14,957
4
votes
1 answer

Ubuntu 16.04 kills session when resource usage is extremely high

I'm working on project in hadoop on my computer. When I start some job consuming most of the available resources (100% CPU, high RAM usage) 'something' kills my user's session and all its processes. syslog: Jun 8 21:38:46 michalo-desktop…
michalo2882
  • 41
1
2 3
…
13 14