1

I'm running Hadoop on Ubuntu on a Oracle VM on Windows 7. I'm trying to install Spark on an existing Hadoop installation. For demonstration purposes I do not want to change the existing Hadoop version. Following are the versions of software in my VM.

Ubuntu : 14.04

Java : Java 1.7 and OpenJDK runtime environment (IcedTea 2.5.2)

Hadoop : 2.5.0

I figured out that for Hadoop I need to install Spark 1.6.2. When I try to download Spark 1.6 it doesn't give me the option for Hadoop 2.5 but has options for 2.4 and 2.6. Will Spark 1.6 work on Hadoop 2.4?

I would like to know the appropriate platform requirements for Spark, Scala and Java to install Spark on Hadoop 2.5.

SriniShine
  • 183

1 Answers1

1

You can build spark for any hadoop version by installing maven and follow below steps:

  1. Download spark source code.

  2. Run below commands:

    tar -xvzf spark-1.6.2.tgz

    cd spark-1.6.2

    mvn -Pyarn -Phadoop-2.5 -Dhadoop.version=2.5.0 -DskipTests clean package

karthik
  • 289