Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-17193

Compile Hadoop on Windows natively

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.1.3
    • Fix Version/s: None
    • Component/s: build, native
    • Labels:
      None
    • Environment:

      Windows 10

    • Target Version/s:

      Description

      Hadoop compiles flawlessly on Linux whereas, one has to hack around to compile it on Windows. The reason is obvious, there's a very large proportion of C/C++ codebase that's written in a platform specific manner that's amenable to Linux.

      Thus, compiling Hadoop on Windows involves using the MinGW or the Cygwin toolchains. This third party dependency might be seen as a risk for those who want to deploy Hadoop on Windows nodes in production. The primary reason for this is that the MinGW/Cygwin runtimes on Windows don't produce core dumps. Whereas the Visual C++ runtime is able to do so, which greatly enables debugging.

      Another reason to use the Visual C++ compiler for compiling Hadoop on Windows is that Visual C++ is native to Windows and one will have access to a large collection of tools that can be used for instrumentation, telemetry and fine-tuning the settings of Hadoop to run better on Windows.

      Modern C++ has a plethora of APIs that one can use to write safe, cross-platform code, which can be leveraged to refactor the existing Linux specific code, thereby enabling compilation on Windows using the native Visual C++ compiler.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                gautham Gautham Banasandra
                Reporter:
                gautham Gautham Banasandra
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated: