Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11425

Compilation failed on Windows WIN32 platform

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 2.2.0, 2.6.0
    • None
    • None
    • HDFS on Linux
      Client on Windows 7 (Win32)

    Description

      Past few days, I am struggling with running my Hadoop application from windows client. and. I started with latest stable build 2.6.0. Installed HDFS on Linux.

      My application is running on Windows which submits mapreduce jobs remotely. By default we don’t have windows distribution for Hadoop, So I need to build Hadoop on win32 platform. This turns out the non-trivial tasks as there are so many issue with windows build on both 2.6.0 and 2.2.0. By default Win32 support is not there.
      I am sharing my efforts, so that It can be used by others struggling with same issue.

      By default the targets to build on Windows platforms are missing in Hadoop 2.6.0 and 2.2.0 and other release.

      So modify

      modify native.sln and native.vcproj (Add targets for win32)
      modify libwinutils.vcproj, winutils.sln and winutils.vcproj (Add targets for win32)

      Sample for *.sln modification
      353FDE506F33}.Release|Win32.ActiveCfg = Release|Win32

      {D94B3BD7-39CC-47A0-AE9A-353FDE506F33}

      .Release|Win32.Build.0 = Release|Win32

      {12131AA7-902E-4A6D-9CE3-043261D22A12}.Release|Win32.ActiveCfg = Release|Win32
      {12131AA7-902E-4A6D-9CE3-043261D22A12}

      .Release|Win32.Build.0 = Release|Win32

      Sample for *.vcproj modification - modify properties with Win32 platform conditional conf.

      <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">

      Critical Issue - In native.vcproj the following file is not compiled.
      This gives linker error.

      <ClCompile Include="src\org\apache\hadoop\io\compress\lz4\lz4hc.c" />

      We also need to document to setup the build environment for windows. As there are several issues creating Windows dev setup for Hadoop. May be we need to documents the steps.

      Steps to set up you windows build environment-

      1. Download and install Microsoft Windows SDK v7.1. From link http://www.microsoft.com/en-in/download/details.aspx?id=8279. Please follow the guidelines provided. You may have to uninstall .Net 4.5 before installing SDK version 7.x
      2. You may get following error File Microsoft.Cpp.props not found, Patch SDK. Visual C++ 2010 SP1 Compiler Update for the Windows SDK 7.1. Download from link http://www.microsoft.com/en-in/download/details.aspx?id=4422
      3. You may also get following fatal error C1083: Cannot open include file: 'ammintrin.h': No such file or directory. This is mIcrosoft SDK bug. Get a patch to download ammintrin.h from https://connect.microsoft.com/VisualStudio/feedback/details/660584/
      4. Now your Windows environment is ready to build. Hope there is no other errors reported on your system.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              jainman Manoj Kumar jain
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - 72h
                  72h
                  Remaining:
                  Remaining Estimate - 72h
                  72h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified