Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11985

Improve Solaris support in Hadoop

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.7.0
    • Fix Version/s: None
    • Component/s: build, conf
    • Labels:
    • Environment:

      Solaris x86, Solaris sparc

    • Tags:
      Solaris Native C x86 sparc

      Description

      At present the Hadoop native components aren't fully supported on Solaris primarily due to differences between Linux and Solaris. This top-level task will be used to group together both existing and new issues related to this work. A second goal is to improve Hadoop performance on Solaris wherever possible.

      Steve Loughran suggested a top-level JIRA was the best way to manage the work.

        Attachments

          Issue Links

          1.
          NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values Sub-task Resolved Martin Walsh  
          2.
          Set up a nightly Jenkins build on Solaris. Sub-task Open Unassigned  
          3.
          Fix FIONREAD #include on Solaris Sub-task Resolved Alan Burlison  
          4.
          Native compilation fails on Solaris due to use of getgrouplist function. Sub-task Resolved Malcolm Kavalsky

          0%

          Original Estimate - 168h
          Remaining Estimate - 168h
          5.
          Native compilation fails on Solaris due to use of syscall function. Sub-task Open Malcolm Kavalsky

          0%

          Original Estimate - 168h
          Remaining Estimate - 168h
          6.
          Native code needs to be built to match the 32/64 bitness of the JVM Sub-task Resolved Alan Burlison  
          7.
          Solaris does not support RLIMIT_MEMLOCK as in Linux Sub-task Resolved Alan Burlison

          0%

          Original Estimate - 24h
          Remaining Estimate - 24h
          8.
          JNI build should use default cmake FindJNI.cmake Sub-task Resolved Alan Burlison  
          9.
          CMake CMAKE_C_FLAGS are non-portable Sub-task Resolved Alan Burlison  
          10.
          Investigate providing SPARC hardware-accelerated CRC32 code Sub-task Open Unassigned  
          11.
          Investigate JNI for improving byte array comparison performance Sub-task Open Alan Burlison  
          12.
          Consolidate all of the cmake extensions in one directory Sub-task Resolved Alan Burlison  
          13.
          Migrate Hadoop Pipes native build to new CMake framework Sub-task Resolved Alan Burlison  
          14.
          Make hadoop-common-project Native code -Wall-clean Sub-task Resolved Alan Burlison  
          15.
          hadoop-common's JNIFlags.cmake is redundant and can be removed Sub-task Resolved Alan Burlison  
          16.
          Remove unused Linux-specific constants in NativeIO Sub-task Resolved Martin Walsh  
          17.
          Surefire needs to make sure the JVMs it fires up fit within the memory available Sub-task Patch Available Alan Burlison  
          18.
          Make DomainSocket.close() portable Sub-task Patch Available Alan Burlison  
          19.
          DomainSocket: Solaris does not support timeouts on AF_UNIX sockets Sub-task Patch Available Alan Burlison  
          20.
          Improve versioning and compatibility support in native library for downstream hadoop-common users. Sub-task Open Alan Burlison  
          21.
          Hadoop needs a SysInfo class for Solaris Sub-task Open Alan Burlison  
          22.
          ShellBasedIdMapping needs suport for Solaris Sub-task Resolved Alan Burlison  
          23.
          Sundry symlink problems on Solaris Sub-task Open Alan Burlison  
          24.
          Misaligned memory accesses in CRC32 native code causes coredumps Sub-task Open Alan Burlison  
          25.
          Misuse of sun.misc.Unsafe by org.apache.hadoop.io.FastByteComparisons causes misaligned memory access coredumps Sub-task Open Alan Burlison  
          26.
          Misuse of sun.misc.Unsafe by org.apache.hadoop.io.FastByteComparisons$LexicographicalComparerHolder$UnsafeComparer.compareTo causes misaligned memory access coredumps Sub-task Open Alan Burlison  
          27.
          Minor cleanups needed for CMake 3.X Sub-task Resolved Alan Burlison  
          28.
          Checks for SPARC architecture need to include 64-bit SPARC Sub-task Resolved Alan Burlison  
          29.
          lz4.c does not detect 64-bit mode properly Sub-task Resolved Alan Burlison  

            Activity

              People

              • Assignee:
                alanburlison Alan Burlison
                Reporter:
                alanburlison Alan Burlison
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - 360h
                  360h
                  Remaining:
                  Remaining Estimate - 360h
                  360h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified