Hadoop Common
  1. Hadoop Common
  2. HADOOP-6330

Integrating IBM General Parallel File System implementation of Hadoop Filesystem interface

    Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: fs
    • Labels:
      None
    • Environment:

      Where GPFS is supported

      Description

      GPFS is a high performance parallel file system for GNU/Linux clusters. This patch contains the implementation of the Hadoop Filesystem Interface. There is dependency on the availability of GPFS on the host where the JNI implementation can be built.
      The patch consists of fs/gpfs classes and the JNI module in c++/libgpfs.
      http://www-03.ibm.com/systems/clusters/software/gpfs/index.html

        Activity

        Reshu Jain created issue -
        Hide
        Steve Loughran added a comment -

        It would be very tricky to integrate this with the current release process, as now you'd need a GPFS FS on the 32 bit and 64 bit machines you want to build on. It would work if the GPFS binding was a separate artifact released in sync (or shortly after) the main Hadoop releases

        Someone with a GPFS cluster also needs to set up Hudson to run CI builds and tests on HDFS to catch regressions that break GPFS support sooner rather than later. Its the testing of these third-party filestores that is very weak right now.

        Show
        Steve Loughran added a comment - It would be very tricky to integrate this with the current release process, as now you'd need a GPFS FS on the 32 bit and 64 bit machines you want to build on. It would work if the GPFS binding was a separate artifact released in sync (or shortly after) the main Hadoop releases Someone with a GPFS cluster also needs to set up Hudson to run CI builds and tests on HDFS to catch regressions that break GPFS support sooner rather than later. Its the testing of these third-party filestores that is very weak right now.
        Owen O'Malley made changes -
        Field Original Value New Value
        Fix Version/s 0.18.3 [ 12313494 ]
        Hide
        Allen Wittenauer added a comment -

        Supplemental FileSystem/FileContext code comes from their respective projects instead of Hadoop now.

        Show
        Allen Wittenauer added a comment - Supplemental FileSystem/FileContext code comes from their respective projects instead of Hadoop now.
        Allen Wittenauer made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Won't Fix [ 2 ]

          People

          • Assignee:
            Unassigned
            Reporter:
            Reshu Jain
          • Votes:
            0 Vote for this issue
            Watchers:
            16 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development