Avro
  1. Avro
  2. AVRO-1539

Add FileSystem-based FsInput Constructor

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: 1.7.6
    • Fix Version/s: 1.8.0
    • Component/s: java
    • Labels:
      None

      Description

      The current FsInput constructor uses a Configuration object to retrieve the FileSystem. It would be convenient to have an additional constructor that directly took a FileSystem for instances where consuming code already has that available.

      1. AVRO-1539.patch
        1 kB
        Allan Shoup
      2. AVRO-1539b.patch
        1 kB
        Allan Shoup
      3. AVRO-1539c.patch
        6 kB
        Allan Shoup

        Activity

        Hide
        Allan Shoup added a comment -

        Any objections to this trivial patch?

        Show
        Allan Shoup added a comment - Any objections to this trivial patch?
        Hide
        Doug Cutting added a comment -

        This looks like a reasonable addition, but you should continue to call path.getFileSystem(conf) to get the FileSystem instance so that the path is still used to create the correct filesystem.

        Show
        Doug Cutting added a comment - This looks like a reasonable addition, but you should continue to call path.getFileSystem(conf) to get the FileSystem instance so that the path is still used to create the correct filesystem.
        Hide
        Allan Shoup added a comment -

        Good point. Attaching AVRO-1539b.patch.

        Show
        Allan Shoup added a comment - Good point. Attaching AVRO-1539b.patch .
        Hide
        Doug Cutting added a comment -

        Looks good. We should probably also add a test that uses the new constructor.

        Show
        Doug Cutting added a comment - Looks good. We should probably also add a test that uses the new constructor.
        Hide
        Allan Shoup added a comment -

        Added tests for FsInput. I also included some missing @Override annotations.

        Show
        Allan Shoup added a comment - Added tests for FsInput. I also included some missing @Override annotations.
        Hide
        ASF subversion and git services added a comment -

        Commit 1646365 from Doug Cutting in branch 'avro/trunk'
        [ https://svn.apache.org/r1646365 ]

        AVRO-1539. Java: Add FileSystem based FsInput constructor. Contributed by Allan Shoup.

        Show
        ASF subversion and git services added a comment - Commit 1646365 from Doug Cutting in branch 'avro/trunk' [ https://svn.apache.org/r1646365 ] AVRO-1539 . Java: Add FileSystem based FsInput constructor. Contributed by Allan Shoup.
        Hide
        Doug Cutting added a comment -

        I committed this. Thanks, Allan!

        Show
        Doug Cutting added a comment - I committed this. Thanks, Allan!
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in AvroJava #507 (See https://builds.apache.org/job/AvroJava/507/)
        AVRO-1539. Java: Add FileSystem based FsInput constructor. Contributed by Allan Shoup. (cutting: rev 1646365)

        • /avro/trunk/CHANGES.txt
        • /avro/trunk/lang/java/mapred/pom.xml
        • /avro/trunk/lang/java/mapred/src/main/java/org/apache/avro/mapred/FsInput.java
        • /avro/trunk/lang/java/mapred/src/test/java/org/apache/avro/mapreduce/TestFsInput.java
        Show
        Hudson added a comment - SUCCESS: Integrated in AvroJava #507 (See https://builds.apache.org/job/AvroJava/507/ ) AVRO-1539 . Java: Add FileSystem based FsInput constructor. Contributed by Allan Shoup. (cutting: rev 1646365) /avro/trunk/CHANGES.txt /avro/trunk/lang/java/mapred/pom.xml /avro/trunk/lang/java/mapred/src/main/java/org/apache/avro/mapred/FsInput.java /avro/trunk/lang/java/mapred/src/test/java/org/apache/avro/mapreduce/TestFsInput.java

          People

          • Assignee:
            Allan Shoup
            Reporter:
            Allan Shoup
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development