Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13427

Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()}

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: fs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      We're cleaning up Hive and Spark's use of FileSystem.exists, because it is often the case we see code of exists+open, exists+delete, when the exists probe is needless. Against object stores, expensive needless.

      Hadoop can set an example here by stripping them out. It will also show where there are opportunities to optimise things better and/or improve reporting.

        Attachments

        1. HADOOP-13427.009.patch
          60 kB
          Mingliang Liu
        2. HADOOP-13427.009.patch
          60 kB
          Mingliang Liu
        3. HADOOP-13427.008.patch
          60 kB
          Mingliang Liu
        4. HADOOP-13427.008.patch
          60 kB
          Mingliang Liu
        5. HADOOP-13427.007.patch
          60 kB
          Steve Loughran
        6. HADOOP-13427.006.patch
          60 kB
          Mingliang Liu
        7. HADOOP-13427.005.patch
          60 kB
          Mingliang Liu
        8. HADOOP-13427.004.patch
          60 kB
          Mingliang Liu
        9. HADOOP-13427.003.patch
          56 kB
          Mingliang Liu
        10. HADOOP-13427-002.patch
          53 kB
          Steve Loughran
        11. HADOOP-13427-001.patch
          53 kB
          Steve Loughran

        Issue Links

          Activity

            People

            • Assignee:
              liuml07 Mingliang Liu
              Reporter:
              stevel@apache.org Steve Loughran

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment