Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-6893

FileContext needs a method to test for existence of a file or directory.

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Invalid
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: fs
    • Labels:
      None

      Description

      FileContext is missing a method to ask if a file or directory exists. Currently the only mechanism is to use getFileStatus, which throws an exception if the file doesn't exist, which I don't think is an appropriate use of that method.

        Activity

        Hide
        boky01 Andras Bokor added a comment -

        I read through HADOOP-6678.
        It seems there was a consensus in community that check existence of a file or directory is preferred through filestatus which is make sense.
        This did not change in the last 7 years.

        Show
        boky01 Andras Bokor added a comment - I read through HADOOP-6678 . It seems there was a consensus in community that check existence of a file or directory is preferred through filestatus which is make sense. This did not change in the last 7 years.
        Hide
        eli Eli Collins added a comment -

        Hey Owen,

        FileContext#exists was removed in HADOOP-6678. The motivation from Hairong was "in our production clusters, we often see that the number of getFileStatus calls is multiple times of the open calls. If we remove isFile, isDirectory, and exists from FileContext, users have to explicitly call getFileStatus first, it is more likely that they will write more efficient code"

        Adding AFS#exists that doesn't throw an exception seems reasonable, out of curiosity what's the particular change you need this for?

        Thanks,
        Eli

        Show
        eli Eli Collins added a comment - Hey Owen, FileContext#exists was removed in HADOOP-6678 . The motivation from Hairong was "in our production clusters, we often see that the number of getFileStatus calls is multiple times of the open calls. If we remove isFile, isDirectory, and exists from FileContext, users have to explicitly call getFileStatus first, it is more likely that they will write more efficient code" Adding AFS#exists that doesn't throw an exception seems reasonable, out of curiosity what's the particular change you need this for? Thanks, Eli
        Hide
        owen.omalley Owen O'Malley added a comment -

        I see now that there is a Util method that checks exist, but it still uses getFileStatus and catches the exception. There should be some way of asking an AbstractFileSystem whether a name exists that doesn't throw an exception.

        Show
        owen.omalley Owen O'Malley added a comment - I see now that there is a Util method that checks exist, but it still uses getFileStatus and catches the exception. There should be some way of asking an AbstractFileSystem whether a name exists that doesn't throw an exception.

          People

          • Assignee:
            Unassigned
            Reporter:
            owen.omalley Owen O'Malley
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development