Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.19.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Hide
      Added HDFS file access times. By default, access times will be precise to the most recent hour boundary. A configuration parameter dfs.access.time.precision (milliseconds) is used to control this precision. Setting a value of 0 will disable persisting access times for HDFS files.
      Show
      Added HDFS file access times. By default, access times will be precise to the most recent hour boundary. A configuration parameter dfs.access.time.precision (milliseconds) is used to control this precision. Setting a value of 0 will disable persisting access times for HDFS files.

      Description

      HDFS should support some type of statistics that allows an administrator to determine when a file was last accessed.

      Since HDFS does not have quotas yet, it is likely that users keep on accumulating files in their home directories without much regard to the amount of space they are occupying. This causes memory-related problems with the namenode.

      Access times are costly to maintain. AFS does not maintain access times. I thind DCE-DFS does maintain access times with a coarse granularity.

      One proposal for HDFS would be to implement something like an "access bit".
      1. This access-bit is set when a file is accessed. If the access bit is already set, then this call does not result in a transaction.
      2. A FileSystem.clearAccessBits() indicates that the access bits of all files need to be cleared.

      An administrator can effectively use the above mechanism (maybe a daily cron job) to determine files that are recently used.

      1. accessTime6.patch
        45 kB
        dhruba borthakur
      2. accessTime5.patch
        41 kB
        dhruba borthakur
      3. accessTime4.patch
        41 kB
        dhruba borthakur
      4. accessTime1.patch
        37 kB
        dhruba borthakur

        Issue Links

          Activity

            People

            • Assignee:
              dhruba borthakur
              Reporter:
              dhruba borthakur
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development