Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-158

HDFS soft lease limit should be configurable per-file

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Environment:

      All operating systems

      Description

      In Hypertable, there is a process called the RangeServer which is responsible for handling updates and queries on portions (ranges) of a database table. When an update arrives at the RangeServer, it first gets written to a commit log and then it goes into an in-memory table structure. If at any time, the RangeServer dies, when it comes back up, it replays the commit log to reconstruct the state of the in-memory table structure. As it stands now, when the RangeServer comes up, it cannot read the commit log file until the soft lease limit has expired. This causes the RangeServer to hang with all of the ranges that it was managing unaccessible.

      We would like to be able to configure the soft lease limit per-file so that we can set the limit to 0 for our commit log files. That would allow Hypertable to recover immediately after a crash.

        Issue Links

          Activity

          Hide
          Aaron T. Myers added a comment -

          This was long ago fixed through the implementation of sync and the recoverLease API.

          Show
          Aaron T. Myers added a comment - This was long ago fixed through the implementation of sync and the recoverLease API.

            People

            • Assignee:
              Unassigned
              Reporter:
              Doug Judd
            • Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development