Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 0.20.2
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      duiod on IRC was asking about locks on the write path so I reviewed it a bit and saw that we are very good at not locking except on row level in HRegion and others but in HLog we actually synchronize everything on updateLock. I'm pretty sure that's not what we want. I guess a RWLock would be more useful.

        Activity

        Hide
        apurtell Andrew Purtell added a comment -

        Block branch too.

        Show
        apurtell Andrew Purtell added a comment - Block branch too.
        Hide
        stack stack added a comment -

        I don't think there'll ever be a 0.20.4. Our focus has shifted to 0.21 hbase. I'd suggest that we get this into a 0.20.3 RC2. If it adds some speedup to server, it'd be a nice addition.

        Show
        stack stack added a comment - I don't think there'll ever be a 0.20.4. Our focus has shifted to 0.21 hbase. I'd suggest that we get this into a 0.20.3 RC2. If it adds some speedup to server, it'd be a nice addition.
        Hide
        ryanobjc ryan rawson added a comment -

        I think the underlying sequence file is synchronized, so I'm not sure how much this would help...

        Show
        ryanobjc ryan rawson added a comment - I think the underlying sequence file is synchronized, so I'm not sure how much this would help...
        Hide
        stack stack added a comment -

        Yeah, I was going to say what Ryan says, that down in SequenceFile, its synchronized anyway so we'd be narrowing the synchronization for sure but ain't sure the narrowing would make for that much in performance improvement.

        Chatting too w/ J-D, we probably want the current synchronization to ensure that the get of the next sequence number happens in the same synchronization block that does the actual write. Otherwise, we could have edits going in out of order which could be a prob. if a high number edit goes in before a lower number edit.... we might lose a few edits if we crash after the high number goes in but before the lower edits have a chance to go in.

        We might want to revisit if we want to run a pool of WAL-writers.

        Show
        stack stack added a comment - Yeah, I was going to say what Ryan says, that down in SequenceFile, its synchronized anyway so we'd be narrowing the synchronization for sure but ain't sure the narrowing would make for that much in performance improvement. Chatting too w/ J-D, we probably want the current synchronization to ensure that the get of the next sequence number happens in the same synchronization block that does the actual write. Otherwise, we could have edits going in out of order which could be a prob. if a high number edit goes in before a lower number edit.... we might lose a few edits if we crash after the high number goes in but before the lower edits have a chance to go in. We might want to revisit if we want to run a pool of WAL-writers.
        Hide
        stack stack added a comment -

        This is not a blocker, after discussion. Nor is it to be addressed in 0.20.4. Moving it out.

        Show
        stack stack added a comment - This is not a blocker, after discussion. Nor is it to be addressed in 0.20.4. Moving it out.
        Hide
        jdcryans Jean-Daniel Cryans added a comment -

        This is fixed by HDFS-895 and we complement this with HBASE-2467.

        Show
        jdcryans Jean-Daniel Cryans added a comment - This is fixed by HDFS-895 and we complement this with HBASE-2467 .

          People

          • Assignee:
            jdcryans Jean-Daniel Cryans
            Reporter:
            jdcryans Jean-Daniel Cryans
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development