Hadoop Common
  1. Hadoop Common
  2. HADOOP-7901

Have Configuration use Read/Write Locks

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.23.1
    • Fix Version/s: None
    • Component/s: conf
    • Labels:
      None

      Description

      We can potentially improve performance by moving to read/write locks for configuration instead of the current synchronization.

        Activity

        Hide
        Todd Lipcon added a comment -

        Where are you seeing contention on the Configuration object? The downside of rwlock is (a) increased memory usage, and (b) higher constant overhead, in my experience. Not that I'm against it, just would like to know more about the motivation.

        Show
        Todd Lipcon added a comment - Where are you seeing contention on the Configuration object? The downside of rwlock is (a) increased memory usage, and (b) higher constant overhead, in my experience. Not that I'm against it, just would like to know more about the motivation.
        Hide
        Robert Joseph Evans added a comment -

        I have not measured any contention in the Configuration object. That is a very good point I really should not be doing optimizations without measuring first to know if they are needless or not. I really should rename this measure lock contention in Configuration object and then possibly use Read/Write Locks.

        This was filed partly in response to MAPREDUCE-3519. There was a deadlock introduced because one thread grabbed a lock on a Configuration object and then tried to grab a different lock while a separate thread tried to grab the locks in reverse order. This seems especially odd to me because my gut feeling is that most of the time all we are trying to do is to read data from Configuration. Very rarely do we want to change it, so I thought that it would be a potential performance improvement, in addition to removing some of the potential for these deadlocks in the future.

        So before doing any code changes I will measure the level of lock contention.

        Show
        Robert Joseph Evans added a comment - I have not measured any contention in the Configuration object. That is a very good point I really should not be doing optimizations without measuring first to know if they are needless or not. I really should rename this measure lock contention in Configuration object and then possibly use Read/Write Locks. This was filed partly in response to MAPREDUCE-3519 . There was a deadlock introduced because one thread grabbed a lock on a Configuration object and then tried to grab a different lock while a separate thread tried to grab the locks in reverse order. This seems especially odd to me because my gut feeling is that most of the time all we are trying to do is to read data from Configuration. Very rarely do we want to change it, so I thought that it would be a potential performance improvement, in addition to removing some of the potential for these deadlocks in the future. So before doing any code changes I will measure the level of lock contention.

          People

          • Assignee:
            Robert Joseph Evans
            Reporter:
            Robert Joseph Evans
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:

              Development