Accumulo
  1. Accumulo
  2. ACCUMULO-2615

Simplify server configuration caching and ZK watchers

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.6.1, 1.7.0
    • Component/s: None
    • Labels:
      None

      Description

      The mechanism used for caching server-related configuration objects and processing ZK events could do with some simplification. For example:

      • The watcher objects obtain their corresponding configuration objects by reaching through the ServerConfiguration cache, instead of just having a more direct reference.
      • The watcher objects set log4j logging levels for both ZK and o.a.hadoop.io.compress to WARN, which prohibits debugging when needed. That should be done in log4j properties/XML instead.
      • Possible bug: NamespaceConfWatcher expires table configuration observers, not namespace configuration observers.
      • There is high duplication between the static and instance methods of ServerConfiguration. The class serves as both a cache container and a factory. Moreover, despite its name, it isn't actually a configuration object.

      I may be missing some subtleties of why things are as they are, so please fill me in as needed.

        Issue Links

          Activity

          Bill Havanki created issue -
          Bill Havanki made changes -
          Field Original Value New Value
          Assignee Bill Havanki [ bhavanki ]
          Bill Havanki made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Bill Havanki made changes -
          Link This issue is depended upon by ACCUMULO-2603 [ ACCUMULO-2603 ]
          Bill Havanki made changes -
          Description The mechanism used for caching server-related configuration objects and processing ZK events could do with some simplification. For example:

          * {{NamespaceConfWatcher}} and {{TableConfWatcher}} objects guess at what they are supposed to be watching by looking for namespace/table IDs in ZK events; they do not save the IDs they pertain to.
          * The watcher objects obtain their corresponding configuration objects by reaching through the {{ServerConfiguration}} cache, instead of just having a reference field.
          * The watcher objects set log4j logging levels for both ZK and o.a.hadoop.io.compress to WARN, which prohibits debugging when needed. That should be done in log4j properties/XML instead.
          * Possible bug: {{NamespaceConfWatcher}} expires table configuration observers, not namespace configuration observers.
          * There is high duplication between the static and instance methods of {{ServerConfiguration}}. The class serves as both a cache container and a factory. Moreover, despite its name, it isn't actually a configuration object.

          I may be missing some subtleties of why things are as they are, so please fill me in as needed.
          The mechanism used for caching server-related configuration objects and processing ZK events could do with some simplification. For example:

          * The watcher objects obtain their corresponding configuration objects by reaching through the {{ServerConfiguration}} cache, instead of just having a reference field.
          * The watcher objects set log4j logging levels for both ZK and o.a.hadoop.io.compress to WARN, which prohibits debugging when needed. That should be done in log4j properties/XML instead.
          * Possible bug: {{NamespaceConfWatcher}} expires table configuration observers, not namespace configuration observers.
          * There is high duplication between the static and instance methods of {{ServerConfiguration}}. The class serves as both a cache container and a factory. Moreover, despite its name, it isn't actually a configuration object.

          I may be missing some subtleties of why things are as they are, so please fill me in as needed.
          Bill Havanki made changes -
          Description The mechanism used for caching server-related configuration objects and processing ZK events could do with some simplification. For example:

          * The watcher objects obtain their corresponding configuration objects by reaching through the {{ServerConfiguration}} cache, instead of just having a reference field.
          * The watcher objects set log4j logging levels for both ZK and o.a.hadoop.io.compress to WARN, which prohibits debugging when needed. That should be done in log4j properties/XML instead.
          * Possible bug: {{NamespaceConfWatcher}} expires table configuration observers, not namespace configuration observers.
          * There is high duplication between the static and instance methods of {{ServerConfiguration}}. The class serves as both a cache container and a factory. Moreover, despite its name, it isn't actually a configuration object.

          I may be missing some subtleties of why things are as they are, so please fill me in as needed.
          The mechanism used for caching server-related configuration objects and processing ZK events could do with some simplification. For example:

          * The watcher objects obtain their corresponding configuration objects by reaching through the {{ServerConfiguration}} cache, instead of just having a more direct reference.
          * The watcher objects set log4j logging levels for both ZK and o.a.hadoop.io.compress to WARN, which prohibits debugging when needed. That should be done in log4j properties/XML instead.
          * Possible bug: {{NamespaceConfWatcher}} expires table configuration observers, not namespace configuration observers.
          * There is high duplication between the static and instance methods of {{ServerConfiguration}}. The class serves as both a cache container and a factory. Moreover, despite its name, it isn't actually a configuration object.

          I may be missing some subtleties of why things are as they are, so please fill me in as needed.
          Bill Havanki made changes -
          Status In Progress [ 3 ] Open [ 1 ]
          Christopher Tubbs made changes -
          Fix Version/s 1.7.0 [ 12324607 ]
          Bill Havanki made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Bill Havanki made changes -
          Remote Link This issue links to "Review (Web Link)" [ 15399 ]
          Bill Havanki made changes -
          Status In Progress [ 3 ] Patch Available [ 10002 ]
          ASF subversion and git services logged work - 14/Jul/14 19:30
          • Time Spent:
            10m
             
            Commit d53d45ccd3002765cf05aef0540666eccbd63dc1 in accumulo's branch refs/heads/1.6.1-SNAPSHOT from [~bhavanki]
            [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=d53d45c ]

            ACCUMULO-2615 Refactor server configurations

            * ServerConfiguration is deprecated in favor of the new ServerConfigurationFactory.
            * ServerConfigurationFactory caches configurations not just but namespace or table ID, but
              also by instance ID.
            * ZooConfigurationFactory takes over the creation logic from ZooConfiguration. A
              ZooConfiguration instance is no longer a singleton, but an ordinary instance.
              ZooConfigurationFactory caches created ones by instance ID.
            * NamespaceConfiguration and TableConfiguration each keep a static map of ZooCache instances,
              rather than a single static instance. The map is keyed by instance ID and namespace/table
              (resp.) ID.
            * getParentConfiguration() was added to all appropriate configurations.
            * The logic for retrieving properties from ZooCache instances is refactored to a new
              ZooCachePropertyAccessor object.
          ASF subversion and git services made changes -
          Remaining Estimate 0h [ 0 ]
          Time Spent 10m [ 600 ]
          Worklog Id 16651 [ 16651 ]
          ASF subversion and git services logged work - 14/Jul/14 19:30
          • Time Spent:
            10m
             
            Commit d53d45ccd3002765cf05aef0540666eccbd63dc1 in accumulo's branch refs/heads/master from [~bhavanki]
            [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=d53d45c ]

            ACCUMULO-2615 Refactor server configurations

            * ServerConfiguration is deprecated in favor of the new ServerConfigurationFactory.
            * ServerConfigurationFactory caches configurations not just but namespace or table ID, but
              also by instance ID.
            * ZooConfigurationFactory takes over the creation logic from ZooConfiguration. A
              ZooConfiguration instance is no longer a singleton, but an ordinary instance.
              ZooConfigurationFactory caches created ones by instance ID.
            * NamespaceConfiguration and TableConfiguration each keep a static map of ZooCache instances,
              rather than a single static instance. The map is keyed by instance ID and namespace/table
              (resp.) ID.
            * getParentConfiguration() was added to all appropriate configurations.
            * The logic for retrieving properties from ZooCache instances is refactored to a new
              ZooCachePropertyAccessor object.
          ASF subversion and git services made changes -
          Time Spent 10m [ 600 ] 20m [ 1200 ]
          Worklog Id 16652 [ 16652 ]
          ASF subversion and git services logged work - 14/Jul/14 19:30
          ASF subversion and git services made changes -
          Time Spent 20m [ 1200 ] 0.5h [ 1800 ]
          Worklog Id 16653 [ 16653 ]
          Bill Havanki made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          ASF subversion and git services logged work - 22/Jul/14 22:26
          ASF subversion and git services made changes -
          Time Spent 0.5h [ 1800 ] 40m [ 2400 ]
          Worklog Id 16740 [ 16740 ]
          ASF subversion and git services logged work - 22/Jul/14 22:27
          ASF subversion and git services made changes -
          Time Spent 40m [ 2400 ] 50m [ 3000 ]
          Worklog Id 16742 [ 16742 ]
          ASF subversion and git services logged work - 22/Jul/14 22:27
          ASF subversion and git services made changes -
          Time Spent 50m [ 3000 ] 1h [ 3600 ]
          Worklog Id 16743 [ 16743 ]
          Hide
          Eric Newton added a comment -

          Reopened until sub-tasks are complete.

          Show
          Eric Newton added a comment - Reopened until sub-tasks are complete.
          Eric Newton made changes -
          Resolution Fixed [ 1 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          Assignee Bill Havanki [ bhavanki ] Eric Newton [ ecn ]
          Bill Havanki made changes -
          Link This issue breaks ACCUMULO-3019 [ ACCUMULO-3019 ]
          Eric Newton made changes -
          Status Reopened [ 4 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          ASF subversion and git services logged work - 23/Aug/14 01:37
          ASF subversion and git services made changes -
          Time Spent 1h [ 3600 ] 1h 10m [ 4200 ]
          Worklog Id 17263 [ 17263 ]
          ASF subversion and git services logged work - 23/Aug/14 01:38
          ASF subversion and git services made changes -
          Time Spent 1h 10m [ 4200 ] 1h 20m [ 4800 ]
          Worklog Id 17272 [ 17272 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          In Progress In Progress Open Open
          2d 4h 20m 1 Bill Havanki 04/Apr/14 22:34
          Open Open In Progress In Progress
          59d 22h 46m 2 Bill Havanki 03/Jun/14 21:20
          In Progress In Progress Patch Available Patch Available
          13d 20h 52m 1 Bill Havanki 17/Jun/14 18:12
          Patch Available Patch Available Resolved Resolved
          27d 1h 18m 1 Bill Havanki 14/Jul/14 19:31
          Resolved Resolved Reopened Reopened
          10d 19h 6m 1 Eric Newton 25/Jul/14 14:38
          Reopened Reopened Resolved Resolved
          7d 5h 6m 1 Eric Newton 01/Aug/14 19:45

            People

            • Assignee:
              Eric Newton
              Reporter:
              Bill Havanki
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 2.5h
                2.5h

                  Development