Accumulo
  1. Accumulo
  2. ACCUMULO-2984

Enable running MAC using an existing instance

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.6.2, 1.7.0
    • Component/s: mini
    • Labels:
      None

      Description

      Currently MAC expects to run it's own zookeeper and do initialization of the instance for the given site.xml. I believe with a few minor changes, it could also be used to point against a live instance. Mainly, this requires using the existing accumulo-site.xml file in lieu of the pseudo-provided/generated one and not starting it's own ZK.

        Issue Links

          Activity

          Hide
          John Vines added a comment - - edited

          I would like this in 1.6.1 (or 1.6.x in general) and I'm going to attempt to get it to land there unless it means API alterations.

          Show
          John Vines added a comment - - edited I would like this in 1.6.1 (or 1.6.x in general) and I'm going to attempt to get it to land there unless it means API alterations.
          Hide
          Sean Busbey added a comment -

          is the plan for this to be another implementation of the MiniAccumuloCluster class ala Josh's work to have a cluster that spins up in yarn? or is the plan for it to be alterations on MiniAccumuloClusterImpl?

          I'd prefer the former if you're looking to run against an Accumulo config rather than e.g. just running against an existing HDFS/ZK.

          Show
          Sean Busbey added a comment - is the plan for this to be another implementation of the MiniAccumuloCluster class ala Josh's work to have a cluster that spins up in yarn? or is the plan for it to be alterations on MiniAccumuloClusterImpl? I'd prefer the former if you're looking to run against an Accumulo config rather than e.g. just running against an existing HDFS/ZK.
          Hide
          John Vines added a comment -

          The latter. This is exactly like existing MAC, but just a case of A. providing an external ZK B. (may be optional) providing an external site.xml file and C. telling it not to try to initialize.

          Why would you prefer the former?

          Show
          John Vines added a comment - The latter. This is exactly like existing MAC, but just a case of A. providing an external ZK B. (may be optional) providing an external site.xml file and C. telling it not to try to initialize. Why would you prefer the former?
          Hide
          Sean Busbey added a comment -

          Because running on multiple machines is a substantially different behavior. I'd like to minimize the risk around a partial implementation delaying 1.6.1.

          FWIW, I really want something like this so I can run the ITs against a real cluster. When just trying to get MAC to use an external HDFS / MR we ran into ACCUMULO-2855. That'll probably have to be fixed as a part of this.

          Show
          Sean Busbey added a comment - Because running on multiple machines is a substantially different behavior. I'd like to minimize the risk around a partial implementation delaying 1.6.1. FWIW, I really want something like this so I can run the ITs against a real cluster. When just trying to get MAC to use an external HDFS / MR we ran into ACCUMULO-2855 . That'll probably have to be fixed as a part of this.
          Hide
          John Vines added a comment -

          This ticket isn't about running ITs against a real cluster, that sounds like something related to Josh's work you referenced before. Work that is orthogonal to this. This ticket is about running an fake cluster in memory against a real instance, not about running a real cluster against a fake instance.

          As for ACCUMULO-2855, that's a non-issue because this is about NOT using a generated site.xml, but rather using a provided one.

          Show
          John Vines added a comment - This ticket isn't about running ITs against a real cluster, that sounds like something related to Josh's work you referenced before. Work that is orthogonal to this. This ticket is about running an fake cluster in memory against a real instance, not about running a real cluster against a fake instance. As for ACCUMULO-2855 , that's a non-issue because this is about NOT using a generated site.xml, but rather using a provided one.
          Hide
          Sean Busbey added a comment -

          so is this essentially just allowing for the creation of a MiniAccumuloConfig based on a provided accumulo-site.xml?

          Show
          Sean Busbey added a comment - so is this essentially just allowing for the creation of a MiniAccumuloConfig based on a provided accumulo-site.xml?
          Hide
          John Vines added a comment -

          That + not starting ZK (because it's already running) + not initializing it (because it's already initialized)

          Show
          John Vines added a comment - That + not starting ZK (because it's already running) + not initializing it (because it's already initialized)
          Hide
          Sean Busbey added a comment -

          that addresses my concerns then.

          Show
          Sean Busbey added a comment - that addresses my concerns then.
          Hide
          John Vines added a comment -

          Beautiful

          Show
          John Vines added a comment - Beautiful

            People

            • Assignee:
              John Vines
              Reporter:
              John Vines
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 1h
                1h

                  Development