Accumulo
  1. Accumulo
  2. ACCUMULO-1854

AccumuloInputFormat can be less strict about contents of Configuration

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.4.4, 1.5.0
    • Fix Version/s: 1.4.5, 1.5.1, 1.6.0
    • Component/s: None
    • Labels:
      None

      Description

      The AccumuloInputFormat required that the same exact Configuration object which was provided to its static configuration methods is also provided to its getSplits() and createRecordReader() methods.

      In practice, some tools, like MultipleInputs or Pig, don't guarantee that the same Configuration object is provided to getSplits() and createRecordReader(). This tends to be a common method of interaction to ensure that multiple InputFormats don't collide with one another in the same Configuration object (e.g. FileInputFormat). By serialization the necessary information from the Configuration regarding how to query Accumulo into the InputSplit AccumuloInputFormat creates, we can alleviate this requirement and make client interaction a little more standard.

        Issue Links

          Activity

          Christopher Tubbs made changes -
          Link This issue incorporates ACCUMULO-2648 [ ACCUMULO-2648 ]
          Josh Elser made changes -
          Description I noticed that I was unable to properly use MultipleInputs (or any code which uses a similar approach) with the AccumuloInputFormat class because of the way it builds up information in the Configuration object.

          It would be useful to be able to have multiple instances of AIF (and AOF) configured within one Job (Configuration).
          The AccumuloInputFormat required that the same exact Configuration object which was provided to its static configuration methods is also provided to its getSplits() and createRecordReader() methods.

          In practice, some tools, like MultipleInputs or Pig, don't guarantee that the same Configuration object is provided to getSplits() and createRecordReader(). This tends to be a common method of interaction to ensure that multiple InputFormats don't collide with one another in the same Configuration object (e.g. FileInputFormat). By serialization the necessary information from the Configuration regarding how to query Accumulo into the InputSplit AccumuloInputFormat creates, we can alleviate this requirement and make client interaction a little more standard.
          Josh Elser made changes -
          Summary Accumulo{Input,Output}Format can't handle multiple configurations AccumuloInputFormat can be less strict about contents of Configuration
          Josh Elser made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Josh Elser made changes -
          Fix Version/s 1.6.0 [ 12322468 ]
          Fix Version/s 1.6.1 [ 12325441 ]
          Josh Elser made changes -
          Fix Version/s 1.4.5 [ 12324754 ]
          Fix Version/s 1.5.1 [ 12324399 ]
          Fix Version/s 1.6.1 [ 12325441 ]
          Josh Elser made changes -
          Field Original Value New Value
          Link This issue blocks ACCUMULO-1783 [ ACCUMULO-1783 ]
          Josh Elser created issue -

            People

            • Assignee:
              Josh Elser
              Reporter:
              Josh Elser
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development