Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-2773

HiveStorageHandler.configureTableJobProperites() should let the handler know wether it is configuration for input or output

Log workAgile BoardRank to TopRank to BottomVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.0
    • Component/s: None
    • Release Note:
      HiveStorageHandler.configureTableJobProperties() has been replaced with configureInputJobProperties() and configureOutputJobProperties().

      Description

      HiveStorageHandler.configureTableJobProperties() is called to allow the storage handler to setup any properties that the underlying inputformat/outputformat/serde may need. But the handler implementation does not know whether it is being called for configuring input or output. This makes it a problem for handlers which sets an external state. In the case of HCatalog's HBase storageHandler, whenever a write needs to be configured we create a write transaction which needs to be committed or aborted later on. In this case configuring for both input and output each time configureTableJobProperties() is called would not be desirable. This has become an issue since HCatalog is dropping storageDrivers for SerDe and StorageHandler (see HCATALOG-237).

      My proposal is to replace configureTableJobProperties() with two methods:

      configureInputJobProperties()
      configureOutputJobProperties()

      Each method will have the same signature. I cursory look at the code and I believe changes should be straighforward also given that we are not really changing anything just splitting responsibility. If the community is fine with this approach I will go ahead and create a aptch.

        Attachments

        1. ASF.LICENSE.NOT.GRANTED--HIVE-2773.D1815.1.patch
          10 kB
          Phabricator
        2. ASF.LICENSE.NOT.GRANTED--HIVE-2773.D2007.1.patch
          10 kB
          Phabricator
        3. ASF.LICENSE.NOT.GRANTED--HIVE-2773.D2415.1.patch
          11 kB
          Phabricator
        4. HIVE-2773.patch
          10 kB
          Francis Christopher Liu

        Issue Links

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

            • Assignee:
              toffer Francis Christopher Liu Assign to me
              Reporter:
              toffer Francis Christopher Liu

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment