Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-5317 Implement insert, update, and delete in Hive with full ACID support
  3. HIVE-5687

Streaming support in Hive

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 0.13.0
    • None
    • New transactional APIs to support Streaming data directly into Hive.
    • Streaming ACID

    Description

      Implement support for Streaming data into HIVE.

      • Provide a client streaming API
      • Transaction support: Clients should be able to periodically commit a batch of records atomically
      • Immediate visibility: Records should be immediately visible to queries on commit
      • Should not overload HDFS with too many small files

      Use Cases:

      • Streaming logs `into HIVE via Flume
      • Streaming results of computations from Storm

      Attachments

        1. 5687-api-spec4.pdf
          136 kB
          Roshan Naik
        2. 5687-draft-api-spec.pdf
          37 kB
          Roshan Naik
        3. 5687-draft-api-spec2.pdf
          30 kB
          Roshan Naik
        4. 5687-draft-api-spec3.pdf
          30 kB
          Roshan Naik
        5. HIVE-5687.patch
          39 kB
          Roshan Naik
        6. HIVE-5687.v2.patch
          54 kB
          Roshan Naik
        7. HIVE-5687.v3.patch
          122 kB
          Roshan Naik
        8. HIVE-5687.v4.patch
          126 kB
          Roshan Naik
        9. HIVE-5687.v5.patch
          129 kB
          Roshan Naik
        10. HIVE-5687.v6.patch
          250 kB
          Roshan Naik
        11. HIVE-5687.v7.patch
          139 kB
          Roshan Naik
        12. HIVE-5687-unit-test-fix.patch
          5 kB
          Owen O'Malley
        13. Hive Streaming Ingest API for v3 patch.pdf
          161 kB
          Roshan Naik
        14. Hive Streaming Ingest API for v4 patch.pdf
          172 kB
          Roshan Naik
        15. package.html
          7 kB
          Owen O'Malley

        Issue Links

          depends upon

          Sub-task - The sub-task of the issue HIVE-5843 Transaction manager for Hive

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HIVE-6721 Streaming ingest needs to be able to send many heartbeats together

          • Major - Major loss of function.
          • Resolved
          is depended upon by

          New Feature - A new feature of the product, which has yet to be developed. FLUME-1734 Create a Hive Sink based on the new Hive Streaming support

          • Major - Major loss of function.
          • Resolved
          is part of

          New Feature - A new feature of the product, which has yet to be developed. HIVE-5317 Implement insert, update, and delete in Hive with full ACID support

          • Major - Major loss of function.
          • Closed
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HIVE-6885 Address style and docs feedback in HIVE-5687

          • Major - Major loss of function.
          • Open
          supercedes

          New Feature - A new feature of the product, which has yet to be developed. HIVE-4196 Support for Streaming Partitions in Hive

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link Review Board link

          Activity

            People

              roshan_naik Roshan Naik
              roshan_naik Roshan Naik
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: