Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-16865 Handle replication bootstrap of large databases
  3. HIVE-16892

Move creation of _files from ReplCopyTask to analysis phase for boostrap replication

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 3.0.0
    • 3.0.0
    • HiveServer2
    • None

    Description

      during replication boostrap we create the _files via ReplCopyTask for partitions and tables, this can be done inline as part of analysis phase rather than creating the replCopytask,

      This is done to prevent creation of huge number of these tasks in memory before giving it to the execution engine.

      Attachments

        1. HIVE-16892.1.patch
          37 kB
          Anishek Agarwal
        2. HIVE-16892.2.patch
          37 kB
          Anishek Agarwal
        3. HIVE-16892.3.patch
          41 kB
          Anishek Agarwal
        4. HIVE-16892.4.patch
          42 kB
          Anishek Agarwal
        5. HIVE-16892.5.patch
          42 kB
          Anishek Agarwal
        6. HIVE-16892.6.patch
          41 kB
          Anishek Agarwal

        Issue Links

          links to

          Web Link GitHub Pull Request #196

          Activity

            People

              anishek Anishek Agarwal
              anishek Anishek Agarwal
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: