Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-23496

Add a flag to disable DB scan during Hive Server startup

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.3.7
    • Fix Version/s: None
    • Component/s: HiveServer2

      Description

      At start-up, HiveServer2 will do a full DB & Table scan. For large Hive Metastores, this operation could take 10+ minutes to complete. The goal of this task is to make this “verification” optional if possible.

      Currently, Hive 2.3.7 has the following in HiveMaterializedViewsRegistry that is causing this behavior.

       

      for (String dbName : db.getAllDatabases()) {
        // TODO: We should enhance metastore API such that it returns only
        // materialized views instead of all tables
        tables.addAll(db.getAllTableObjects(dbName));
      }
      

      It seems that <= Hive 2.3.x && >= Hive 3.0 does not have this behavior.

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                davidrabinowitz David Rabinowitz
              • Votes:
                1 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 0.5h
                  0.5h