Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-14841 Replication - Phase 2
  3. HIVE-16530

Add HS2 operation logs and improve logs for REPL commands



    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.2.0
    • 3.0.0
    • repl


      This is the log format that is being proposed for Hive Repl query logs

      For bootstrap case:
      Hive will log a message for each object as it is being bootstrapped and it will be in the following sequence

      • Tables first (views are tables for this purpose) at time including partitions (depth first), followed by functions, constraints
      • The ordering is based on the ordering of listStatus API of HDFS
      • For each object, a message at the beginning of the replication will be logged
      • Every partition bootstrapped will be followed by a message saying the number of partitions bootstrapped so far (for the table) and the partition name
      • And a message at the end of bootstrap of an object

      Incremental case:

      • We will have DB Name, event id and event type will be part of the log header (for debugging/troubleshooting)
      • We will have information of current event ID and total number of events to replicate for every event replicated.


        1. Bootstrap_ReplDump_Console_Log.png
          601 kB
          Sankar Hariappan
        2. Bootstrap_ReplLoad_Console_Log.png
          476 kB
          Sankar Hariappan
        3. Incremental_ReplDump_Console_Log.png
          396 kB
          Sankar Hariappan
        4. HIVE-16530.01.patch
          10 kB
          Sankar Hariappan
        5. Incremental_ReplLoad_Console_Log.png
          602 kB
          Sankar Hariappan
        6. HIVE-16530.02.patch
          11 kB
          Sankar Hariappan

        Issue Links



              sankarh Sankar Hariappan
              sankarh Sankar Hariappan
              0 Vote for this issue
              5 Start watching this issue