Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-19912

Schema evolution checks prints a log line in INFO mode for each vectorized rowbatch, impacts performance

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0
    • Fix Version/s: 3.1.0, 3.0.1, 4.0.0
    • Component/s: Hive
    • Labels:
      None

      Description

      While benchmarking query96, noticed 17K log lines printed for each vector rowbactch

       

      In file ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java

       

      @@ -2554,8 +2554,8 @@ public static TypeDescription getDesiredRowTypeDescr(Configuration conf,
           }
      
           if (haveSchemaEvolutionProperties) {
      -      if (LOG.isInfoEnabled()) {
      -        LOG.info("Using schema evolution configuration variables schema.evolution.columns " +
      +      if (LOG.isDebugEnabled()) {
      +        LOG.debug("Using schema evolution configuration variables schema.evolution.columns " +
                   schemaEvolutionColumnNames.toString() +
                   " / schema.evolution.columns.types " +
                   schemaEvolutionTypeDescrs.toString() +

       

        Attachments

        1. HIVE-19912.1.patch
          0.8 kB
          Nita Dembla
        2. HIVE-19912.2.patch
          1 kB
          Prasanth Jayachandran

          Activity

            People

            • Assignee:
              ndembla Nita Dembla
              Reporter:
              ndembla Nita Dembla
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: