Apache Drill
  1. Apache Drill
  2. DRILL-425

mock-sub-scan is not setting 'lastSet' in NullableVarCharVector or NullableVar16CharVector

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.4.0
    • Component/s: None
    • Labels:
      None

      Description

      Following plan outputs empty values for nullableStr1 column after 1st row.

      {
          head:{
              type:"APACHE_DRILL_PHYSICAL",
              version:"1",
              generator:{
                  type:"manual"
              }
          },
          graph:[
              {
                  @id:1,
                  pop:"mock-sub-scan",
                  url: "http://apache.org",
                  entries:[
                      {records: 100, types: [
                         {name: "nullableStr1", type: "VARCHAR", mode: "OPTIONAL"}
                      ]}
                  ]
              },
              {
                  @id:2,
                  child: 1,
                  pop:"project",
                  exprs: [
                      { ref: "nullableStr1", expr: "nullableStr1" }
                  ]
              },
              {
                  @id: 3,
                  child: 2,
                  pop: "screen"
              }
          ]
      }
      

      After transferring the ValueVector from 'scan' to 'project', ProjectRecordBatch calls NullableVarCharVector.setValueCount. As part of this method, all records from "lastSet +1" to "count" are set to empty byte array, because 'lastSet' is not set as part of the mock-sub-scan (default to 0), all but the first record have empty string data.

          public void setValueCount(int valueCount) {
           .....
            for (int i = lastSet + 1; i < valueCount; i++) {
              values.getMutator().set(i, new byte[]{});
            }
           ....
          }
      
      1. DRILL-425.patch
        1 kB
        Steven Phillips

        Issue Links

          Activity

          Tony Stevenson made changes -
          Workflow no-reopen-closed, patch-avail, testing [ 12859935 ] Drill workflow [ 12934223 ]
          Jacques Nadeau made changes -
          Fix Version/s 0.4.0 [ 12324963 ]
          Jake Farrell made changes -
          Workflow no-reopen-closed, patch-avail [ 12849394 ] no-reopen-closed, patch-avail, testing [ 12859935 ]
          Jacques Nadeau made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          Jacques Nadeau added a comment -

          Fixed in 5a94bf1

          Show
          Jacques Nadeau added a comment - Fixed in 5a94bf1
          Steven Phillips made changes -
          Attachment DRILL-425.patch [ 12635709 ]
          Venki Korukanti made changes -
          Link This issue is related to DRILL-417 [ DRILL-417 ]
          Venki Korukanti made changes -
          Field Original Value New Value
          Description Following plan outputs empty values for nullableStr1 column after 1st row.

          {
              head:{
                  type:"APACHE_DRILL_PHYSICAL",
                  version:"1",
                  generator:{
                      type:"manual"
                  }
              },
              graph:[
                  {
                      @id:1,
                      pop:"mock-sub-scan",
                      url: "http://apache.org",
                      entries:[
                          {records: 100, types: [
                             {name: "nullableStr1", type: "VARCHAR", mode: "OPTIONAL"},
                          ]}
                      ]
                  },
                  {
                      @id:2,
                      child: 1,
                      pop:"project",
                      exprs: [
                          { ref: "nullableStr1", expr: "nullableStr1" }
                      ]
                  },
                  {
                      @id: 3,
                      child: 2,
                      pop: "screen"
                  }
              ]
          }

          After transferring the ValueVector from 'scan' to 'project', ProjectRecordBatch calls NullableVarCharVector.setValueCount. As part of this method, all records from "lastSet +1" to "count" are set to empty byte array, because 'lastSet' is not set as part of the mock-sub-scan (default to 0), all but the first record have empty string data.

          {code}
              public void setValueCount(int valueCount) {
               .....
                for (int i = lastSet + 1; i < valueCount; i++) {
                  values.getMutator().set(i, new byte[]{});
                }
               ....
              }
          {code}
          Following plan outputs empty values for nullableStr1 column after 1st row.

          {code}
          {
              head:{
                  type:"APACHE_DRILL_PHYSICAL",
                  version:"1",
                  generator:{
                      type:"manual"
                  }
              },
              graph:[
                  {
                      @id:1,
                      pop:"mock-sub-scan",
                      url: "http://apache.org",
                      entries:[
                          {records: 100, types: [
                             {name: "nullableStr1", type: "VARCHAR", mode: "OPTIONAL"}
                          ]}
                      ]
                  },
                  {
                      @id:2,
                      child: 1,
                      pop:"project",
                      exprs: [
                          { ref: "nullableStr1", expr: "nullableStr1" }
                      ]
                  },
                  {
                      @id: 3,
                      child: 2,
                      pop: "screen"
                  }
              ]
          }
          {code}

          After transferring the ValueVector from 'scan' to 'project', ProjectRecordBatch calls NullableVarCharVector.setValueCount. As part of this method, all records from "lastSet +1" to "count" are set to empty byte array, because 'lastSet' is not set as part of the mock-sub-scan (default to 0), all but the first record have empty string data.

          {code}
              public void setValueCount(int valueCount) {
               .....
                for (int i = lastSet + 1; i < valueCount; i++) {
                  values.getMutator().set(i, new byte[]{});
                }
               ....
              }
          {code}
          Venki Korukanti created issue -

            People

            • Assignee:
              Steven Phillips
              Reporter:
              Venki Korukanti
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development