Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-4949

Need better handling of empty parquet files

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.9.0
    • 1.17.0
    • Storage - Parquet
    • None

    Description

      I have an empty parquet file created from hive. When I tried to query against this table I got "IllegalArgumentException".

      select * from `test_dir/voter_empty`;
      Error: SYSTEM ERROR: IllegalArgumentException: MinorFragmentId 0 has no read entries assigned
      
        (org.apache.drill.exec.work.foreman.ForemanException) Unexpected exception during fragment initialization: MinorFragmentId 0 has no read entries assigned
          org.apache.drill.exec.work.foreman.Foreman.run():281
          java.util.concurrent.ThreadPoolExecutor.runWorker():1145
          java.util.concurrent.ThreadPoolExecutor$Worker.run():615
          java.lang.Thread.run():745
        Caused By (java.lang.IllegalArgumentException) MinorFragmentId 0 has no read entries assigned
          com.google.common.base.Preconditions.checkArgument():122
          org.apache.drill.exec.store.parquet.ParquetGroupScan.getSpecificScan():824
          org.apache.drill.exec.store.parquet.ParquetGroupScan.getSpecificScan():101
          org.apache.drill.exec.planner.fragment.Materializer.visitGroupScan():68
          org.apache.drill.exec.planner.fragment.Materializer.visitGroupScan():35
          org.apache.drill.exec.physical.base.AbstractGroupScan.accept():63
          org.apache.drill.exec.planner.fragment.Materializer.visitOp():102
          org.apache.drill.exec.planner.fragment.Materializer.visitOp():35
          org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitProject():79
          org.apache.drill.exec.physical.config.Project.accept():51
          org.apache.drill.exec.planner.fragment.Materializer.visitStore():82
          org.apache.drill.exec.planner.fragment.Materializer.visitStore():35
          org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitScreen():202
          org.apache.drill.exec.physical.config.Screen.accept():98
          org.apache.drill.exec.planner.fragment.SimpleParallelizer.generateWorkUnit():283
          org.apache.drill.exec.planner.fragment.SimpleParallelizer.getFragments():127
          org.apache.drill.exec.work.foreman.Foreman.getQueryWorkUnit():596
          org.apache.drill.exec.work.foreman.Foreman.runPhysicalPlan():426
          org.apache.drill.exec.work.foreman.Foreman.runSQL():1010
          org.apache.drill.exec.work.foreman.Foreman.run():264
          java.util.concurrent.ThreadPoolExecutor.runWorker():1145
          java.util.concurrent.ThreadPoolExecutor$Worker.run():615
          java.lang.Thread.run():745 (state=,code=0)
      

      Either drill should block the query and display a user friendly error message or allow the query to run and return empty result.

      Attachments

        Activity

          People

            Unassigned Unassigned
            knguyen Krystal
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: