Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-6239

Remote data load breaks with "LOAD DATA LOCAL INPATH": Invalid path

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: Impala 2.11.0
    • Fix Version/s: Impala 2.11.0
    • Component/s: Infrastructure
    • Labels:
      None
    • Epic Color:
      ghx-label-5

      Description

      A recent patch to correct IMPALA-6068 introduced the use of LOAD DATA LOCAL throughout our data load process. This improved the efficiency of loading data onto the local mini-cluster, but it breaks the ability to use those same scripts to load data onto a remote cluster.

      With https://github.com/apache/incubator-impala/commit/e4f585240ac8f478e25402806f4ea38531b4bf84

      0: jdbc:hive2://impala-remote-cluster-tests-1> LOAD DATA LOCAL INPATH '/data1/systest/cdh/testing/infra/quasar/deploy/rt/common_tests/impala/Impala/testdata/avro_null_c
      har/000000_0'
      . . . . . . . . . . . . . . . . . . . . . . .> OVERWRITE INTO TABLE functional_avro_snap.avro_unicode_nulls;
      going to print operations logs
      printed operations logs
      Getting log thread is interrupted, since query is done!
      Error: Error while compiling statement: FAILED: SemanticException Line 1:23 Invalid path ''/data1/systest/cdh/testing/infra/quasar/deploy/rt/common_tests/impala/Impala/testda
      ta/avro_null_char/000000_0'': No files matching path file:/data1/systest/cdh/testing/infra/quasar/deploy/rt/common_tests/impala/Impala/testdata/avro_null_char/000000_0 (state
      =42000,code=40000)
      org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: SemanticException Line 1:23 Invalid path ''/data1/systest/cdh/testing/infra/quasar/deploy/rt/common_tests/impala/Impala/testdata/avro_null_char/000000_0'': No files matching path file:/data1/systest/cdh/testing/infra/quasar/deploy/rt/common_tests/impala/Impala/testdata/avro_null_char/000000_0
      

      Behavior before the patch:

      0: jdbc:hive2://impala-remote-cluster-tests-1> create external table if not exists functional_avro_snap.avro_unicode_nulls like functional.liketbl stored as avro LOCATION '/test-warehouse/avro_null_char';
      going to print operations logs
      printed operations logs
      Getting log thread is interrupted, since query is done!
      INFO  : Compiling command(queryId=hive_20171124171616_d48b8d8e-388e-420e-be10-fb1dd6ef5fa9): create external table if not exists functional_avro_snap.avro_unicode_nulls like functional.liketbl stored as avro LOCATION '/test-warehouse/avro_null_char'
      INFO  : Semantic Analysis Completed
      INFO  : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
      INFO  : Completed compiling command(queryId=hive_20171124171616_d48b8d8e-388e-420e-be10-fb1dd6ef5fa9); Time taken: 0.002 seconds
      INFO  : Executing command(queryId=hive_20171124171616_d48b8d8e-388e-420e-be10-fb1dd6ef5fa9): create external table if not exists functional_avro_snap.avro_unicode_nulls like functional.liketbl stored as avro LOCATION '/test-warehouse/avro_null_char'
      INFO  : Starting task [Stage-0:DDL] in serial mode
      INFO  : Completed executing command(queryId=hive_20171124171616_d48b8d8e-388e-420e-be10-fb1dd6ef5fa9); Time taken: 0.022 seconds
      INFO  : OK
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                dknupp David Knupp
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: