Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-8113

test_aggregation and test_avro_primitive_in_list fail in S3

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: Impala 3.2.0
    • Fix Version/s: Impala 3.2.0
    • Component/s: Infrastructure
    • Labels:
      None

      Description

      Likely more victims of our infra in S3.

      query_test/test_aggregation.py:138: in test_aggregation
          result = self.execute_query(query, vector.get_value('exec_option'))
      common/impala_test_suite.py:597: in wrapper
          return function(*args, **kwargs)
      common/impala_test_suite.py:628: in execute_query
          return self.__execute_query(self.client, query, query_options)
      common/impala_test_suite.py:695: in __execute_query
          return impalad_client.execute(query, user=user)
      common/impala_connection.py:174: in execute
          return self.__beeswax_client.execute(sql_stmt, user=user)
      beeswax/impala_beeswax.py:182: in execute
          handle = self.__execute_query(query_string.strip(), user=user)
      beeswax/impala_beeswax.py:359: in __execute_query
          self.wait_for_finished(handle)
      beeswax/impala_beeswax.py:380: in wait_for_finished
          raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
      E   ImpalaBeeswaxException: ImpalaBeeswaxException:
      E    Query aborted:Disk I/O error: Error reading from HDFS file: s3a://impala-test-uswest2-1/test-warehouse/alltypesagg_parquet/year=2010/month=1/day=8/5642b2da93dae1ad-494132e000000005_592013737_data.0.parq
      E   Error(255): Unknown error 255
      E   Root cause: SdkClientException: Data read has a different length than the expected: dataLength=0; expectedLength=45494; includeSkipped=true; in.getClass()=class com.amazonaws.services.s3.AmazonS3Client$2; markedSupported=false; marked=0; resetSinceLastMarked=false; markCount=0; resetCount=0
      
      query_test/test_nested_types.py:263: in test_avro_primitive_in_list
          "AvroPrimitiveInList.parquet", vector)
      query_test/test_nested_types.py:287: in __test_primitive_in_list
          result = self.execute_query("select item from %s.col1" % full_name, qopts)
      common/impala_test_suite.py:597: in wrapper
          return function(*args, **kwargs)
      common/impala_test_suite.py:628: in execute_query
          return self.__execute_query(self.client, query, query_options)
      common/impala_test_suite.py:695: in __execute_query
          return impalad_client.execute(query, user=user)
      common/impala_connection.py:174: in execute
          return self.__beeswax_client.execute(sql_stmt, user=user)
      beeswax/impala_beeswax.py:182: in execute
          handle = self.__execute_query(query_string.strip(), user=user)
      beeswax/impala_beeswax.py:359: in __execute_query
          self.wait_for_finished(handle)
      beeswax/impala_beeswax.py:380: in wait_for_finished
          raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
      E   ImpalaBeeswaxException: ImpalaBeeswaxException:
      E    Query aborted:Disk I/O error: Failed to open HDFS file s3a://impala-test-uswest2-1/test-warehouse/test_avro_primitive_in_list_38f182c4.db/AvroPrimitiveInList/AvroPrimitiveInList.parquet
      E   Error(2): No such file or directory
      E   Root cause: FileNotFoundException: No such file or directory: s3a://impala-test-uswest2-1/test-warehouse/test_avro_primitive_in_list_38f182c4.db/AvroPrimitiveInList/AvroPrimitiveInList.parquet
      

        Attachments

          Activity

            People

            • Assignee:
              mikeb Michael Brown
              Reporter:
              mikeb Michael Brown
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: