Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.0
    • Component/s: None
    • Labels:
      None

      Description

      Quick fix a bug caused by HIVE-243

      HIVE-234 removed the codes to wait for the threads to finish and use ThreadPoolExector.shutdown() to wait for the results. The usage of ThreadPoolExecutor.shutdown(), however, is wrong. The codes assume that the function blocks until all threads finish running but it actually only marks status and won't block. It caused wrong result of Utilities.getInputSummary() and caused many jobs are executed as local mode while they have huge data.

      Revert those changes quickly. We can have a follow-up to see how to deal with this more efficiently if you want.

      1. HIVE-2211.1.patch
        1.0 kB
        Siying Dong

        Activity

        Hide
        Ning Zhang added a comment -

        Committed. Thanks Siying!

        Show
        Ning Zhang added a comment - Committed. Thanks Siying!
        Hide
        John Sichi added a comment -

        It looks like this has already been committed? Can we set assignee and close it?

        Show
        John Sichi added a comment - It looks like this has already been committed? Can we set assignee and close it?
        Hide
        Hudson added a comment -

        Integrated in Hive-trunk-h0.21 #771 (See https://builds.apache.org/job/Hive-trunk-h0.21/771/)
        HIVE-2211. Fix a bug caused by HIVE-243 (Siying Dong via Ning Zhang)

        nzhang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1134179
        Files :

        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
        Show
        Hudson added a comment - Integrated in Hive-trunk-h0.21 #771 (See https://builds.apache.org/job/Hive-trunk-h0.21/771/ ) HIVE-2211 . Fix a bug caused by HIVE-243 (Siying Dong via Ning Zhang) nzhang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1134179 Files : /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
        Hide
        Ning Zhang added a comment -

        +1. In the followup we should add a unit test for this function to compare the contentsummary returned by parallel execution and serial execution.

        Show
        Ning Zhang added a comment - +1. In the followup we should add a unit test for this function to compare the contentsummary returned by parallel execution and serial execution.
        Hide
        Siying Dong added a comment -

        Just a simple revert. I did a small modification: when catching InterruptedException, stop waiting pending threads and exit.

        Show
        Siying Dong added a comment - Just a simple revert. I did a small modification: when catching InterruptedException, stop waiting pending threads and exit.

          People

          • Assignee:
            Siying Dong
            Reporter:
            Siying Dong
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development