Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.0
    • Component/s: None
    • Labels:
      None

      Description

      Quick fix a bug caused by HIVE-243

      HIVE-234 removed the codes to wait for the threads to finish and use ThreadPoolExector.shutdown() to wait for the results. The usage of ThreadPoolExecutor.shutdown(), however, is wrong. The codes assume that the function blocks until all threads finish running but it actually only marks status and won't block. It caused wrong result of Utilities.getInputSummary() and caused many jobs are executed as local mode while they have huge data.

      Revert those changes quickly. We can have a follow-up to see how to deal with this more efficiently if you want.

      1. HIVE-2211.1.patch
        1.0 kB
        Siying Dong

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        3m 42s 1 Siying Dong 09/Jun/11 22:30
        Patch Available Patch Available Resolved Resolved
        12d 17m 1 Ning Zhang 21/Jun/11 22:47
        Resolved Resolved Closed Closed
        178d 2h 8m 1 Carl Steinbach 16/Dec/11 23:56
        Carl Steinbach made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Ning Zhang made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Assignee Siying Dong [ sdong ]
        Fix Version/s 0.8.0 [ 12316178 ]
        Resolution Fixed [ 1 ]
        Hide
        Ning Zhang added a comment -

        Committed. Thanks Siying!

        Show
        Ning Zhang added a comment - Committed. Thanks Siying!
        Hide
        John Sichi added a comment -

        It looks like this has already been committed? Can we set assignee and close it?

        Show
        John Sichi added a comment - It looks like this has already been committed? Can we set assignee and close it?
        Hide
        Hudson added a comment -

        Integrated in Hive-trunk-h0.21 #771 (See https://builds.apache.org/job/Hive-trunk-h0.21/771/)
        HIVE-2211. Fix a bug caused by HIVE-243 (Siying Dong via Ning Zhang)

        nzhang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1134179
        Files :

        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
        Show
        Hudson added a comment - Integrated in Hive-trunk-h0.21 #771 (See https://builds.apache.org/job/Hive-trunk-h0.21/771/ ) HIVE-2211 . Fix a bug caused by HIVE-243 (Siying Dong via Ning Zhang) nzhang : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1134179 Files : /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
        Hide
        Ning Zhang added a comment -

        +1. In the followup we should add a unit test for this function to compare the contentsummary returned by parallel execution and serial execution.

        Show
        Ning Zhang added a comment - +1. In the followup we should add a unit test for this function to compare the contentsummary returned by parallel execution and serial execution.
        Siying Dong made changes -
        Summary Revert Fix a bug caused by HIVE-243
        Siying Dong made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Siying Dong made changes -
        Field Original Value New Value
        Attachment HIVE-2211.1.patch [ 12481982 ]
        Hide
        Siying Dong added a comment -

        Just a simple revert. I did a small modification: when catching InterruptedException, stop waiting pending threads and exit.

        Show
        Siying Dong added a comment - Just a simple revert. I did a small modification: when catching InterruptedException, stop waiting pending threads and exit.
        Siying Dong created issue -

          People

          • Assignee:
            Siying Dong
            Reporter:
            Siying Dong
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development