Pig
  1. Pig
  2. PIG-4100

TEZ bug broke hadoop compilation 2 in trunk

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Not A Problem
    • Affects Version/s: None
    • Fix Version/s: 0.14.0
    • Component/s: tez
    • Labels:
      None

      Description

      Cannot compile pig trunk (with parameter -Dhadoopversion=23) due to this error

      [javac] /home/hoang/DATA/trunk-test/trunk/src/org/apache/pig/backend/hadoop/executionengine/tez/TezDagBuilder.java:594: error: cannot find symbol
      [javac] vertex.setLocationHint(new VertexLocationHint(tezOp.getLoaderInfo().getInputSplitInfo().getTaskLocationHints()));
      [javac] ^
      [javac] symbol: method setLocationHint(VertexLocationHint)
      [javac] location: variable vertex of type Vertex

        Activity

        Quang-Nhat HOANG-XUAN created issue -
        Quang-Nhat HOANG-XUAN made changes -
        Field Original Value New Value
        Fix Version/s 0.14.0 [ 12326954 ]
        Quang-Nhat HOANG-XUAN made changes -
        Description Cannot compile pig trunk due to this error

        javac] /home/hoang/DATA/trunk-test/trunk/src/org/apache/pig/backend/hadoop/executionengine/tez/TezDagBuilder.java:594: error: cannot find symbol
            [javac] vertex.setLocationHint(new VertexLocationHint(tezOp.getLoaderInfo().getInputSplitInfo().getTaskLocationHints()));
            [javac] ^
            [javac] symbol: method setLocationHint(VertexLocationHint)
            [javac] location: variable vertex of type Vertex
        Cannot compile pig trunk (with parameter -Dhadoopversion=23) due to this error

        javac] /home/hoang/DATA/trunk-test/trunk/src/org/apache/pig/backend/hadoop/executionengine/tez/TezDagBuilder.java:594: error: cannot find symbol
            [javac] vertex.setLocationHint(new VertexLocationHint(tezOp.getLoaderInfo().getInputSplitInfo().getTaskLocationHints()));
            [javac] ^
            [javac] symbol: method setLocationHint(VertexLocationHint)
            [javac] location: variable vertex of type Vertex
        Quang-Nhat HOANG-XUAN made changes -
        Component/s tez [ 12321016 ]
        Quang-Nhat HOANG-XUAN made changes -
        Description Cannot compile pig trunk (with parameter -Dhadoopversion=23) due to this error

        javac] /home/hoang/DATA/trunk-test/trunk/src/org/apache/pig/backend/hadoop/executionengine/tez/TezDagBuilder.java:594: error: cannot find symbol
            [javac] vertex.setLocationHint(new VertexLocationHint(tezOp.getLoaderInfo().getInputSplitInfo().getTaskLocationHints()));
            [javac] ^
            [javac] symbol: method setLocationHint(VertexLocationHint)
            [javac] location: variable vertex of type Vertex
        Cannot compile pig trunk (with parameter -Dhadoopversion=23) due to this error

        [javac] /home/hoang/DATA/trunk-test/trunk/src/org/apache/pig/backend/hadoop/executionengine/tez/TezDagBuilder.java:594: error: cannot find symbol
            [javac] vertex.setLocationHint(new VertexLocationHint(tezOp.getLoaderInfo().getInputSplitInfo().getTaskLocationHints()));
            [javac] ^
            [javac] symbol: method setLocationHint(VertexLocationHint)
            [javac] location: variable vertex of type Vertex
        Hide
        Cheolsoo Park added a comment - - edited

        Sorry for the inconvenience. This is because of PIG-4098.

        Tez snapshot jars are not published to maven yet, so you're seeing an error. But it will be resolved as soon as Tez jenkin build run. One way to workaround is to manually run mvn install -DskipTests with Tez trunk.

        Show
        Cheolsoo Park added a comment - - edited Sorry for the inconvenience. This is because of PIG-4098 . Tez snapshot jars are not published to maven yet, so you're seeing an error. But it will be resolved as soon as Tez jenkin build run. One way to workaround is to manually run mvn install -DskipTests with Tez trunk .
        Hide
        Quang-Nhat HOANG-XUAN added a comment -

        I got it.
        Thank you.

        Show
        Quang-Nhat HOANG-XUAN added a comment - I got it. Thank you.
        Hide
        Koji Noguchi added a comment -

        Tez snapshot jars are not published to maven yet, so you're seeing an error.

        We should have waited for this to happen before committing PIG-4098?

        Show
        Koji Noguchi added a comment - Tez snapshot jars are not published to maven yet, so you're seeing an error. We should have waited for this to happen before committing PIG-4098 ?
        Hide
        Cheolsoo Park added a comment -

        Indeed, my bad.

        Show
        Cheolsoo Park added a comment - Indeed, my bad.
        Hide
        Daniel Dai added a comment -

        Let me ask Tez folks to publish a SNAPSHOT.

        Show
        Daniel Dai added a comment - Let me ask Tez folks to publish a SNAPSHOT.
        Hide
        Cheolsoo Park added a comment -

        If it's delayed, I can revert PIG-4098 for now. Let me know.

        Show
        Cheolsoo Park added a comment - If it's delayed, I can revert PIG-4098 for now. Let me know.
        Hide
        Daniel Dai added a comment -

        SNAPSHOT published.

        Show
        Daniel Dai added a comment - SNAPSHOT published.
        Hide
        Cheolsoo Park added a comment -

        Closing the jira now.

        Show
        Cheolsoo Park added a comment - Closing the jira now.
        Cheolsoo Park made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Not a Problem [ 8 ]
        Hide
        Quang-Nhat HOANG-XUAN added a comment -

        Hi, After checking out, I tried to build by "ant -Dhadoopversion=23" and still got these errors:

        [javac] /home/hoang/DATA/trunk-test/trunk/src/org/apache/pig/backend/hadoop/executionengine/tez/TezDagBuilder.java:594: error: cannot find symbol
        [javac] vertex.setLocationHint(new VertexLocationHint(tezOp.getLoaderInfo().getInputSplitInfo().getTaskLocationHints()));
        [javac] ^
        [javac] symbol: method setLocationHint(VertexLocationHint)
        [javac] location: variable vertex of type Vertex
        [javac] /home/hoang/DATA/trunk-test/trunk/src/org/apache/pig/backend/hadoop/executionengine/tez/TezLauncher.java:80: error: cannot find symbol
        [javac] pc.getProperties().setProperty(TezRuntimeConfiguration.TEZ_RUNTIME_OPTIMIZE_LOCAL_FETCH, "true");
        [javac] ^
        [javac] symbol: variable TEZ_RUNTIME_OPTIMIZE_LOCAL_FETCH
        [javac] location: class TezRuntimeConfiguration
        [javac] Note: Some input files use or override a deprecated API.
        [javac] Note: Recompile with -Xlint:deprecation for details.
        [javac] Note: Some input files use unchecked or unsafe operations.
        [javac] Note: Recompile with -Xlint:unchecked for details.
        [javac] 2 errors
        [javac] 3 warnings

        Show
        Quang-Nhat HOANG-XUAN added a comment - Hi, After checking out, I tried to build by "ant -Dhadoopversion=23" and still got these errors: [javac] /home/hoang/DATA/trunk-test/trunk/src/org/apache/pig/backend/hadoop/executionengine/tez/TezDagBuilder.java:594: error: cannot find symbol [javac] vertex.setLocationHint(new VertexLocationHint(tezOp.getLoaderInfo().getInputSplitInfo().getTaskLocationHints())); [javac] ^ [javac] symbol: method setLocationHint(VertexLocationHint) [javac] location: variable vertex of type Vertex [javac] /home/hoang/DATA/trunk-test/trunk/src/org/apache/pig/backend/hadoop/executionengine/tez/TezLauncher.java:80: error: cannot find symbol [javac] pc.getProperties().setProperty(TezRuntimeConfiguration.TEZ_RUNTIME_OPTIMIZE_LOCAL_FETCH, "true"); [javac] ^ [javac] symbol: variable TEZ_RUNTIME_OPTIMIZE_LOCAL_FETCH [javac] location: class TezRuntimeConfiguration [javac] Note: Some input files use or override a deprecated API. [javac] Note: Recompile with -Xlint:deprecation for details. [javac] Note: Some input files use unchecked or unsafe operations. [javac] Note: Recompile with -Xlint:unchecked for details. [javac] 2 errors [javac] 3 warnings
        Hide
        Cheolsoo Park added a comment -

        Quang-Nhat HOANG-XUAN, that looks like your workspace issue. I can build cleanly, and the apache jenkin builds also succeeds-
        https://builds.apache.org/job/Pig-trunk/1623/changes

        Can you try remove ~/.ivy2/cache/org.apache.tez and ~/.m2/repository/org/apache/tez and rebuild?

        Show
        Cheolsoo Park added a comment - Quang-Nhat HOANG-XUAN , that looks like your workspace issue. I can build cleanly, and the apache jenkin builds also succeeds- https://builds.apache.org/job/Pig-trunk/1623/changes Can you try remove ~/.ivy2/cache/org.apache.tez and ~/.m2/repository/org/apache/tez and rebuild?
        Hide
        Quang-Nhat HOANG-XUAN added a comment -

        Compiled perfectly!
        Thank you.

        Show
        Quang-Nhat HOANG-XUAN added a comment - Compiled perfectly! Thank you.
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Resolved Resolved
        14h 26m 1 Cheolsoo Park 05/Aug/14 00:03

          People

          • Assignee:
            Unassigned
            Reporter:
            Quang-Nhat HOANG-XUAN
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development