Giraph
  1. Giraph
  2. GIRAPH-361

Hive output partition parsing is broken

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      While running some jobs I've been seeing that the outputPartition option (-O in HiveGiraphRunner) is not parsing strings correctly.

      1. GIRAPH-361-3.patch
        9 kB
        Nitay Joffe
      2. GIRAPH-361-2.patch
        7 kB
        Nitay Joffe
      3. GIRAPH-361.patch
        7 kB
        Nitay Joffe

        Activity

        Hide
        Nitay Joffe added a comment -

        Here's fix. I replaced the hand-written parsing logic with using Guava's Splitter. With this things are much more consistent and the parsing seems more correct.

        Show
        Nitay Joffe added a comment - Here's fix. I replaced the hand-written parsing logic with using Guava's Splitter. With this things are much more consistent and the parsing seems more correct.
        Hide
        Alessandro Presta added a comment -

        Looks good, +1.
        I'll commit this tomorrow if no one objects.

        Show
        Alessandro Presta added a comment - Looks good, +1. I'll commit this tomorrow if no one objects.
        Hide
        Avery Ching added a comment -

        +1, might as well commit now since only Nitay is using the Hive output stuff and this is in contrib.

        Show
        Avery Ching added a comment - +1, might as well commit now since only Nitay is using the Hive output stuff and this is in contrib.
        Hide
        Alessandro Presta added a comment -

        Rat complains that HiveUtils.java and TestHiveUtils.java are missing the Apache License header. Nitay, can you please add it?

        Show
        Alessandro Presta added a comment - Rat complains that HiveUtils.java and TestHiveUtils.java are missing the Apache License header. Nitay, can you please add it?
        Hide
        Nitay Joffe added a comment -

        Yep, try this one.

        Show
        Nitay Joffe added a comment - Yep, try this one.
        Hide
        Hudson added a comment -

        Integrated in Giraph-trunk-Commit #232 (See https://builds.apache.org/job/Giraph-trunk-Commit/232/)
        Adding missing files from GIRAPH-361 (Revision 1396443)
        GIRAPH-361: Hive output partition parsing is broken (nitay via apresta) (Revision 1396441)

        Result = SUCCESS
        apresta : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1396443
        Files :

        • /giraph/trunk/giraph-formats-contrib/src/main/java/org/apache/giraph/io/hcatalog/HiveUtils.java
        • /giraph/trunk/giraph-formats-contrib/src/test/java/org/apache/giraph/io/hcatalog
        • /giraph/trunk/giraph-formats-contrib/src/test/java/org/apache/giraph/io/hcatalog/TestHiveUtils.java

        apresta : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1396441
        Files :

        • /giraph/trunk/CHANGELOG
        • /giraph/trunk/giraph-formats-contrib/pom.xml
        • /giraph/trunk/giraph-formats-contrib/src/main/java/org/apache/giraph/io/hcatalog/HiveGiraphRunner.java
        Show
        Hudson added a comment - Integrated in Giraph-trunk-Commit #232 (See https://builds.apache.org/job/Giraph-trunk-Commit/232/ ) Adding missing files from GIRAPH-361 (Revision 1396443) GIRAPH-361 : Hive output partition parsing is broken (nitay via apresta) (Revision 1396441) Result = SUCCESS apresta : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1396443 Files : /giraph/trunk/giraph-formats-contrib/src/main/java/org/apache/giraph/io/hcatalog/HiveUtils.java /giraph/trunk/giraph-formats-contrib/src/test/java/org/apache/giraph/io/hcatalog /giraph/trunk/giraph-formats-contrib/src/test/java/org/apache/giraph/io/hcatalog/TestHiveUtils.java apresta : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1396441 Files : /giraph/trunk/CHANGELOG /giraph/trunk/giraph-formats-contrib/pom.xml /giraph/trunk/giraph-formats-contrib/src/main/java/org/apache/giraph/io/hcatalog/HiveGiraphRunner.java
        Hide
        Avery Ching added a comment -

        Since Hudson reported success, resolving.

        Show
        Avery Ching added a comment - Since Hudson reported success, resolving.

          People

          • Assignee:
            Nitay Joffe
            Reporter:
            Nitay Joffe
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development