Hive
  1. Hive
  2. HIVE-3995

PostgreSQL upgrade scripts are not valid

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.11.0
    • Component/s: Metastore
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      I've noticed that scripts for upgrading metastore backed up on PostgreSQL are not valid.

      1. postgre_update_issue.patch
        2 kB
        Jarek Jarcec Cecho
      2. postgre_update_issue.patch
        3 kB
        Jarek Jarcec Cecho

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-h0.21 #1963 (See https://builds.apache.org/job/Hive-trunk-h0.21/1963/)
          HIVE-3995 PostgreSQL upgrade scripts are not valid
          (Jarek and Mark via namit) (Revision 1444332)

          Result = SUCCESS
          namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1444332
          Files :

          • /hive/trunk/metastore/scripts/upgrade/postgres/010-HIVE-3072.postgres.sql
          • /hive/trunk/metastore/scripts/upgrade/postgres/011-HIVE-3649.postgres.sql
          • /hive/trunk/metastore/scripts/upgrade/postgres/012-HIVE-1362.postgres.sql
          Show
          Hudson added a comment - Integrated in Hive-trunk-h0.21 #1963 (See https://builds.apache.org/job/Hive-trunk-h0.21/1963/ ) HIVE-3995 PostgreSQL upgrade scripts are not valid (Jarek and Mark via namit) (Revision 1444332) Result = SUCCESS namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1444332 Files : /hive/trunk/metastore/scripts/upgrade/postgres/010- HIVE-3072 .postgres.sql /hive/trunk/metastore/scripts/upgrade/postgres/011- HIVE-3649 .postgres.sql /hive/trunk/metastore/scripts/upgrade/postgres/012- HIVE-1362 .postgres.sql
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-hadoop2 #114 (See https://builds.apache.org/job/Hive-trunk-hadoop2/114/)
          HIVE-3995 PostgreSQL upgrade scripts are not valid
          (Jarek and Mark via namit) (Revision 1444332)

          Result = FAILURE
          namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1444332
          Files :

          • /hive/trunk/metastore/scripts/upgrade/postgres/010-HIVE-3072.postgres.sql
          • /hive/trunk/metastore/scripts/upgrade/postgres/011-HIVE-3649.postgres.sql
          • /hive/trunk/metastore/scripts/upgrade/postgres/012-HIVE-1362.postgres.sql
          Show
          Hudson added a comment - Integrated in Hive-trunk-hadoop2 #114 (See https://builds.apache.org/job/Hive-trunk-hadoop2/114/ ) HIVE-3995 PostgreSQL upgrade scripts are not valid (Jarek and Mark via namit) (Revision 1444332) Result = FAILURE namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1444332 Files : /hive/trunk/metastore/scripts/upgrade/postgres/010- HIVE-3072 .postgres.sql /hive/trunk/metastore/scripts/upgrade/postgres/011- HIVE-3649 .postgres.sql /hive/trunk/metastore/scripts/upgrade/postgres/012- HIVE-1362 .postgres.sql
          Hide
          Mark Grover added a comment -

          Thanks, Namit!

          Show
          Mark Grover added a comment - Thanks, Namit!
          Hide
          Hudson added a comment -

          Integrated in hive-trunk-hadoop1 #75 (See https://builds.apache.org/job/hive-trunk-hadoop1/75/)
          HIVE-3995 PostgreSQL upgrade scripts are not valid
          (Jarek and Mark via namit) (Revision 1444332)

          Result = ABORTED
          namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1444332
          Files :

          • /hive/trunk/metastore/scripts/upgrade/postgres/010-HIVE-3072.postgres.sql
          • /hive/trunk/metastore/scripts/upgrade/postgres/011-HIVE-3649.postgres.sql
          • /hive/trunk/metastore/scripts/upgrade/postgres/012-HIVE-1362.postgres.sql
          Show
          Hudson added a comment - Integrated in hive-trunk-hadoop1 #75 (See https://builds.apache.org/job/hive-trunk-hadoop1/75/ ) HIVE-3995 PostgreSQL upgrade scripts are not valid (Jarek and Mark via namit) (Revision 1444332) Result = ABORTED namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1444332 Files : /hive/trunk/metastore/scripts/upgrade/postgres/010- HIVE-3072 .postgres.sql /hive/trunk/metastore/scripts/upgrade/postgres/011- HIVE-3649 .postgres.sql /hive/trunk/metastore/scripts/upgrade/postgres/012- HIVE-1362 .postgres.sql
          Hide
          Namit Jain added a comment -

          Committed. Thanks Jarek and Mark

          Show
          Namit Jain added a comment - Committed. Thanks Jarek and Mark
          Hide
          Mark Grover added a comment -

          +1 (non-committer)

          Thanks Jarcec! I left a comment (no action required from your side) on Reviewboard that "false" seems to be the right value to put for existing rows. Perhaps, Gang Tim Liu or [~namitjain], who were involved with HIVE-3649 can confirm.

          I like the new approach of adding the new column as Nullable, updating the existing null values and then setting the column to not-null. The only other approach I can think of is to use a default value (of false) but that seems more risky in the long run.

          Show
          Mark Grover added a comment - +1 (non-committer) Thanks Jarcec! I left a comment (no action required from your side) on Reviewboard that "false" seems to be the right value to put for existing rows. Perhaps, Gang Tim Liu or [~namitjain] , who were involved with HIVE-3649 can confirm. I like the new approach of adding the new column as Nullable, updating the existing null values and then setting the column to not-null. The only other approach I can think of is to use a default value (of false) but that seems more risky in the long run.
          Hide
          Jarek Jarcec Cecho added a comment -

          Attaching updated patch including Mark's notes from review board.

          Show
          Jarek Jarcec Cecho added a comment - Attaching updated patch including Mark's notes from review board.
          Hide
          Mark Grover added a comment -

          Jarcec, thanks for this! The PostgreSQL scripts were not supported until recently, so it's possible the syntax in existing scripts is incorrect.

          I left some comments on the review.

          Show
          Mark Grover added a comment - Jarcec, thanks for this! The PostgreSQL scripts were not supported until recently, so it's possible the syntax in existing scripts is incorrect. I left some comments on the review.
          Hide
          Jarek Jarcec Cecho added a comment -

          I'm attaching patch that is fixing the upgrade scripts. I've tried to assign the jira to myself but it seems that I'm not in the "contributor" group in the jira. Might I ask someone to add me?

          Show
          Jarek Jarcec Cecho added a comment - I'm attaching patch that is fixing the upgrade scripts. I've tried to assign the jira to myself but it seems that I'm not in the "contributor" group in the jira. Might I ask someone to add me?

            People

            • Assignee:
              Jarek Jarcec Cecho
              Reporter:
              Jarek Jarcec Cecho
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development