Hive
  1. Hive
  2. HIVE-3995

PostgreSQL upgrade scripts are not valid

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.11.0
    • Component/s: Metastore
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      I've noticed that scripts for upgrading metastore backed up on PostgreSQL are not valid.

      1. postgre_update_issue.patch
        2 kB
        Jarek Jarcec Cecho
      2. postgre_update_issue.patch
        3 kB
        Jarek Jarcec Cecho

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-h0.21 #1963 (See https://builds.apache.org/job/Hive-trunk-h0.21/1963/)
          HIVE-3995 PostgreSQL upgrade scripts are not valid
          (Jarek and Mark via namit) (Revision 1444332)

          Result = SUCCESS
          namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1444332
          Files :

          • /hive/trunk/metastore/scripts/upgrade/postgres/010-HIVE-3072.postgres.sql
          • /hive/trunk/metastore/scripts/upgrade/postgres/011-HIVE-3649.postgres.sql
          • /hive/trunk/metastore/scripts/upgrade/postgres/012-HIVE-1362.postgres.sql
          Show
          Hudson added a comment - Integrated in Hive-trunk-h0.21 #1963 (See https://builds.apache.org/job/Hive-trunk-h0.21/1963/ ) HIVE-3995 PostgreSQL upgrade scripts are not valid (Jarek and Mark via namit) (Revision 1444332) Result = SUCCESS namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1444332 Files : /hive/trunk/metastore/scripts/upgrade/postgres/010- HIVE-3072 .postgres.sql /hive/trunk/metastore/scripts/upgrade/postgres/011- HIVE-3649 .postgres.sql /hive/trunk/metastore/scripts/upgrade/postgres/012- HIVE-1362 .postgres.sql
          Hide
          Hudson added a comment -

          Integrated in Hive-trunk-hadoop2 #114 (See https://builds.apache.org/job/Hive-trunk-hadoop2/114/)
          HIVE-3995 PostgreSQL upgrade scripts are not valid
          (Jarek and Mark via namit) (Revision 1444332)

          Result = FAILURE
          namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1444332
          Files :

          • /hive/trunk/metastore/scripts/upgrade/postgres/010-HIVE-3072.postgres.sql
          • /hive/trunk/metastore/scripts/upgrade/postgres/011-HIVE-3649.postgres.sql
          • /hive/trunk/metastore/scripts/upgrade/postgres/012-HIVE-1362.postgres.sql
          Show
          Hudson added a comment - Integrated in Hive-trunk-hadoop2 #114 (See https://builds.apache.org/job/Hive-trunk-hadoop2/114/ ) HIVE-3995 PostgreSQL upgrade scripts are not valid (Jarek and Mark via namit) (Revision 1444332) Result = FAILURE namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1444332 Files : /hive/trunk/metastore/scripts/upgrade/postgres/010- HIVE-3072 .postgres.sql /hive/trunk/metastore/scripts/upgrade/postgres/011- HIVE-3649 .postgres.sql /hive/trunk/metastore/scripts/upgrade/postgres/012- HIVE-1362 .postgres.sql
          Hide
          Mark Grover added a comment -

          Thanks, Namit!

          Show
          Mark Grover added a comment - Thanks, Namit!
          Hide
          Hudson added a comment -

          Integrated in hive-trunk-hadoop1 #75 (See https://builds.apache.org/job/hive-trunk-hadoop1/75/)
          HIVE-3995 PostgreSQL upgrade scripts are not valid
          (Jarek and Mark via namit) (Revision 1444332)

          Result = ABORTED
          namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1444332
          Files :

          • /hive/trunk/metastore/scripts/upgrade/postgres/010-HIVE-3072.postgres.sql
          • /hive/trunk/metastore/scripts/upgrade/postgres/011-HIVE-3649.postgres.sql
          • /hive/trunk/metastore/scripts/upgrade/postgres/012-HIVE-1362.postgres.sql
          Show
          Hudson added a comment - Integrated in hive-trunk-hadoop1 #75 (See https://builds.apache.org/job/hive-trunk-hadoop1/75/ ) HIVE-3995 PostgreSQL upgrade scripts are not valid (Jarek and Mark via namit) (Revision 1444332) Result = ABORTED namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1444332 Files : /hive/trunk/metastore/scripts/upgrade/postgres/010- HIVE-3072 .postgres.sql /hive/trunk/metastore/scripts/upgrade/postgres/011- HIVE-3649 .postgres.sql /hive/trunk/metastore/scripts/upgrade/postgres/012- HIVE-1362 .postgres.sql
          Hide
          Namit Jain added a comment -

          Committed. Thanks Jarek and Mark

          Show
          Namit Jain added a comment - Committed. Thanks Jarek and Mark
          Hide
          Mark Grover added a comment -

          +1 (non-committer)

          Thanks Jarcec! I left a comment (no action required from your side) on Reviewboard that "false" seems to be the right value to put for existing rows. Perhaps, Gang Tim Liu or Namit Jain, who were involved with HIVE-3649 can confirm.

          I like the new approach of adding the new column as Nullable, updating the existing null values and then setting the column to not-null. The only other approach I can think of is to use a default value (of false) but that seems more risky in the long run.

          Show
          Mark Grover added a comment - +1 (non-committer) Thanks Jarcec! I left a comment (no action required from your side) on Reviewboard that "false" seems to be the right value to put for existing rows. Perhaps, Gang Tim Liu or Namit Jain , who were involved with HIVE-3649 can confirm. I like the new approach of adding the new column as Nullable, updating the existing null values and then setting the column to not-null. The only other approach I can think of is to use a default value (of false) but that seems more risky in the long run.
          Hide
          Jarek Jarcec Cecho added a comment -

          Attaching updated patch including Mark's notes from review board.

          Show
          Jarek Jarcec Cecho added a comment - Attaching updated patch including Mark's notes from review board.
          Hide
          Mark Grover added a comment -

          Jarcec, thanks for this! The PostgreSQL scripts were not supported until recently, so it's possible the syntax in existing scripts is incorrect.

          I left some comments on the review.

          Show
          Mark Grover added a comment - Jarcec, thanks for this! The PostgreSQL scripts were not supported until recently, so it's possible the syntax in existing scripts is incorrect. I left some comments on the review.
          Hide
          Jarek Jarcec Cecho added a comment -

          I'm attaching patch that is fixing the upgrade scripts. I've tried to assign the jira to myself but it seems that I'm not in the "contributor" group in the jira. Might I ask someone to add me?

          Show
          Jarek Jarcec Cecho added a comment - I'm attaching patch that is fixing the upgrade scripts. I've tried to assign the jira to myself but it seems that I'm not in the "contributor" group in the jira. Might I ask someone to add me?

            People

            • Assignee:
              Jarek Jarcec Cecho
              Reporter:
              Jarek Jarcec Cecho
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development