Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-3740 Query generator support for Kudu
  3. IMPALA-4340

validate postgres 9.5 for random query generator reference database

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: Impala 2.8.0
    • Fix Version/s: Impala 2.8.0
    • Component/s: Infrastructure
    • Labels:
      None

      Description

      Currently, the random query generator is run against postgres 9.3 for a reference database, but it needs postgres 9.5 features to better verify Impala/Kudu functionality. Specifically it's the ON CONFLICT clause here https://www.postgresql.org/docs/9.5/static/sql-insert.html . This is needed both to handle INSERT with implied IGNORE, which Impala will do, and UPSERT.

      The validation should show that data migration and loading and query execution and comparison is possible. It's mostly a smoke/sanity test.

        Activity

        Hide
        mikesbrown Michael Brown added a comment -

        This is already in progress, and I'm mostly done with it.

        Show
        mikesbrown Michael Brown added a comment - This is already in progress, and I'm mostly done with it.
        Hide
        mikesbrown Michael Brown added a comment -

        Looks like we're in good shape. I can't find any reason that it's not safe to use postgres 9.5 for the query generator. https://gerrit.cloudera.org/#/c/4846/

        Show
        mikesbrown Michael Brown added a comment - Looks like we're in good shape. I can't find any reason that it's not safe to use postgres 9.5 for the query generator. https://gerrit.cloudera.org/#/c/4846/
        Hide
        mikesbrown Michael Brown added a comment -

        commit 7fc31b534d4c5cb118c559e16556a6c1ae6ca7fc
        Author: Michael Brown <mikeb@cloudera.com>
        Date: Tue Oct 25 12:36:31 2016 -0700

        IMPALA-4340: explain how to install postgresql-9.5 or higher

        The random query generator needs to compare against PostgresQL 9.5 or
        higher to take advantage of some of the more recent features, especially
        as it pertains to Impala/Kudu INSERT and UPSERT queries. Developers
        will need assistance setting up their development environments if they
        need to use the random query generator.

        This patch provides instructions on how to do so. We provide
        instructions, not automation, since this will have side-effects on
        developers' workstations: we can't presume to know how a developer might
        want to install or configure postgres, and we haven't tested on anything
        except our own development environment.

        Change-Id: I1e3b510120451fcb5af97145fa47ccb4c53f00d9
        Reviewed-on: http://gerrit.cloudera.org:8080/4846
        Reviewed-by: Alex Behm <alex.behm@cloudera.com>
        Tested-by: Tim Armstrong <tarmstrong@cloudera.com>

        Show
        mikesbrown Michael Brown added a comment - commit 7fc31b534d4c5cb118c559e16556a6c1ae6ca7fc Author: Michael Brown <mikeb@cloudera.com> Date: Tue Oct 25 12:36:31 2016 -0700 IMPALA-4340 : explain how to install postgresql-9.5 or higher The random query generator needs to compare against PostgresQL 9.5 or higher to take advantage of some of the more recent features, especially as it pertains to Impala/Kudu INSERT and UPSERT queries. Developers will need assistance setting up their development environments if they need to use the random query generator. This patch provides instructions on how to do so. We provide instructions, not automation, since this will have side-effects on developers' workstations: we can't presume to know how a developer might want to install or configure postgres, and we haven't tested on anything except our own development environment. Change-Id: I1e3b510120451fcb5af97145fa47ccb4c53f00d9 Reviewed-on: http://gerrit.cloudera.org:8080/4846 Reviewed-by: Alex Behm <alex.behm@cloudera.com> Tested-by: Tim Armstrong <tarmstrong@cloudera.com>

          People

          • Assignee:
            mikesbrown Michael Brown
            Reporter:
            mikesbrown Michael Brown
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development