Sqoop
  1. Sqoop
  2. SQOOP-1142

Sqoop2: Provide creater and last edited by to metadata structures

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.99.2
    • Fix Version/s: 1.99.3
    • Component/s: None
    • Labels:
      None

      Description

      Connection and Job metadata could be created and updated by different users. We need to store "create by" and "last update by" metadata to know who did these operations.

      This is also true for Submission, but things are a little bit different. For a submission, user could update the information through 'start' and 'stop' command, while for most of other time, it's updated by a Sqoop job.

      1. SQOOP-1142.6.patch
        51 kB
        Mengwei Ding
      2. SQOOP-1142.5.patch
        51 kB
        Mengwei Ding
      3. SQOOP-1142.4.patch
        51 kB
        Mengwei Ding
      4. SQOOP-1142.3.patch
        49 kB
        Mengwei Ding
      5. SQOOP-1142.2.patch
        42 kB
        Mengwei Ding
      6. SQOOP-1142.1.patch
        43 kB
        Mengwei Ding

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Sqoop2-hadoop100 #555 (See https://builds.apache.org/job/Sqoop2-hadoop100/555/)
          SQOOP-1142: Sqoop2: Provide creater and last edited by to metadata structures (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=c07c857589bfea989b818017e11c138f5201eebc)

          • repository/repository-derby/src/main/java/org/apache/sqoop/repository/derby/DerbySchemaConstants.java
          • common/src/main/java/org/apache/sqoop/json/SubmissionBean.java
          • common/src/test/java/org/apache/sqoop/json/TestConnectionBean.java
          • repository/repository-derby/src/main/java/org/apache/sqoop/repository/derby/DerbyRepositoryHandler.java
          • shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java
          • common/src/main/java/org/apache/sqoop/json/ConnectionBean.java
          • server/src/main/java/org/apache/sqoop/handler/ConnectionRequestHandler.java
          • core/src/main/java/org/apache/sqoop/framework/JobManager.java
          • shell/src/main/java/org/apache/sqoop/shell/utils/SubmissionDisplayer.java
          • shell/src/main/resources/shell-resource.properties
          • server/src/main/java/org/apache/sqoop/handler/SubmissionRequestHandler.java
          • common/src/main/java/org/apache/sqoop/json/JobBean.java
          • common/src/test/java/org/apache/sqoop/model/TestMAccountableEntity.java
          • core/src/main/java/org/apache/sqoop/request/HttpEventContext.java
          • shell/src/main/java/org/apache/sqoop/shell/core/Constants.java
          • shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java
          • common/src/main/java/org/apache/sqoop/model/MAccountableEntity.java
          • repository/repository-derby/src/main/java/org/apache/sqoop/repository/derby/DerbySchemaQuery.java
          • common/src/main/java/org/apache/sqoop/json/util/FormSerialization.java
          • common/src/test/java/org/apache/sqoop/json/TestSubmissionBean.java
          • server/src/main/java/org/apache/sqoop/handler/JobRequestHandler.java
          • repository/repository-derby/src/test/java/org/apache/sqoop/repository/derby/DerbyTestCase.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Sqoop2-hadoop100 #555 (See https://builds.apache.org/job/Sqoop2-hadoop100/555/ ) SQOOP-1142 : Sqoop2: Provide creater and last edited by to metadata structures (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=c07c857589bfea989b818017e11c138f5201eebc ) repository/repository-derby/src/main/java/org/apache/sqoop/repository/derby/DerbySchemaConstants.java common/src/main/java/org/apache/sqoop/json/SubmissionBean.java common/src/test/java/org/apache/sqoop/json/TestConnectionBean.java repository/repository-derby/src/main/java/org/apache/sqoop/repository/derby/DerbyRepositoryHandler.java shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java common/src/main/java/org/apache/sqoop/json/ConnectionBean.java server/src/main/java/org/apache/sqoop/handler/ConnectionRequestHandler.java core/src/main/java/org/apache/sqoop/framework/JobManager.java shell/src/main/java/org/apache/sqoop/shell/utils/SubmissionDisplayer.java shell/src/main/resources/shell-resource.properties server/src/main/java/org/apache/sqoop/handler/SubmissionRequestHandler.java common/src/main/java/org/apache/sqoop/json/JobBean.java common/src/test/java/org/apache/sqoop/model/TestMAccountableEntity.java core/src/main/java/org/apache/sqoop/request/HttpEventContext.java shell/src/main/java/org/apache/sqoop/shell/core/Constants.java shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java common/src/main/java/org/apache/sqoop/model/MAccountableEntity.java repository/repository-derby/src/main/java/org/apache/sqoop/repository/derby/DerbySchemaQuery.java common/src/main/java/org/apache/sqoop/json/util/FormSerialization.java common/src/test/java/org/apache/sqoop/json/TestSubmissionBean.java server/src/main/java/org/apache/sqoop/handler/JobRequestHandler.java repository/repository-derby/src/test/java/org/apache/sqoop/repository/derby/DerbyTestCase.java
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Sqoop2-hadoop200 #494 (See https://builds.apache.org/job/Sqoop2-hadoop200/494/)
          SQOOP-1142: Sqoop2: Provide creater and last edited by to metadata structures (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=c07c857589bfea989b818017e11c138f5201eebc)

          • common/src/test/java/org/apache/sqoop/json/TestSubmissionBean.java
          • common/src/test/java/org/apache/sqoop/json/TestConnectionBean.java
          • core/src/main/java/org/apache/sqoop/framework/JobManager.java
          • repository/repository-derby/src/main/java/org/apache/sqoop/repository/derby/DerbySchemaConstants.java
          • common/src/main/java/org/apache/sqoop/json/SubmissionBean.java
          • shell/src/main/java/org/apache/sqoop/shell/core/Constants.java
          • shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java
          • common/src/main/java/org/apache/sqoop/json/JobBean.java
          • server/src/main/java/org/apache/sqoop/handler/SubmissionRequestHandler.java
          • server/src/main/java/org/apache/sqoop/handler/ConnectionRequestHandler.java
          • common/src/main/java/org/apache/sqoop/json/util/FormSerialization.java
          • common/src/main/java/org/apache/sqoop/json/ConnectionBean.java
          • common/src/main/java/org/apache/sqoop/model/MAccountableEntity.java
          • shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java
          • shell/src/main/resources/shell-resource.properties
          • shell/src/main/java/org/apache/sqoop/shell/utils/SubmissionDisplayer.java
          • common/src/test/java/org/apache/sqoop/model/TestMAccountableEntity.java
          • core/src/main/java/org/apache/sqoop/request/HttpEventContext.java
          • repository/repository-derby/src/main/java/org/apache/sqoop/repository/derby/DerbySchemaQuery.java
          • repository/repository-derby/src/test/java/org/apache/sqoop/repository/derby/DerbyTestCase.java
          • repository/repository-derby/src/main/java/org/apache/sqoop/repository/derby/DerbyRepositoryHandler.java
          • server/src/main/java/org/apache/sqoop/handler/JobRequestHandler.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Sqoop2-hadoop200 #494 (See https://builds.apache.org/job/Sqoop2-hadoop200/494/ ) SQOOP-1142 : Sqoop2: Provide creater and last edited by to metadata structures (jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=c07c857589bfea989b818017e11c138f5201eebc ) common/src/test/java/org/apache/sqoop/json/TestSubmissionBean.java common/src/test/java/org/apache/sqoop/json/TestConnectionBean.java core/src/main/java/org/apache/sqoop/framework/JobManager.java repository/repository-derby/src/main/java/org/apache/sqoop/repository/derby/DerbySchemaConstants.java common/src/main/java/org/apache/sqoop/json/SubmissionBean.java shell/src/main/java/org/apache/sqoop/shell/core/Constants.java shell/src/main/java/org/apache/sqoop/shell/ShowConnectionFunction.java common/src/main/java/org/apache/sqoop/json/JobBean.java server/src/main/java/org/apache/sqoop/handler/SubmissionRequestHandler.java server/src/main/java/org/apache/sqoop/handler/ConnectionRequestHandler.java common/src/main/java/org/apache/sqoop/json/util/FormSerialization.java common/src/main/java/org/apache/sqoop/json/ConnectionBean.java common/src/main/java/org/apache/sqoop/model/MAccountableEntity.java shell/src/main/java/org/apache/sqoop/shell/ShowJobFunction.java shell/src/main/resources/shell-resource.properties shell/src/main/java/org/apache/sqoop/shell/utils/SubmissionDisplayer.java common/src/test/java/org/apache/sqoop/model/TestMAccountableEntity.java core/src/main/java/org/apache/sqoop/request/HttpEventContext.java repository/repository-derby/src/main/java/org/apache/sqoop/repository/derby/DerbySchemaQuery.java repository/repository-derby/src/test/java/org/apache/sqoop/repository/derby/DerbyTestCase.java repository/repository-derby/src/main/java/org/apache/sqoop/repository/derby/DerbyRepositoryHandler.java server/src/main/java/org/apache/sqoop/handler/JobRequestHandler.java
          Jarek Jarcec Cecho made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          Jarek Jarcec Cecho added a comment -

          Thank you Mengwei Ding for your contribution!

          Show
          Jarek Jarcec Cecho added a comment - Thank you Mengwei Ding for your contribution!
          Hide
          ASF subversion and git services added a comment -

          Commit c07c857589bfea989b818017e11c138f5201eebc in branch refs/heads/sqoop2 from Jarek Jarcec Cecho
          [ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=c07c857 ]

          SQOOP-1142: Sqoop2: Provide creater and last edited by to metadata structures

          (Mengwei Ding via Jarek Jarcec Cecho)

          Show
          ASF subversion and git services added a comment - Commit c07c857589bfea989b818017e11c138f5201eebc in branch refs/heads/sqoop2 from Jarek Jarcec Cecho [ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=c07c857 ] SQOOP-1142 : Sqoop2: Provide creater and last edited by to metadata structures (Mengwei Ding via Jarek Jarcec Cecho)
          Hide
          Sqoop QA bot added a comment -

          Here are the results of testing the latest attachment
          https://issues.apache.org/jira/secure/attachment/12598052/SQOOP-1142.6.patch against branch sqoop2.

          Overall: +1 all checks pass

          SUCCESS: Clean was successful
          SUCCESS: Patch applied correctly
          SUCCESS: Patch compiled
          SUCCESS: All tests passed

          Console output: https://builds.apache.org/job/PreCommit-SQOOP-Build/121/console

          This message is automatically generated.

          Show
          Sqoop QA bot added a comment - Here are the results of testing the latest attachment https://issues.apache.org/jira/secure/attachment/12598052/SQOOP-1142.6.patch against branch sqoop2. Overall: +1 all checks pass SUCCESS: Clean was successful SUCCESS: Patch applied correctly SUCCESS: Patch compiled SUCCESS: All tests passed Console output: https://builds.apache.org/job/PreCommit-SQOOP-Build/121/console This message is automatically generated.
          Mengwei Ding made changes -
          Attachment SQOOP-1142.6.patch [ 12598052 ]
          Hide
          Sqoop QA bot added a comment -

          Here are the results of testing the latest attachment
          https://issues.apache.org/jira/secure/attachment/12598003/SQOOP-1142.5.patch against branch sqoop2.

          Overall: +1 all checks pass

          SUCCESS: Clean was successful
          SUCCESS: Patch applied correctly
          SUCCESS: Patch compiled
          SUCCESS: All tests passed

          Console output: https://builds.apache.org/job/PreCommit-SQOOP-Build/120/console

          This message is automatically generated.

          Show
          Sqoop QA bot added a comment - Here are the results of testing the latest attachment https://issues.apache.org/jira/secure/attachment/12598003/SQOOP-1142.5.patch against branch sqoop2. Overall: +1 all checks pass SUCCESS: Clean was successful SUCCESS: Patch applied correctly SUCCESS: Patch compiled SUCCESS: All tests passed Console output: https://builds.apache.org/job/PreCommit-SQOOP-Build/120/console This message is automatically generated.
          Mengwei Ding made changes -
          Attachment SQOOP-1142.5.patch [ 12598003 ]
          Hide
          Sqoop QA bot added a comment -

          Here are the results of testing the latest attachment
          https://issues.apache.org/jira/secure/attachment/12595678/SQOOP-1142.4.patch against branch sqoop2.

          Overall: +1 all checks pass

          SUCCESS: Clean was successful
          SUCCESS: Patch applied correctly
          SUCCESS: Patch compiled
          SUCCESS: All tests passed

          Console output: https://builds.apache.org/job/PreCommit-SQOOP-Build/109/console

          This message is automatically generated.

          Show
          Sqoop QA bot added a comment - Here are the results of testing the latest attachment https://issues.apache.org/jira/secure/attachment/12595678/SQOOP-1142.4.patch against branch sqoop2. Overall: +1 all checks pass SUCCESS: Clean was successful SUCCESS: Patch applied correctly SUCCESS: Patch compiled SUCCESS: All tests passed Console output: https://builds.apache.org/job/PreCommit-SQOOP-Build/109/console This message is automatically generated.
          Mengwei Ding made changes -
          Attachment SQOOP-1142.4.patch [ 12595678 ]
          Hide
          Sqoop QA bot added a comment -

          Here are the results of testing the latest attachment
          https://issues.apache.org/jira/secure/attachment/12593988/SQOOP-1142.3.patch against branch sqoop2.

          Overall: +1 all checks pass

          SUCCESS: Clean was successful
          SUCCESS: Patch applied correctly
          SUCCESS: Patch compiled
          SUCCESS: All tests passed

          Console output: https://builds.apache.org/job/PreCommit-SQOOP-Build/90/console

          This message is automatically generated.

          Show
          Sqoop QA bot added a comment - Here are the results of testing the latest attachment https://issues.apache.org/jira/secure/attachment/12593988/SQOOP-1142.3.patch against branch sqoop2. Overall: +1 all checks pass SUCCESS: Clean was successful SUCCESS: Patch applied correctly SUCCESS: Patch compiled SUCCESS: All tests passed Console output: https://builds.apache.org/job/PreCommit-SQOOP-Build/90/console This message is automatically generated.
          Mengwei Ding made changes -
          Attachment SQOOP-1142.3.patch [ 12593988 ]
          Mengwei Ding made changes -
          Attachment SQOOP-1142.3.patch [ 12593761 ]
          Hide
          Sqoop QA bot added a comment -

          Here are the results of testing the latest attachment
          https://issues.apache.org/jira/secure/attachment/12593761/SQOOP-1142.3.patch against branch sqoop2.

          Overall: +1 all checks pass

          SUCCESS: Clean was successful
          SUCCESS: Patch applied correctly
          SUCCESS: Patch compiled
          SUCCESS: All tests passed

          Console output: https://builds.apache.org/job/PreCommit-SQOOP-Build/87/console

          This message is automatically generated.

          Show
          Sqoop QA bot added a comment - Here are the results of testing the latest attachment https://issues.apache.org/jira/secure/attachment/12593761/SQOOP-1142.3.patch against branch sqoop2. Overall: +1 all checks pass SUCCESS: Clean was successful SUCCESS: Patch applied correctly SUCCESS: Patch compiled SUCCESS: All tests passed Console output: https://builds.apache.org/job/PreCommit-SQOOP-Build/87/console This message is automatically generated.
          Hide
          Sqoop QA bot added a comment -

          Here are the results of testing the latest attachment
          https://issues.apache.org/jira/secure/attachment/12593761/SQOOP-1142.3.patch against branch sqoop2.

          Overall: +1 all checks pass

          SUCCESS: Clean was successful
          SUCCESS: Patch applied correctly
          SUCCESS: Patch compiled
          SUCCESS: All tests passed

          Console output: https://builds.apache.org/job/PreCommit-SQOOP-Build/84/console

          This message is automatically generated.

          Show
          Sqoop QA bot added a comment - Here are the results of testing the latest attachment https://issues.apache.org/jira/secure/attachment/12593761/SQOOP-1142.3.patch against branch sqoop2. Overall: +1 all checks pass SUCCESS: Clean was successful SUCCESS: Patch applied correctly SUCCESS: Patch compiled SUCCESS: All tests passed Console output: https://builds.apache.org/job/PreCommit-SQOOP-Build/84/console This message is automatically generated.
          Mengwei Ding made changes -
          Attachment SQOOP-1142.3.patch [ 12593761 ]
          Hide
          Sqoop QA bot added a comment -

          Here are the results of testing the latest attachment
          https://issues.apache.org/jira/secure/attachment/12593003/SQOOP-1142.2.patch against branch sqoop2.

          Overall: +1 all checks pass

          SUCCESS: Clean was successful
          SUCCESS: Patch applied correctly
          SUCCESS: Patch compiled
          SUCCESS: All tests passed

          Console output: https://builds.apache.org/job/PreCommit-SQOOP-Build/76/console

          This message is automatically generated.

          Show
          Sqoop QA bot added a comment - Here are the results of testing the latest attachment https://issues.apache.org/jira/secure/attachment/12593003/SQOOP-1142.2.patch against branch sqoop2. Overall: +1 all checks pass SUCCESS: Clean was successful SUCCESS: Patch applied correctly SUCCESS: Patch compiled SUCCESS: All tests passed Console output: https://builds.apache.org/job/PreCommit-SQOOP-Build/76/console This message is automatically generated.
          Mengwei Ding made changes -
          Attachment SQOOP-1142.2.patch [ 12593003 ]
          Hide
          Mengwei Ding added a comment -

          Update the way to get username in the server side.

          Show
          Mengwei Ding added a comment - Update the way to get username in the server side.
          Hide
          Sqoop QA bot added a comment -

          Here are the results of testing the latest attachment
          https://issues.apache.org/jira/secure/attachment/12592883/SQOOP-1142.1.patch against branch sqoop2.

          Overall: +1 all checks pass

          SUCCESS: Clean was successful
          SUCCESS: Patch applied correctly
          SUCCESS: Patch compiled
          SUCCESS: All tests passed

          Console output: https://builds.apache.org/job/PreCommit-SQOOP-Build/74/console

          This message is automatically generated.

          Show
          Sqoop QA bot added a comment - Here are the results of testing the latest attachment https://issues.apache.org/jira/secure/attachment/12592883/SQOOP-1142.1.patch against branch sqoop2. Overall: +1 all checks pass SUCCESS: Clean was successful SUCCESS: Patch applied correctly SUCCESS: Patch compiled SUCCESS: All tests passed Console output: https://builds.apache.org/job/PreCommit-SQOOP-Build/74/console This message is automatically generated.
          Mengwei Ding made changes -
          Issue Type Bug [ 1 ] Improvement [ 4 ]
          Mengwei Ding made changes -
          Description Connection and Job metadata could be created and updated by different users. We need to store "create by" and "last update by" metadata to know who did these operations.

          This is also true for Submission, but things are a little bit different. For a submission, user could update the information through 'start' and 'stop' command, while for most of other time, it's updated by a Sqoop job.
          Mengwei Ding made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Mengwei Ding added a comment -

          Submit patch for review.

          Show
          Mengwei Ding added a comment - Submit patch for review.
          Mengwei Ding made changes -
          Attachment SQOOP-1142.1.patch [ 12592883 ]
          Mengwei Ding made changes -
          Field Original Value New Value
          Remote Link This issue links to "ReviewBoard (Web Link)" [ 12424 ]
          Mengwei Ding created issue -

            People

            • Assignee:
              Mengwei Ding
              Reporter:
              Mengwei Ding
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development