Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-1625

Override Interpreter settings at User level

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Core
    • Labels:
      None

      Description

      When multiple users share a zeppelin instance, it is sometimes desirable to use the same interpreter with different settings.

      It should be possible an administrator to setup some of the Interpreter settings as overridable by setting a place holder and a default value
      It should be possible for a user to specify the actual values via properties.

      An example,

      For JDBC interpreter, common.max_count specifies the maximum number of rows to displayed.

      We can configure it to be overridden by users using max_rows property as follows:

      Change interpreter settings common.max_count = $

      {max_rows,100}

      If the user using the interpreter has not specified a value for max_rows, then the default value of 100 will be used.

      The user can set property by going to the Properties menu and adding a new property.

      set max_rows=500

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user benoyantony opened a pull request:

          https://github.com/apache/zeppelin/pull/1611

          ZEPPELIN-1625 - Add support of user specific properties

              1. What is this PR for?
                User should be able to define properties and these properties must be passed to Interpreter via Interpreter Context.
                Add functionality so that Interpreter can substitute the its property value with the value specified by the user.
              1. What type of PR is it?
                Feature
              1. Todos
              1. What is the Jira issue?
                ZEPPELIN-1625
              1. How should this be tested?
                unit tests provided
              1. Screenshots (if appropriate)
              1. Questions:
          • Does the licenses files need update? No
          • Is there breaking changes for older versions? No
          • Does this needs documentation? Yes

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/benoyantony/zeppelin master

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/zeppelin/pull/1611.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1611


          commit 971a431de5a4373ae43a0bf664605475298eff78
          Author: Benoy Antony <benoy@apache.org>
          Date: 2016-11-07T22:53:42Z

          ZEPPELIN-1625 - Add support of user specific properties


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user benoyantony opened a pull request: https://github.com/apache/zeppelin/pull/1611 ZEPPELIN-1625 - Add support of user specific properties What is this PR for? User should be able to define properties and these properties must be passed to Interpreter via Interpreter Context. Add functionality so that Interpreter can substitute the its property value with the value specified by the user. What type of PR is it? Feature Todos What is the Jira issue? ZEPPELIN-1625 How should this be tested? unit tests provided Screenshots (if appropriate) Questions: Does the licenses files need update? No Is there breaking changes for older versions? No Does this needs documentation? Yes You can merge this pull request into a Git repository by running: $ git pull https://github.com/benoyantony/zeppelin master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1611.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1611 commit 971a431de5a4373ae43a0bf664605475298eff78 Author: Benoy Antony <benoy@apache.org> Date: 2016-11-07T22:53:42Z ZEPPELIN-1625 - Add support of user specific properties
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user 1ambda commented on the issue:

          https://github.com/apache/zeppelin/pull/1611

          Hi ๐Ÿ˜ƒ @benoyantony!

          I'v checked out your PR and tested, but zeppelin doesn't start rendering 503 page.

          ```
          $ zeppelin-master git:(pr/1611) mvn clean package -DskipTests
          $ ...
          ```

          <img width="363" alt="screen shot 2016-11-08 at 11 28 50 am" src="https://cloud.githubusercontent.com/assets/4968473/20084622/f870f012-a5a6-11e6-89a2-c52ef0c47382.png">

          Show
          githubbot ASF GitHub Bot added a comment - Github user 1ambda commented on the issue: https://github.com/apache/zeppelin/pull/1611 Hi ๐Ÿ˜ƒ @benoyantony! I'v checked out your PR and tested, but zeppelin doesn't start rendering 503 page. ``` $ zeppelin-master git:(pr/1611) mvn clean package -DskipTests $ ... ``` <img width="363" alt="screen shot 2016-11-08 at 11 28 50 am" src="https://cloud.githubusercontent.com/assets/4968473/20084622/f870f012-a5a6-11e6-89a2-c52ef0c47382.png">
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user benoyantony opened a pull request:

          https://github.com/apache/zeppelin/pull/1623

          ZEPPELIN-1625 add support for user specific properties

              1. What is this PR for?
                User should be able to define properties and these properties must be passed to Interpreter via Interpreter Context.
                Add functionality so that Interpreter can substitute the its property value with the value specified by the user.
              1. What type of PR is it?
                Feature
              1. Todos
              1. What is the Jira issue?
                https://issues.apache.org/jira/browse/ZEPPELIN-1625
              1. How should this be tested?
                unit tests provided
                Manual testing steps:
                Add a property, update the property and then delete it
              1. Screenshots (if appropriate)
                Available in documentation
              1. Questions:
          • Does the licenses files need update? NO
          • Is there breaking changes for older versions? NO
          • Does this needs documentation? Yes, Documentation is provided.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/benoyantony/zeppelin master

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/zeppelin/pull/1623.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1623


          commit 0a9e7655d50811988ecd60ab291be44e3927b7c2
          Author: Benoy Antony <benoy@apache.org>
          Date: 2016-11-11T19:06:09Z

          add support for user specific properties


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user benoyantony opened a pull request: https://github.com/apache/zeppelin/pull/1623 ZEPPELIN-1625 add support for user specific properties What is this PR for? User should be able to define properties and these properties must be passed to Interpreter via Interpreter Context. Add functionality so that Interpreter can substitute the its property value with the value specified by the user. What type of PR is it? Feature Todos What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1625 How should this be tested? unit tests provided Manual testing steps: Add a property, update the property and then delete it Screenshots (if appropriate) Available in documentation Questions: Does the licenses files need update? NO Is there breaking changes for older versions? NO Does this needs documentation? Yes, Documentation is provided. You can merge this pull request into a Git repository by running: $ git pull https://github.com/benoyantony/zeppelin master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1623.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1623 commit 0a9e7655d50811988ecd60ab291be44e3927b7c2 Author: Benoy Antony <benoy@apache.org> Date: 2016-11-11T19:06:09Z add support for user specific properties
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user benoyantony closed the pull request at:

          https://github.com/apache/zeppelin/pull/1611

          Show
          githubbot ASF GitHub Bot added a comment - Github user benoyantony closed the pull request at: https://github.com/apache/zeppelin/pull/1611
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user benoyantony commented on the issue:

          https://github.com/apache/zeppelin/pull/1611

          Opened a new pull - https://github.com/apache/zeppelin/pull/1623

          Show
          githubbot ASF GitHub Bot added a comment - Github user benoyantony commented on the issue: https://github.com/apache/zeppelin/pull/1611 Opened a new pull - https://github.com/apache/zeppelin/pull/1623
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user benoyantony closed the pull request at:

          https://github.com/apache/zeppelin/pull/1623

          Show
          githubbot ASF GitHub Bot added a comment - Github user benoyantony closed the pull request at: https://github.com/apache/zeppelin/pull/1623
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user benoyantony reopened a pull request:

          https://github.com/apache/zeppelin/pull/1623

          ZEPPELIN-1625 add support for user specific properties

              1. What is this PR for?
                User should be able to define properties and these properties must be passed to Interpreter via Interpreter Context.
                Add functionality so that Interpreter can substitute the its property value with the value specified by the user.
              1. What type of PR is it?
                Feature
              1. Todos
              1. What is the Jira issue?
                https://issues.apache.org/jira/browse/ZEPPELIN-1625
              1. How should this be tested?
                unit tests provided
                Manual testing steps:
                Add a property, update the property and then delete it
              1. Screenshots (if appropriate)
                Available in documentation
              1. Questions:
          • Does the licenses files need update? NO
          • Is there breaking changes for older versions? NO
          • Does this needs documentation? Yes, Documentation is provided.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/benoyantony/zeppelin master

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/zeppelin/pull/1623.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1623


          commit 0a9e7655d50811988ecd60ab291be44e3927b7c2
          Author: Benoy Antony <benoy@apache.org>
          Date: 2016-11-11T19:06:09Z

          add support for user specific properties

          commit aeead93d2e1bd882e239ef38cbc91a0380eb6057
          Author: Benoy Antony <benoy@apache.org>
          Date: 2016-11-11T20:50:25Z

          Add licenses to new files


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user benoyantony reopened a pull request: https://github.com/apache/zeppelin/pull/1623 ZEPPELIN-1625 add support for user specific properties What is this PR for? User should be able to define properties and these properties must be passed to Interpreter via Interpreter Context. Add functionality so that Interpreter can substitute the its property value with the value specified by the user. What type of PR is it? Feature Todos What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1625 How should this be tested? unit tests provided Manual testing steps: Add a property, update the property and then delete it Screenshots (if appropriate) Available in documentation Questions: Does the licenses files need update? NO Is there breaking changes for older versions? NO Does this needs documentation? Yes, Documentation is provided. You can merge this pull request into a Git repository by running: $ git pull https://github.com/benoyantony/zeppelin master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1623.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1623 commit 0a9e7655d50811988ecd60ab291be44e3927b7c2 Author: Benoy Antony <benoy@apache.org> Date: 2016-11-11T19:06:09Z add support for user specific properties commit aeead93d2e1bd882e239ef38cbc91a0380eb6057 Author: Benoy Antony <benoy@apache.org> Date: 2016-11-11T20:50:25Z Add licenses to new files
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user benoyantony closed the pull request at:

          https://github.com/apache/zeppelin/pull/1623

          Show
          githubbot ASF GitHub Bot added a comment - Github user benoyantony closed the pull request at: https://github.com/apache/zeppelin/pull/1623
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user benoyantony reopened a pull request:

          https://github.com/apache/zeppelin/pull/1623

          ZEPPELIN-1625 add support for user specific properties

              1. What is this PR for?
                User should be able to define properties and these properties must be passed to Interpreter via Interpreter Context.
                Add functionality so that Interpreter can substitute the its property value with the value specified by the user.
              1. What type of PR is it?
                Feature
              1. Todos
              1. What is the Jira issue?
                https://issues.apache.org/jira/browse/ZEPPELIN-1625
              1. How should this be tested?
                unit tests provided
                Manual testing steps:
                Add a property, update the property and then delete it
              1. Screenshots (if appropriate)
                Available in documentation
              1. Questions:
          • Does the licenses files need update? NO
          • Is there breaking changes for older versions? NO
          • Does this needs documentation? Yes, Documentation is provided.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/benoyantony/zeppelin master

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/zeppelin/pull/1623.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1623


          commit 0a9e7655d50811988ecd60ab291be44e3927b7c2
          Author: Benoy Antony <benoy@apache.org>
          Date: 2016-11-11T19:06:09Z

          add support for user specific properties

          commit aeead93d2e1bd882e239ef38cbc91a0380eb6057
          Author: Benoy Antony <benoy@apache.org>
          Date: 2016-11-11T20:50:25Z

          Add licenses to new files

          commit d07f0bb04f99f281d8f9837f3c102529c582f165
          Author: Benoy Antony <benoy@apache.org>
          Date: 2016-11-12T06:19:14Z

          adding license


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user benoyantony reopened a pull request: https://github.com/apache/zeppelin/pull/1623 ZEPPELIN-1625 add support for user specific properties What is this PR for? User should be able to define properties and these properties must be passed to Interpreter via Interpreter Context. Add functionality so that Interpreter can substitute the its property value with the value specified by the user. What type of PR is it? Feature Todos What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1625 How should this be tested? unit tests provided Manual testing steps: Add a property, update the property and then delete it Screenshots (if appropriate) Available in documentation Questions: Does the licenses files need update? NO Is there breaking changes for older versions? NO Does this needs documentation? Yes, Documentation is provided. You can merge this pull request into a Git repository by running: $ git pull https://github.com/benoyantony/zeppelin master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1623.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1623 commit 0a9e7655d50811988ecd60ab291be44e3927b7c2 Author: Benoy Antony <benoy@apache.org> Date: 2016-11-11T19:06:09Z add support for user specific properties commit aeead93d2e1bd882e239ef38cbc91a0380eb6057 Author: Benoy Antony <benoy@apache.org> Date: 2016-11-11T20:50:25Z Add licenses to new files commit d07f0bb04f99f281d8f9837f3c102529c582f165 Author: Benoy Antony <benoy@apache.org> Date: 2016-11-12T06:19:14Z adding license
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user 1ambda commented on the issue:

          https://github.com/apache/zeppelin/pull/1623

          @benoyantony

          Hi, I am trying to test this feature. May I ask 2 things about this PR?

          • What is possible if this feature is merged in terms of usage? (e.g user scenario)
          • Are there another features which can be implemented upon this PR?
          Show
          githubbot ASF GitHub Bot added a comment - Github user 1ambda commented on the issue: https://github.com/apache/zeppelin/pull/1623 @benoyantony Hi, I am trying to test this feature. May I ask 2 things about this PR? What is possible if this feature is merged in terms of usage? (e.g user scenario) Are there another features which can be implemented upon this PR?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user benoyantony commented on the issue:

          https://github.com/apache/zeppelin/pull/1623

          thanks @1ambda , Sorry for the delay.

          1. If this feature is merged, then the properties can be referred in the interpreter properties. Each user can override the property value.

          2. Once the feature is merged, I will have another pull request on the Livy Interpreter which make use of the feature for its properties.

          I have tested this feature at our company. Attached is screenshot where the administrator setup the Livy property - livy.spark.yarn.queue to $

          {queue,defaultQueue}

          ![livy-interpreter-properties](https://cloud.githubusercontent.com/assets/1928906/20405416/c9c1b968-acbd-11e6-8306-6b398445ad94.png)

          In the second screenshot, the user override the queue as benoyqueue. If the user does not override, then the property will remain as defaultQueue.

          ![user-defined-properties](https://cloud.githubusercontent.com/assets/1928906/20405423/d28122d2-acbd-11e6-8b91-d58de2497148.png)

          Show
          githubbot ASF GitHub Bot added a comment - Github user benoyantony commented on the issue: https://github.com/apache/zeppelin/pull/1623 thanks @1ambda , Sorry for the delay. 1. If this feature is merged, then the properties can be referred in the interpreter properties. Each user can override the property value. 2. Once the feature is merged, I will have another pull request on the Livy Interpreter which make use of the feature for its properties. I have tested this feature at our company. Attached is screenshot where the administrator setup the Livy property - livy.spark.yarn.queue to $ {queue,defaultQueue} ! [livy-interpreter-properties] ( https://cloud.githubusercontent.com/assets/1928906/20405416/c9c1b968-acbd-11e6-8306-6b398445ad94.png ) In the second screenshot, the user override the queue as benoyqueue. If the user does not override, then the property will remain as defaultQueue. ! [user-defined-properties] ( https://cloud.githubusercontent.com/assets/1928906/20405423/d28122d2-acbd-11e6-8b91-d58de2497148.png )
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user benoyantony closed the pull request at:

          https://github.com/apache/zeppelin/pull/1623

          Show
          githubbot ASF GitHub Bot added a comment - Github user benoyantony closed the pull request at: https://github.com/apache/zeppelin/pull/1623
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user benoyantony opened a pull request:

          https://github.com/apache/zeppelin/pull/2301

          ZEPPELIN-1625 Override Interpreter settings at User level

              1. What is this PR for?
                When multiple users share a zeppelin instance, it is sometimes desirable to use the same interpreter with different settings.

          It should be possible an administrator to setup some of the Interpreter settings as overridable by setting a place holder and a default value
          It should be possible for a user to specify the actual values via properties.

              1. What type of PR is it?
                Feature
              1. Todos
                Override interpreter setting at notebook level
              1. What is the Jira issue?
                https://issues.apache.org/jira/browse/ZEPPELIN-1625
              1. How should this be tested?

          Unit tests are added.
          Please see screen shots for manual testing

              1. Screenshots

          ![override-interpreter-setting](https://cloud.githubusercontent.com/assets/1928906/25549166/2b5073f4-2c25-11e7-84f7-17ab8fc23537.gif)
          ![set-property](https://cloud.githubusercontent.com/assets/1928906/25549165/2b4e6a96-2c25-11e7-90cd-700afdac83ab.gif)

              1. Questions:
          • Does the licenses files need update? NO
          • Is there breaking changes for older versions? NO
          • Does this needs documentation? Yes , added

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/benoyantony/zeppelin user_properties

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/zeppelin/pull/2301.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #2301


          commit 6972b7cf094ef07a2a1e295169966656f638f361
          Author: Benoy Antony <benoy@apache.org>
          Date: 2017-04-28T22:08:45Z

          ZEPPELIN-1625 Override Interpreter settings at User level


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user benoyantony opened a pull request: https://github.com/apache/zeppelin/pull/2301 ZEPPELIN-1625 Override Interpreter settings at User level What is this PR for? When multiple users share a zeppelin instance, it is sometimes desirable to use the same interpreter with different settings. It should be possible an administrator to setup some of the Interpreter settings as overridable by setting a place holder and a default value It should be possible for a user to specify the actual values via properties. What type of PR is it? Feature Todos Override interpreter setting at notebook level What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1625 How should this be tested? Unit tests are added. Please see screen shots for manual testing Screenshots ! [override-interpreter-setting] ( https://cloud.githubusercontent.com/assets/1928906/25549166/2b5073f4-2c25-11e7-84f7-17ab8fc23537.gif ) ! [set-property] ( https://cloud.githubusercontent.com/assets/1928906/25549165/2b4e6a96-2c25-11e7-90cd-700afdac83ab.gif ) Questions: Does the licenses files need update? NO Is there breaking changes for older versions? NO Does this needs documentation? Yes , added You can merge this pull request into a Git repository by running: $ git pull https://github.com/benoyantony/zeppelin user_properties Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/2301.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2301 commit 6972b7cf094ef07a2a1e295169966656f638f361 Author: Benoy Antony <benoy@apache.org> Date: 2017-04-28T22:08:45Z ZEPPELIN-1625 Override Interpreter settings at User level
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user benoyantony commented on the issue:

          https://github.com/apache/zeppelin/pull/2301

          @Leemoonsoo , Could you please review this feature ?

          Show
          githubbot ASF GitHub Bot added a comment - Github user benoyantony commented on the issue: https://github.com/apache/zeppelin/pull/2301 @Leemoonsoo , Could you please review this feature ?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zjffdu commented on the issue:

          https://github.com/apache/zeppelin/pull/2301

          @benoyantony Did you check ZEPPELIN-1338 ? Your approach seems some kind of workaround to me.

          Show
          githubbot ASF GitHub Bot added a comment - Github user zjffdu commented on the issue: https://github.com/apache/zeppelin/pull/2301 @benoyantony Did you check ZEPPELIN-1338 ? Your approach seems some kind of workaround to me.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user benoyantony commented on the issue:

          https://github.com/apache/zeppelin/pull/2301

          @zjffdu , Thanks for pointing me to ZEPPELIN-1338. I did not know about it. I believe, the intent is same. I agree that the approach is different. We wanted to shield the user from interpreter settings.

          The approach in this change is to specify a property name and default value in the interpreter setting. Users can set a value for the property. When the interpreter is created, then the user's property value is used as the interpreter setting.
          User's properties are persisted.
          One advantage of this approach is to use the same property across multiple interpreter settings.
          We have been using this feature at our company for last 3 months and its been effective.

          I'll review ZEPPELIN-1338 carefully to understand the approach mentioned in that. Thanks again for letting me know of the related effort. Its good to know that its a feature needed by many.

          Show
          githubbot ASF GitHub Bot added a comment - Github user benoyantony commented on the issue: https://github.com/apache/zeppelin/pull/2301 @zjffdu , Thanks for pointing me to ZEPPELIN-1338 . I did not know about it. I believe, the intent is same. I agree that the approach is different. We wanted to shield the user from interpreter settings. The approach in this change is to specify a property name and default value in the interpreter setting. Users can set a value for the property. When the interpreter is created, then the user's property value is used as the interpreter setting. User's properties are persisted. One advantage of this approach is to use the same property across multiple interpreter settings. We have been using this feature at our company for last 3 months and its been effective. I'll review ZEPPELIN-1338 carefully to understand the approach mentioned in that. Thanks again for letting me know of the related effort. Its good to know that its a feature needed by many.

            People

            • Assignee:
              Unassigned
              Reporter:
              benoyantony Benoy Antony
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:

                Development