Uploaded image for project: 'Infrastructure'
  1. Infrastructure
  2. INFRA-15075

Admin rights for continuous integration on MXNet to https://github.com/MXNetEdgeBot

    Details

    • Type: Github Integration
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Fix Version/s: Sep 2017
    • Component/s: Github
    • Labels:
      None
    • Project:
      Incubator
    • Git Repository Name:
      incubator-mxnet
    • Github Integration:
      Enable

      Activity

      Hide
      gstein Greg Stein added a comment -
      Talk to your project Mentors.
      Show
      gstein Greg Stein added a comment - Talk to your project Mentors.
      Hide
      larroy Pedro Larroy added a comment - - edited Reporter
      Hi Greg, all.

      Then would it be an option that I sign the ICLA and do it from my personal account with an OAuth token as you described? I'm already a contributor to MXNet. What would be the next steps to do that?
      Thank you so much for your help.

      Pedro.
      Show
      larroy Pedro Larroy added a comment - - edited Reporter Hi Greg, all. Then would it be an option that I sign the ICLA and do it from my personal account with an OAuth token as you described? I'm already a contributor to MXNet. What would be the next steps to do that? Thank you so much for your help. Pedro.
      Hide
      gstein Greg Stein added a comment -
      Hi Pedro,

      External Jenkins is fine. If you'd like, we can attach those slaves to the ASF master. We've done that plenty before. Please open a ticket for when you're ready to do that.

      Regarding use of the Status API. You can do that today, if you generate an OAuth token from one of your group's users with write access to the repository (meaning: an ICLA on file, linked to a GitHub account, and 2FA enabled. The OAuth token can then be stored on your private slaves, and used to manipulate the status values.

      We cannot give a 'bot write access because it doesn't have an ICLA on file. So you'll need to do this with one of the committers' accounts.

      Cheers,
      Greg Stein, InfraAdmin, ASF
      Show
      gstein Greg Stein added a comment - Hi Pedro, External Jenkins is fine. If you'd like, we can attach those slaves to the ASF master. We've done that plenty before. Please open a ticket for when you're ready to do that. Regarding use of the Status API. You can do that today, if you generate an OAuth token from one of your group's users with write access to the repository (meaning: an ICLA on file, linked to a GitHub account, and 2FA enabled. The OAuth token can then be stored on your private slaves, and used to manipulate the status values. We cannot give a 'bot write access because it doesn't have an ICLA on file. So you'll need to do this with one of the committers' accounts. Cheers, Greg Stein, InfraAdmin, ASF
      Hide
      larroy Pedro Larroy added a comment - Reporter
      Hi John, Daniel

      Thanks a lot for your responses. The reason to use an external Jenkins is that we want to test in real hardware / mobile / embedded. We could do that by using your Jenkins as master and setting up our slaves.

      Another option is to make an exception for this case. This is up to you to decide. For us would be best if you could make an exception, but this is up to you.

      Thanks.



      Show
      larroy Pedro Larroy added a comment - Reporter Hi John, Daniel Thanks a lot for your responses. The reason to use an external Jenkins is that we want to test in real hardware / mobile / embedded. We could do that by using your Jenkins as master and setting up our slaves. Another option is to make an exception for this case. This is up to you to decide. For us would be best if you could make an exception, but this is up to you. Thanks.
      Hide
      johndament John D. Ament added a comment -
      [~larroy] I've created INFRA-15076 for what the proper fix for this should be. One of the issues is that you're using an external Jenkins. We want you to use our resources, as a part of your becoming a TLP here at Apache. I believe we can solve your ask, but using our Jenkins (where we are managing the permissions) rather than your private Jenkins (where we as outsiders can't even see the links you've sent out).
      Show
      johndament John D. Ament added a comment - [~larroy] I've created INFRA-15076 for what the proper fix for this should be. One of the issues is that you're using an external Jenkins. We want you to use our resources, as a part of your becoming a TLP here at Apache. I believe we can solve your ask, but using our Jenkins (where we are managing the permissions) rather than your private Jenkins (where we as outsiders can't even see the links you've sent out).
      Hide
      larroy Pedro Larroy added a comment - - edited Reporter
      The user https://github.com/nswamy is setting the status on PRs for example, in addition to appveyor. If we can't set the status on PRs it increases the work on reviewers since they can't just see that the PR breaks the build with the github integration.

      We are happy to host this service and activities, one of the reasons is that on device testing needs specialized hardware and dedicated slaves.

      Do you prefer to host this infrastructure in Apache? If that's what you imply with your previous comment, could you indicate what would be the steps needed to do this?

      Otherwise how can we work together so we get write access for the bot? Could you provide a solution or give an alternative instead of just closing the doors?

      Thank you very much for your kind cooperation.
      Show
      larroy Pedro Larroy added a comment - - edited Reporter The user https://github.com/nswamy is setting the status on PRs for example, in addition to appveyor. If we can't set the status on PRs it increases the work on reviewers since they can't just see that the PR breaks the build with the github integration. We are happy to host this service and activities, one of the reasons is that on device testing needs specialized hardware and dedicated slaves. Do you prefer to host this infrastructure in Apache? If that's what you imply with your previous comment, could you indicate what would be the steps needed to do this? Otherwise how can we work together so we get write access for the bot? Could you provide a solution or give an alternative instead of just closing the doors? Thank you very much for your kind cooperation.
      Hide
      larroy Pedro Larroy added a comment - Reporter
      We can also create another bot account and share only the Oauth token so the credentials remain under your control. Would that work for you? In any case there's not much practical difference. You could always revoke the permissions in case there's any problem in both cases.
      Show
      larroy Pedro Larroy added a comment - Reporter We can also create another bot account and share only the Oauth token so the credentials remain under your control. Would that work for you? In any case there's not much practical difference. You could always revoke the permissions in case there's any problem in both cases.
      Hide
      humbedooh Daniel Gruno added a comment -
      What other accounts?
      If you host something out of our complete control, you don't get write access - it's as simple as that.
      Show
      humbedooh Daniel Gruno added a comment - What other accounts? If you host something out of our complete control, you don't get write access - it's as simple as that.
      Hide
      larroy Pedro Larroy added a comment - - edited Reporter
      How are we supposed to set statuses then? How are other accounts setting statuses?
      The bot needs write access (not admin access if you manually set the hooks as you did) and just to the mxnet repository.

      Can you please provide a solution?
      Show
      larroy Pedro Larroy added a comment - - edited Reporter How are we supposed to set statuses then? How are other accounts setting statuses? The bot needs write access (not admin access if you manually set the hooks as you did) and just to the mxnet repository. Can you please provide a solution?
      Hide
      larroy Pedro Larroy added a comment - Reporter
      The permissions should be similar to user: https://github.com/nswamy for example as it's setting the statuses:

      https://help.github.com/articles/repository-permission-levels-for-an-organization/
      Show
      larroy Pedro Larroy added a comment - Reporter The permissions should be similar to user: https://github.com/nswamy for example as it's setting the statuses: https://help.github.com/articles/repository-permission-levels-for-an-organization/
      Hide
      humbedooh Daniel Gruno added a comment -
      As stated before, we cannot and will not give write access to bots outside our control.
      Show
      humbedooh Daniel Gruno added a comment - As stated before, we cannot and will not give write access to bots outside our control.
      Hide
      larroy Pedro Larroy added a comment - Reporter
      I have some errors setting the commit status:

      INFO: FileNotFoundException means that the credentials Jenkins is using is probably wrong. Or the user account does not have write access to the repo.
      java.io.FileNotFoundException: {"message":"Not Found","documentation_url":"https://developer.github.com/v3"}
              at org.kohsuke.github.Requester.handleApiError(Requester.java:660)
              at org.kohsuke.github.Requester._to(Requester.java:285)
              at org.kohsuke.github.Requester.to(Requester.java:226)
              at org.kohsuke.github.GHRepository.createCommitStatus(GHRepository.java:1000)
              at org.jenkinsci.plugins.ghprb.extensions.status.GhprbSimpleStatus.onBuildTriggered(GhprbSimpleStatus.java:144)
              at org.jenkinsci.plugins.ghprb.GhprbBuilds.build(GhprbBuilds.java:74)
              at org.jenkinsci.plugins.ghprb.GhprbPullRequest.build(GhprbPullRequest.java:481)
              at org.jenkinsci.plugins.ghprb.GhprbPullRequest.tryBuild(GhprbPullRequest.java:474)
              at org.jenkinsci.plugins.ghprb.GhprbPullRequest.check(GhprbPullRequest.java:166)
              at org.jenkinsci.plugins.ghprb.GhprbRepository.onPullRequestHook(GhprbRepository.java:382)
              at org.jenkinsci.plugins.ghprb.GhprbTrigger.handlePR(GhprbTrigger.java:676)
              at org.jenkinsci.plugins.ghprb.GhprbRootAction$2.run(GhprbRootAction.java:252)
      Caused by: java.io.FileNotFoundException: https://api.github.com/repos/apache/incubator-mxnet/statuses/042bb58b9bb6893fae74002543142cd4bbfe6e73
              at sun.reflect.GeneratedConstructorAccessor121.newInstance(Unknown Source)
              at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
              at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
              at sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1926)
              at sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1921)
              at java.security.AccessController.doPrivileged(Native Method)
              at sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1920)
              at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1490)
              at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474)
              at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254)
              at org.kohsuke.github.Requester.parse(Requester.java:602)
              at org.kohsuke.github.Requester.parse(Requester.java:584)
              at org.kohsuke.github.Requester._to(Requester.java:264)
              ... 10 more
      Caused by: java.io.FileNotFoundException: https://api.github.com/repos/apache/incubator-mxnet/statuses/042bb58b9bb6893fae74002543142cd4bbfe6e73
              at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1872)
              at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474)
              at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
              at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:338)
              at org.kohsuke.github.Requester.parse(Requester.java:592)
              ... 12 more


      The create status needs write permissions on the mxnet repository in addition to the hooks. Could you please also add that?
      Show
      larroy Pedro Larroy added a comment - Reporter I have some errors setting the commit status: INFO: FileNotFoundException means that the credentials Jenkins is using is probably wrong. Or the user account does not have write access to the repo. java.io.FileNotFoundException: {"message":"Not Found","documentation_url":" https://developer.github.com/v3 "}         at org.kohsuke.github.Requester.handleApiError(Requester.java:660)         at org.kohsuke.github.Requester._to(Requester.java:285)         at org.kohsuke.github.Requester.to(Requester.java:226)         at org.kohsuke.github.GHRepository.createCommitStatus(GHRepository.java:1000)         at org.jenkinsci.plugins.ghprb.extensions.status.GhprbSimpleStatus.onBuildTriggered(GhprbSimpleStatus.java:144)         at org.jenkinsci.plugins.ghprb.GhprbBuilds.build(GhprbBuilds.java:74)         at org.jenkinsci.plugins.ghprb.GhprbPullRequest.build(GhprbPullRequest.java:481)         at org.jenkinsci.plugins.ghprb.GhprbPullRequest.tryBuild(GhprbPullRequest.java:474)         at org.jenkinsci.plugins.ghprb.GhprbPullRequest.check(GhprbPullRequest.java:166)         at org.jenkinsci.plugins.ghprb.GhprbRepository.onPullRequestHook(GhprbRepository.java:382)         at org.jenkinsci.plugins.ghprb.GhprbTrigger.handlePR(GhprbTrigger.java:676)         at org.jenkinsci.plugins.ghprb.GhprbRootAction$2.run(GhprbRootAction.java:252) Caused by: java.io.FileNotFoundException: https://api.github.com/repos/apache/incubator-mxnet/statuses/042bb58b9bb6893fae74002543142cd4bbfe6e73         at sun.reflect.GeneratedConstructorAccessor121.newInstance(Unknown Source)         at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)         at java.lang.reflect.Constructor.newInstance(Constructor.java:423)         at sun.net. www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1926)         at sun.net. www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1921)         at java.security.AccessController.doPrivileged(Native Method)         at sun.net. www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1920)         at sun.net. www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1490)         at sun.net. www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474)         at sun.net. www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254)         at org.kohsuke.github.Requester.parse(Requester.java:602)         at org.kohsuke.github.Requester.parse(Requester.java:584)         at org.kohsuke.github.Requester._to(Requester.java:264)         ... 10 more Caused by: java.io.FileNotFoundException: https://api.github.com/repos/apache/incubator-mxnet/statuses/042bb58b9bb6893fae74002543142cd4bbfe6e73         at sun.net. www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1872)         at sun.net. www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474)         at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)         at sun.net. www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:338)         at org.kohsuke.github.Requester.parse(Requester.java:592)         ... 12 more The create status needs write permissions on the mxnet repository in addition to the hooks. Could you please also add that?
      Hide
      larroy Pedro Larroy added a comment - Reporter
      Thank you so much, let me test.
      Show
      larroy Pedro Larroy added a comment - Reporter Thank you so much, let me test.
      Hide
      humbedooh Daniel Gruno added a comment -
      All done!
      Show
      humbedooh Daniel Gruno added a comment - All done!
      Hide
      larroy Pedro Larroy added a comment - Reporter
      This hooks would be needed

      http://ci.mxnet.amazon-ml.com/ghprbhook/ (issue_comment and pull_request)
      http://ci.mxnet.amazon-ml.com/github-webhook/ (push)
      Show
      larroy Pedro Larroy added a comment - Reporter This hooks would be needed http://ci.mxnet.amazon-ml.com/ghprbhook/ (issue_comment and pull_request) http://ci.mxnet.amazon-ml.com/github-webhook/ (push)
      Hide
      humbedooh Daniel Gruno added a comment -
      No, we do not offer commit rights for bots out of our control, and even in those cases, only to a select number of branches tightly controlled by us.
      Show
      humbedooh Daniel Gruno added a comment - No, we do not offer commit rights for bots out of our control, and even in those cases, only to a select number of branches tightly controlled by us.
      Hide
      larroy Pedro Larroy added a comment - Reporter
      Will that give also commit status and PR status permission?
      Show
      larroy Pedro Larroy added a comment - Reporter Will that give also commit status and PR status permission?
      Hide
      humbedooh Daniel Gruno added a comment -
      We can add custom web hooks, however we cannot (and this is not negotiable) give any people or bots admin rights to any repo, not even committers.
      Show
      humbedooh Daniel Gruno added a comment - We can add custom web hooks, however we cannot (and this is not negotiable) give any people or bots admin rights to any repo, not even committers.
      Hide
      larroy Pedro Larroy added a comment - - edited Reporter
      Hi Daniel, all.

      We are maintaining CI infrastructure for MXNet on Android and other embedded devices ( http://ci.mxnet.amazon-ml.com/ ). We would like to recieve PR and commit hooks. For that ideally we would like our bot account to have admin rights so it can manage the hooks and set the status on PRs and commits.

      The account is:

      https://github.com/MXNetEdgeBot

      The account is just for purely automation purposes and it won't be used to commit any code manually.

      Thanks a lot for your help.
      Show
      larroy Pedro Larroy added a comment - - edited Reporter Hi Daniel, all. We are maintaining CI infrastructure for MXNet on Android and other embedded devices ( http://ci.mxnet.amazon-ml.com/ ). We would like to recieve PR and commit hooks. For that ideally we would like our bot account to have admin rights so it can manage the hooks and set the status on PRs and commits. The account is: https://github.com/MXNetEdgeBot The account is just for purely automation purposes and it won't be used to commit any code manually. Thanks a lot for your help.
      Hide
      humbedooh Daniel Gruno added a comment -
      What exactly does this entail? Please elaborate
      Show
      humbedooh Daniel Gruno added a comment - What exactly does this entail? Please elaborate

        People

        • Assignee:
          humbedooh Daniel Gruno
          Reporter:
          larroy Pedro Larroy
          Request participants:
          None
        • Votes:
          0 Vote for this issue
          Watchers:
          4 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Time Tracking

            Estimated:
            Original Estimate - Not Specified
            Not Specified
            Remaining:
            Remaining Estimate - 0h
            0h
            Logged:
            Time Spent - 0.25h
            0.25h