Uploaded image for project: 'Infrastructure'
  1. Infrastructure
  2. INFRA-14267

[superset] move github repo from airbnb to apache

    Details

    • Type: Task
    • Status: Waiting for user
    • Priority: Minor
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: Github
    • Labels:
      None
    • Project:
      Incubator
    1. screenshot-1.png
      178 kB
      Maxime Beauchemin

      Activity

      Hide
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter
      Move from:
      https://github.com/airbnb/superset
      to:
      https://github.com/apache/incubator-superset

      Happy to synchronize to release the repo so that an admin in the Github apache org can land it there.
      Show
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter Move from: https://github.com/airbnb/superset to: https://github.com/apache/incubator-superset Happy to synchronize to release the repo so that an admin in the Github apache org can land it there.
      Show
      daijy Daniel Dai added a comment - Note https://github.com/apache/incubator-superset is just a readonly copy of https://git-wip-us.apache.org/repos/asf/incubator-superset.git .
      Hide
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter
      I want Superset to be setup more or less like https://github.com/apache/incubator-airflow , I need to coordinate to release the repo to someone on the apache org side who can land it along will all of its PRs, issues, starts and forks
      Show
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter I want Superset to be setup more or less like https://github.com/apache/incubator-airflow , I need to coordinate to release the repo to someone on the apache org side who can land it along will all of its PRs, issues, starts and forks
      Hide
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - - edited Reporter
      I failed at trying to open a Jira ticket using the `Infrastructure` "project", I'm hoping that assigning the asfinfra-bot does the trick (but that's just a wild guess)
      Show
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - - edited Reporter I failed at trying to open a Jira ticket using the `Infrastructure` "project", I'm hoping that assigning the asfinfra-bot does the trick (but that's just a wild guess)
      Hide
      Internal
      daijy Daniel Dai added a comment -
      I think airflow is the same, https://git-wip-us.apache.org/repos/asf/incubator-airflow.git is the main repo and https://github.com/apache/incubator-airflow is the read only mirror. Developers can use anything (PRs, issues, folks) against the github copy.
      Show
      Internal
      daijy Daniel Dai added a comment - I think airflow is the same, https://git-wip-us.apache.org/repos/asf/incubator-airflow.git is the main repo and https://github.com/apache/incubator-airflow is the read only mirror. Developers can use anything (PRs, issues, folks) against the github copy.
      Hide
      coreyg Geoffrey Corey added a comment -
      Moved it to the INFRA queue.
      Show
      coreyg Geoffrey Corey added a comment - Moved it to the INFRA queue.
      Hide
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter
      Turns out the Software Grant Agreement (SGA) is a pre-req for this. I'll update the ticket when we've ship that over.
      https://www.apache.org/licenses/software-grant-template.pdf
      Show
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter Turns out the Software Grant Agreement (SGA) is a pre-req for this. I'll update the ticket when we've ship that over. https://www.apache.org/licenses/software-grant-template.pdf
      Hide
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter
      The SAG has been provided to the treasurer, let's move forward with moving the repo!
      Show
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter The SAG has been provided to the treasurer, let's move forward with moving the repo!
      Hide
      pono Daniel Takamori added a comment -
      [~maxime.beauchemin@apache.org] Once you add me, 'pono' on Github, to the AirBnB admin list, I can go ahead and transfer into our Organization.
      Show
      pono Daniel Takamori added a comment - [~ maxime.beauchemin@apache.org ] Once you add me, 'pono' on Github, to the AirBnB admin list, I can go ahead and transfer into our Organization.
      Hide
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter
      Our Admins say that they cannot grant admin priviledges on our org to non-employees. I'm pretty sure that it is the same on your side.

      So can we do it through a "proxy"? Maybe your personal "org" on Github? Our admin's id is `ljharb` (Jordan Harband). Can you grant him temporary admin on your `pono` org and we'll ship the repo over? Then you can integrate it in apache's org?
      Show
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter Our Admins say that they cannot grant admin priviledges on our org to non-employees. I'm pretty sure that it is the same on your side. So can we do it through a "proxy"? Maybe your personal "org" on Github? Our admin's id is `ljharb` (Jordan Harband). Can you grant him temporary admin on your `pono` org and we'll ship the repo over? Then you can integrate it in apache's org?
      Hide
      pono Daniel Takamori added a comment -
      I invited ljharb to https://github.com/orgs/ApacheInfra/ so they can transfer the repository there and then I can transfer to Apache.
      Show
      pono Daniel Takamori added a comment - I invited ljharb to https://github.com/orgs/ApacheInfra/ so they can transfer the repository there and then I can transfer to Apache.
      Hide
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - - edited Reporter
      Ok done. The move can be finalized now. In the meantime all of the Github integrations (Travis-CI, codeclimate, ...) are broken :(
      Show
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - - edited Reporter Ok done. The move can be finalized now. In the meantime all of the Github integrations (Travis-CI, codeclimate, ...) are broken :(
      Hide
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter
      Also please provide me (Github user `mistercrunch`) with merge rights on the repo ASAP as not having the rights prevents me from releasing new versions internally at Airbnb at the moment.

      Thanks!
      Show
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter Also please provide me (Github user `mistercrunch`) with merge rights on the repo ASAP as not having the rights prevents me from releasing new versions internally at Airbnb at the moment. Thanks!
      Hide
      pono Daniel Takamori added a comment -
      [~maxime.beauchemin@apache.org] Added write access to you so you can make your release. Let me know when you are done and I can transfer the repo to our org.
      Show
      pono Daniel Takamori added a comment - [~ maxime.beauchemin@apache.org ] Added write access to you so you can make your release. Let me know when you are done and I can transfer the repo to our org.
      Hide
      johndament John D. Ament added a comment -
      Infra team,

      Before moving forward on this, please ensure that the podling exists in both the status page ( http://incubator.apache.org/guides/mentor.html#Initialize+Podling+Status+Page ) and podlings.xml.

      There is no status page for Superset, so until that happens we shouldn't try to do any repo moves.
      Show
      johndament John D. Ament added a comment - Infra team, Before moving forward on this, please ensure that the podling exists in both the status page ( http://incubator.apache.org/guides/mentor.html#Initialize+Podling+Status+Page ) and podlings.xml. There is no status page for Superset, so until that happens we shouldn't try to do any repo moves.
      Hide
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter
      Hey thanks for the merge rights but that's not working since the CI isn't setup and I can't override/merge.... Please give me more priviledges!
      Show
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter Hey thanks for the merge rights but that's not working since the CI isn't setup and I can't override/merge.... Please give me more priviledges!
      Show
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter ! https://issues.apache.org/jira/secure/attachment/12874522/screenshot-1.png!
      Hide
      pono Daniel Takamori added a comment -
      [~maxime.beauchemin@apache.org] Gave you admin temporarily.

      This really should've been done before handing over the repo, please be mindful of this in the future.
      Show
      pono Daniel Takamori added a comment - [~ maxime.beauchemin@apache.org ] Gave you admin temporarily. This really should've been done before handing over the repo, please be mindful of this in the future.
      Hide
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter
      Show
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter Here it is: http://incubator.apache.org/projects/superset.html
      Hide
      pono Daniel Takamori added a comment -
      [~maxime.beauchemin@apache.org] The repo is ready to be moved then?
      Show
      pono Daniel Takamori added a comment - [~ maxime.beauchemin@apache.org ] The repo is ready to be moved then?
      Hide
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter
      Yes please, go ahead with the move. Also please either set up on Travis-CI, Codeclimate and Coveralls or provide us the rights to do so.
      Show
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter Yes please, go ahead with the move. Also please either set up on Travis-CI, Codeclimate and Coveralls or provide us the rights to do so.
      Hide
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter
      ping.

      A lot of our tooling (Travis, Codeclimate, coveralls) has been disfunctional for a week now. Please help!
      Show
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter ping. A lot of our tooling (Travis, Codeclimate, coveralls) has been disfunctional for a week now. Please help!
      Hide
      pono Daniel Takamori added a comment - - edited
      Moved into the Apache Organization. Travis is catching up on the repo sync so once that's done I'll be able to enable it for the repo.

      https://github.com/apache/incubator-superset/
      Show
      pono Daniel Takamori added a comment - - edited Moved into the Apache Organization. Travis is catching up on the repo sync so once that's done I'll be able to enable it for the repo. https://github.com/apache/incubator-superset/
      Hide
      pono Daniel Takamori added a comment - - edited
      Looks like Travis was quicker than I thought https://travis-ci.org/apache/incubator-superset.svg?branch=ascott-0.17.1rc1

      and here's coveralls: https://coveralls.io/repos/github/apache/incubator-superset/badge.svg

      Unfortunately we can't support CodeClimate at this time as it requires write access to the whole organization.
      Show
      pono Daniel Takamori added a comment - - edited Looks like Travis was quicker than I thought https://travis-ci.org/apache/incubator-superset.svg?branch=ascott-0.17.1rc1 and here's coveralls: https://coveralls.io/repos/github/apache/incubator-superset/badge.svg Unfortunately we can't support CodeClimate at this time as it requires write access to the whole organization.
      Hide
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter
      It appears I've lost my rights to `write` or administer the repo in any way
      Show
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter It appears I've lost my rights to `write` or administer the repo in any way
      Hide
      pono Daniel Takamori added a comment -
      Imported repo and added to Gitbox, closing as this task is done.
      Show
      pono Daniel Takamori added a comment - Imported repo and added to Gitbox, closing as this task is done.
      Hide
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter
      Wait up! As I commented earlier all committers lost their rights in the transfer process. I have no way of merging PRs at the moment... I can't even close issues or label things.
      Show
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter Wait up! As I commented earlier all committers lost their rights in the transfer process. I have no way of merging PRs at the moment... I can't even close issues or label things.
      Hide
      ipv6guru Gavin added a comment -
      Did you recheck those permissions [~maxime.beauchemin@apache.org] ?
      Show
      ipv6guru Gavin added a comment - Did you recheck those permissions [~ maxime.beauchemin@apache.org ] ?
      Hide
      ipv6guru Gavin added a comment -
      You might need to :-

      1. set up 2FA in Github
      2. go through the steps indicated at gitbox.apache.org
      3. wait 1/2 hr and try again
      Show
      ipv6guru Gavin added a comment - You might need to :- 1. set up 2FA in Github 2. go through the steps indicated at gitbox.apache.org 3. wait 1/2 hr and try again
      Hide
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter
      I got 2FA on on Github, followed the steps here: https://gitbox.apache.org/setup/, waited a few hours, and I still don't have write rights on my repo...

      At the bottom of that page, it says:
      ```
      According to LDAP, you will have access to the following repositories:

      airflow:
      No repositories for the airflow project served from gitbox yet...
      incubator:
      No repositories for the incubator project served from gitbox yet...
      ```

      I'm confused, am I supposed to "Create a new GitBox/GitHub repository". I'm guessing not since the repo already exists...

      Also note that over past 10+ days since we started the process it was never clear that we needed to go through these steps. The repository migration process has been pretty disruptive for our community as we've lost control over our repo.
      Show
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter I got 2FA on on Github, followed the steps here: https://gitbox.apache.org/setup/, waited a few hours, and I still don't have write rights on my repo... At the bottom of that page, it says: ``` According to LDAP, you will have access to the following repositories: airflow: No repositories for the airflow project served from gitbox yet... incubator: No repositories for the incubator project served from gitbox yet... ``` I'm confused, am I supposed to "Create a new GitBox/GitHub repository". I'm guessing not since the repo already exists... Also note that over past 10+ days since we started the process it was never clear that we needed to go through these steps. The repository migration process has been pretty disruptive for our community as we've lost control over our repo.
      Hide
      ipv6guru Gavin added a comment -
      I agree that the Incubator and your Mentors should have made you aware that by default, we do not have Github as the writable repository but git-wip-us.apache.org is. Experimental is Gitbox and you should have been made aware you would need to apply for this to continue r/w access to Github. I apologise for the inconvenience; and I encourage you to take your grievances back to your Mentors and the Incubator mailing lists, perhaps documentation and Mentoring can be improved so that this miscommunication doesnt happen to another purely Github based project coming in for the future.

      As for why you still don't have full access, I'm going to ping [~humbedooh] and [~pono] and get them to look into it further.
      Show
      ipv6guru Gavin added a comment - I agree that the Incubator and your Mentors should have made you aware that by default, we do not have Github as the writable repository but git-wip-us.apache.org is. Experimental is Gitbox and you should have been made aware you would need to apply for this to continue r/w access to Github. I apologise for the inconvenience; and I encourage you to take your grievances back to your Mentors and the Incubator mailing lists, perhaps documentation and Mentoring can be improved so that this miscommunication doesnt happen to another purely Github based project coming in for the future. As for why you still don't have full access, I'm going to ping [~humbedooh] and [~pono] and get them to look into it further.
      Hide
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter
      No worries, grateful for being able to continue using Github and I was aware of the experimental state of Github support for ASF projects. I'm also willing to help make things better.

      Is the ASF tooling open source? Is it possible to contribute?
      Show
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter No worries, grateful for being able to continue using Github and I was aware of the experimental state of Github support for ASF projects. I'm also willing to help make things better. Is the ASF tooling open source? Is it possible to contribute?
      Hide
      pono Daniel Takamori added a comment -
      It sure is! https://github.com/apache/infrastructure-puppet/tree/deployment/modules/gitbox

      Where you able to get your 2 factor setup and working?
      Show
      pono Daniel Takamori added a comment - It sure is! https://github.com/apache/infrastructure-puppet/tree/deployment/modules/gitbox Where you able to get your 2 factor setup and working?
      Hide
      humbedooh Daniel Gruno added a comment -
      https://whimsy.apache.org/roster/ppmc/superset - your mentors need to add people to the podling (using this link) before you can get access to the repos.
      Show
      humbedooh Daniel Gruno added a comment - https://whimsy.apache.org/roster/ppmc/superset - your mentors need to add people to the podling (using this link) before you can get access to the repos.
      Hide
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter
      I still cannot merge PRs because of the lingering hooks (integrations) on coveralls and codeclimate that won't trigger since the move. That leaves the merge button grayed out forever.

      Can we remove those hooks/integration that aren't working anyways?
      Show
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter I still cannot merge PRs because of the lingering hooks (integrations) on coveralls and codeclimate that won't trigger since the move. That leaves the merge button grayed out forever. Can we remove those hooks/integration that aren't working anyways?
      Hide
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter
      Show
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter example here: https://github.com/apache/incubator-superset/pull/3120
      Hide
      johndament John D. Ament added a comment -
      I'll try to explain the issue a bit more to see if it helps. To test, I gave myself write access to the superset repo (I'll remove myself shortly).

      When you go to a PR, only 1 of the 4 checks they have configured is working. The other three - codeclimate and coveralls - never executed. I believe we said codeclimate was a no, so we should remove that integration from the repo. Coveralls is something we support, so perhaps we just need to fix that integration.
      Show
      johndament John D. Ament added a comment - I'll try to explain the issue a bit more to see if it helps. To test, I gave myself write access to the superset repo (I'll remove myself shortly). When you go to a PR, only 1 of the 4 checks they have configured is working. The other three - codeclimate and coveralls - never executed. I believe we said codeclimate was a no, so we should remove that integration from the repo. Coveralls is something we support, so perhaps we just need to fix that integration.
      Hide
      pono Daniel Takamori added a comment -
      Disabled codeclimate and re-enabled coveralls: https://coveralls.io/github/apache/incubator-superset

      Once a job runs I think github will re-add the status check, but for now only Travis passing is required for merging.
      Show
      pono Daniel Takamori added a comment - Disabled codeclimate and re-enabled coveralls: https://coveralls.io/github/apache/incubator-superset Once a job runs I think github will re-add the status check, but for now only Travis passing is required for merging.
      Hide
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter
      One [hopefully] last thing, can we uncheck "Require branches to be up to date before merging"
      !http://i.imgur.com/7vKkKYg.png!
      Show
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter One [hopefully] last thing, can we uncheck "Require branches to be up to date before merging" ! http://i.imgur.com/7vKkKYg.png!
      Hide
      pono Daniel Takamori added a comment -
      Unchecked "Require branches to be up to date before merging" and added the coveralls check. Anything else to do here?
      Show
      pono Daniel Takamori added a comment - Unchecked "Require branches to be up to date before merging" and added the coveralls check. Anything else to do here?
      Hide
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter
      I need the new Coveralls token for the repo, I can't access it since I'm not an admin. Can you paste the token here?

      Also, can we setup these services?
      * https://david-dm.org/
      * https://requires.io/
      Show
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter I need the new Coveralls token for the repo, I can't access it since I'm not an admin. Can you paste the token here? Also, can we setup these services? * https://david-dm.org/ * https://requires.io/
      Hide
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter
      [ping]
      We cannot merge anything to master at this time because we have no way to ping coveralls because we have the wrong token. That results in failed checks, then merging button is blocked.

      Can we switch off coveralls until we get that token? Any way I can get the right to override that?

      Or can I get admin rights on the repo temporarily so that I can set up all the tooling/3rd party integrations?
      Show
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter [ping] We cannot merge anything to master at this time because we have no way to ping coveralls because we have the wrong token. That results in failed checks, then merging button is blocked. Can we switch off coveralls until we get that token? Any way I can get the right to override that? Or can I get admin rights on the repo temporarily so that I can set up all the tooling/3rd party integrations?
      Hide
      pono Daniel Takamori added a comment -
      I've emailed you the token after disabled and reenabling coveralls. Github seems to have lost the token. Can you point to a PR that was blocked on it?

      The permission scheme necessary for requires.io is too pervasive and requires Organization wide write access to repos, which is a non-starter at this points for repositories. I'll need to review the permission scheme required for david-dm.org to better understand what it needs.
      Show
      pono Daniel Takamori added a comment - I've emailed you the token after disabled and reenabling coveralls. Github seems to have lost the token. Can you point to a PR that was blocked on it? The permission scheme necessary for requires.io is too pervasive and requires Organization wide write access to repos, which is a non-starter at this points for repositories. I'll need to review the permission scheme required for david-dm.org to better understand what it needs.
      Hide
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter
      Here's a PR that was blocked where I updated the token:
      https://github.com/apache/incubator-superset/pull/3155

      I'm hoping travis will do the right thing and call coveralls and that the check will succeed. Now that I think about it, can we simply make Coveralls checks "non blocking" (only informational) for merges?

      The dependency tools are useful but aren't as critical and we can use them punctually on a fork.

      It would be nice to make a wiki page at the ASF with a list of whitelisted/blacklisted Github services for people to chose from.
      Show
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter Here's a PR that was blocked where I updated the token: https://github.com/apache/incubator-superset/pull/3155 I'm hoping travis will do the right thing and call coveralls and that the check will succeed. Now that I think about it, can we simply make Coveralls checks "non blocking" (only informational) for merges? The dependency tools are useful but aren't as critical and we can use them punctually on a fork. It would be nice to make a wiki page at the ASF with a list of whitelisted/blacklisted Github services for people to chose from.
      Hide
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter
      I want to have the possibility to merge the PR regardless of the Coveralls status check. Sometimes it gives false negative, and some time there's new code that doesn't require to be covered by unit test.

      Can you change the setting to not require Coveralls passing in order to be able to merge the PR?
      Show
      maxime.beauchemin@apache.org Maxime Beauchemin added a comment - Reporter I want to have the possibility to merge the PR regardless of the Coveralls status check. Sometimes it gives false negative, and some time there's new code that doesn't require to be covered by unit test. Can you change the setting to not require Coveralls passing in order to be able to merge the PR?
      Hide
      pono Daniel Takamori added a comment - - edited
      Disabled requiring coveralls for merge.

      We're working on making it much more explicit which Github integrations we allow, sorry about the inconvenience.
      Show
      pono Daniel Takamori added a comment - - edited Disabled requiring coveralls for merge. We're working on making it much more explicit which Github integrations we allow, sorry about the inconvenience.

        People

        • Assignee:
          pono Daniel Takamori
          Reporter:
          maxime.beauchemin@apache.org Maxime Beauchemin
          Request participants:
          None
        • Votes:
          0 Vote for this issue
          Watchers:
          5 Start watching this issue

          Dates

          • Created:
            Updated:

            Time Tracking

            Estimated:
            Original Estimate - Not Specified
            Not Specified
            Remaining:
            Remaining Estimate - 0h
            0h
            Logged:
            Time Spent - 5h
            5h