Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-48094

Reduce GitHub Action usage according to ASF project allowance

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Umbrella
    • Status: Reopened
    • Major
    • Resolution: Unresolved
    • 4.0.0
    • None
    • Project Infra
    • None

    Description

      ASF INFRA POLICY

      MONITORING

      TARGET

      • All workflows MUST have a job concurrency level less than or equal to 20. This means a workflow cannot have more than 20 jobs running at the same time across all matrices.
      • All workflows SHOULD have a job concurrency level less than or equal to 15. Just because 20 is the max, doesn't mean you should strive for 20.
      • The average number of minutes a project uses per calendar week MUST NOT exceed the equivalent of 25 full-time runners (250,000 minutes, or 4,200 hours).
      • The average number of minutes a project uses in any consecutive five-day period MUST NOT exceed the equivalent of 30 full-time runners (216,000 minutes, or 3,600 hours).

      DEADLINE

      17th of May, 2024

      Attachments

        Issue Links

        1.
        Run `build_non_ansi.yml` once per day Sub-task Resolved Dongjoon Hyun Actions
        2.
        Run `build_maven_java21_macos14.yml` every two days Sub-task Resolved Dongjoon Hyun Actions
        3.
        Limit GHA job execution time to up to 3 hours in `build_and_test.yml` Sub-task Resolved Dongjoon Hyun Actions
        4.
        Enable `NOLINT_ON_COMPILE` for all except `lint` job Sub-task Resolved Dongjoon Hyun Actions
        5.
        Run `maven-build` test only on `Java 21 on MacOS 14 (Apple Silicon)` Sub-task Resolved Dongjoon Hyun Actions
        6.
        Run `publish_snapshot.yml` once per day Sub-task Resolved Dongjoon Hyun Actions
        7.
        Use `Python 3.11` in `pyspark` tests of `build_and_test.yml` Sub-task Resolved Dongjoon Hyun Actions
        8.
        Skip `tpcds-1g` and `docker-integration-tests` tests from `RocksDB UI-Backend` job Sub-task Resolved Dongjoon Hyun Actions
        9.
        Enable `k8s-integration-tests` only for `kubernetes` module change Sub-task Resolved Dongjoon Hyun Actions
        10.
        Remove all Maven compilation build Sub-task Resolved Hyukjin Kwon Actions
        11.
        Move tpcds-1g and docker-integration-tests to daily scheduled jobs Sub-task Resolved Hyukjin Kwon Actions
        12.
        Remove `Python 3.11` from `build_python.yml` Sub-task Resolved Dongjoon Hyun Actions
        13.
        Run `pyspark-pandas*` only in PR builder and Daily Python CIs Sub-task Resolved Dongjoon Hyun Actions
        14.
        Run `k8s-integration-tests` only in PR builder and Daily CIs Sub-task Resolved Dongjoon Hyun Actions
        15.
        Run `sparkr` only in PR builders and Daily CIs Sub-task Resolved Dongjoon Hyun Actions
        16.
        Run `buf` and `ui` only in PR builders and Java 21 Daily CI Sub-task Resolved Dongjoon Hyun Actions
        17.
        Run `yarn` test only in PR builders and Maven Daily CIs Sub-task Resolved Dongjoon Hyun Actions
        18.
        Serialize `build_python.yml` to run a single Python version per cron schedule Sub-task Closed Unassigned Actions
        19.
        Run `build` job of `build_and_test.yml` only if needed Sub-task Resolved Dongjoon Hyun Actions
        20.
        Merge `connect` back to the original test pipeline Sub-task Resolved Dongjoon Hyun Actions
        21.
        Run `build/scala-213/java-11-17` jobs of branch-3.5 only if needed Sub-task Resolved Dongjoon Hyun Actions
        22.
        Run `docs` only in PR builders and `build_non_ansi` Daily CI Sub-task Resolved Dongjoon Hyun Actions
        23.
        Enable TPC-DS and docker tests in forked repository Sub-task Resolved Hyukjin Kwon Actions
        24.
        Make `maven-deploy-plugin` retry 3 times Sub-task Resolved Pan Bingkun Actions
        25.
        Split `build_python.yml` into per-version cron jobs Sub-task Resolved Dongjoon Hyun Actions
        26.
        Spin off `pyspark` tests from `build_branch35.yml` Daily CI Sub-task Resolved Dongjoon Hyun Actions
        27.
        Spin off `pyspark` tests from `build_branch34.yml` Daily CI Sub-task Resolved Dongjoon Hyun Actions
        28.
        Run `build/scala-213/java-11-17` jobs of `branch-3.4` only if needed Sub-task Resolved Dongjoon Hyun Actions
        29.
        Update `publish_snapshot.yml` to publish twice per day Sub-task Resolved Dongjoon Hyun Actions
        30.
        Exclude `branch-3.3` from `publish_snapshot.yml` Sub-task Resolved Dongjoon Hyun Actions
        31.
        Skip unidoc step in docker-integration-tests GitHub Action job Sub-task Resolved Dongjoon Hyun Actions
        32.
        Skip UNIDOC and MIMA in build GitHub Action job Sub-task Resolved Dongjoon Hyun Actions
        33.
        Limit GHA job execution time to up to 5 hours in build_and_test.yml Sub-task Resolved Dongjoon Hyun Actions
        34.
        Skip Unidoc in SparkR GitHub Action Job Sub-task Resolved Dongjoon Hyun Actions

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            dongjoon Dongjoon Hyun

            Dates

              Created:
              Updated:

              Slack

                Issue deployment