Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-48094

Reduce GitHub Action usage according to ASF project allowance

    XMLWordPrintableJSON

Details

    • Umbrella
    • Status: Reopened
    • Major
    • Resolution: Unresolved
    • 4.0.0
    • None
    • Project Infra
    • None

    Description

      ASF INFRA POLICY

      MONITORING

      TARGET

      • All workflows MUST have a job concurrency level less than or equal to 20. This means a workflow cannot have more than 20 jobs running at the same time across all matrices.
      • All workflows SHOULD have a job concurrency level less than or equal to 15. Just because 20 is the max, doesn't mean you should strive for 20.
      • The average number of minutes a project uses per calendar week MUST NOT exceed the equivalent of 25 full-time runners (250,000 minutes, or 4,200 hours).
      • The average number of minutes a project uses in any consecutive five-day period MUST NOT exceed the equivalent of 30 full-time runners (216,000 minutes, or 3,600 hours).

      DEADLINE

      17th of May, 2024

      Attachments

        Issue Links

          1.
          Run `build_non_ansi.yml` once per day Sub-task Resolved Dongjoon Hyun
          2.
          Run `build_maven_java21_macos14.yml` every two days Sub-task Resolved Dongjoon Hyun
          3.
          Limit GHA job execution time to up to 3 hours in `build_and_test.yml` Sub-task Resolved Dongjoon Hyun
          4.
          Enable `NOLINT_ON_COMPILE` for all except `lint` job Sub-task Resolved Dongjoon Hyun
          5.
          Run `maven-build` test only on `Java 21 on MacOS 14 (Apple Silicon)` Sub-task Resolved Dongjoon Hyun
          6.
          Run `publish_snapshot.yml` once per day Sub-task Resolved Dongjoon Hyun
          7.
          Use `Python 3.11` in `pyspark` tests of `build_and_test.yml` Sub-task Resolved Dongjoon Hyun
          8.
          Skip `tpcds-1g` and `docker-integration-tests` tests from `RocksDB UI-Backend` job Sub-task Resolved Dongjoon Hyun
          9.
          Enable `k8s-integration-tests` only for `kubernetes` module change Sub-task Resolved Dongjoon Hyun
          10.
          Remove all Maven compilation build Sub-task Resolved Hyukjin Kwon
          11.
          Move tpcds-1g and docker-integration-tests to daily scheduled jobs Sub-task Resolved Hyukjin Kwon
          12.
          Remove `Python 3.11` from `build_python.yml` Sub-task Resolved Dongjoon Hyun
          13.
          Run `pyspark-pandas*` only in PR builder and Daily Python CIs Sub-task Resolved Dongjoon Hyun
          14.
          Run `k8s-integration-tests` only in PR builder and Daily CIs Sub-task Resolved Dongjoon Hyun
          15.
          Run `sparkr` only in PR builders and Daily CIs Sub-task Resolved Dongjoon Hyun
          16.
          Run `buf` and `ui` only in PR builders and Java 21 Daily CI Sub-task Resolved Dongjoon Hyun
          17.
          Run `yarn` test only in PR builders and Maven Daily CIs Sub-task Resolved Dongjoon Hyun
          18.
          Serialize `build_python.yml` to run a single Python version per cron schedule Sub-task Closed Unassigned
          19.
          Run `build` job of `build_and_test.yml` only if needed Sub-task Resolved Dongjoon Hyun
          20.
          Merge `connect` back to the original test pipeline Sub-task Resolved Dongjoon Hyun
          21.
          Run `build/scala-213/java-11-17` jobs of branch-3.5 only if needed Sub-task Resolved Dongjoon Hyun
          22.
          Run `docs` only in PR builders and `build_non_ansi` Daily CI Sub-task Resolved Dongjoon Hyun
          23.
          Enable TPC-DS and docker tests in forked repository Sub-task Resolved Hyukjin Kwon
          24.
          Make `maven-deploy-plugin` retry 3 times Sub-task Resolved BingKun Pan
          25.
          Split `build_python.yml` into per-version cron jobs Sub-task Resolved Dongjoon Hyun
          26.
          Spin off `pyspark` tests from `build_branch35.yml` Daily CI Sub-task Resolved Dongjoon Hyun
          27.
          Spin off `pyspark` tests from `build_branch34.yml` Daily CI Sub-task Resolved Dongjoon Hyun
          28.
          Run `build/scala-213/java-11-17` jobs of `branch-3.4` only if needed Sub-task Resolved Dongjoon Hyun
          29.
          Update `publish_snapshot.yml` to publish twice per day Sub-task Resolved Dongjoon Hyun
          30.
          Exclude `branch-3.3` from `publish_snapshot.yml` Sub-task Resolved Dongjoon Hyun
          31.
          Skip unidoc step in docker-integration-tests GitHub Action job Sub-task Resolved Dongjoon Hyun
          32.
          Skip UNIDOC and MIMA in build GitHub Action job Sub-task Resolved Dongjoon Hyun
          33.
          Limit GHA job execution time to up to 5 hours in build_and_test.yml Sub-task Resolved Dongjoon Hyun
          34.
          Skip Unidoc in SparkR GitHub Action Job Sub-task Resolved Dongjoon Hyun

          Activity

            People

              Unassigned Unassigned
              dongjoon Dongjoon Hyun
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: