Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-35782

leveldbjni doesn't work in Apple Silicon on macOS

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.1.2
    • None
    • Spark Core
    • None

    Description

      leveldbjni doesn't contain the native library for Apple Silicon on macOS. We will need to build native library for Apple Silicon on macOS, and cut a new release so Spark can use it.

      However, it is not maintained for a long time, and the last release was in 2016. Per discussion in spark dev mailing list, other platform also runs into the same support issue. Perhaps, we should we consider racksdb as replacement.

      Note, here is the rocksdb task to support Apple Silicon, https://github.com/facebook/rocksdb/issues/7720

      Attachments

        Issue Links

          is duplicated by

          Sub-task - The sub-task of the issue SPARK-36019 Cannot run leveldb related UTs on Mac OS of M1 architecture

          • Major - Major loss of function.
          • Closed
          is related to

          New Feature - A new feature of the product, which has yet to be developed. SPARK-35781 Support Spark on Apple Silicon on macOS natively on Java 17

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              dbtsai DB Tsai
              Votes:
              1 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated: