Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-2254

Detect and warn about misusage of KuduContext in spark applications

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Invalid
    • 1.5.0
    • n/a
    • spark
    • None

    Description

      Since KuduContext carries a bunch of information for client connection, such as authentication token, KuduContext should be created in the driver and shared with executors.

      Even if we have put comments for KuduContext class (https://github.com/apache/kudu/blob/master/java/kudu-spark/src/main/scala/org/apache/kudu/spark/kudu/KuduContext.scala#L43), it would be great to provide a way to warn users (e.g Logging) about such behavior since this kind of issues are very subtle to detect.

      Attachments

        Activity

          People

            hahao Hao Hao
            hahao Hao Hao
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: