Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-8863

'spark_ec2.py' doesn't check '~/.aws/credentials' even if boto can support '~/.aws/credentials'

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 1.4.0
    • 1.5.0
    • EC2
    • None

    Description

      'spark_ec2.py' use boto to control ec2.
      And boto can support '~/.aws/credentials' which is AWS CLI default configuration file.

      We can check this information from ref of boto.

      "A boto config file is a text file formatted like an .ini configuration file that specifies values for options that control the behavior of the boto library.
      In Unix/Linux systems, on startup, the boto library looks for configuration files in the following locations and in the following order:
      /etc/boto.cfg - for site-wide settings that all users on this machine will use
      (if profile is given) ~/.aws/credentials - for credentials shared between SDKs
      (if profile is given) ~/.boto - for user-specific settings
      ~/.aws/credentials - for credentials shared between SDKs
      ~/.boto - for user-specific settings"

      However 'spark_ec2.py' only check boto config & environment variable even if there is '~/.aws/credentials', and 'spark_ec2.py' is terminated.

      So I changed to check '~/.aws/credentials'.

      Attachments

        Activity

          People

            JPark Juhong Park
            JPark Juhong Park
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: