Whirr
  1. Whirr
  2. WHIRR-101

Hadoop on EC2 does not use the /mnt partition

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.2.0
    • Component/s: service/hadoop
    • Labels:
      None

      Description

      It uses /data which is too small.

      1. WHIRR-101.patch
        1 kB
        Tom White
      2. WHIRR-101.patch
        1 kB
        Tom White
      3. WHIRR-101.patch
        2 kB
        Tom White

        Issue Links

          Activity

          Hide
          Tom White added a comment -

          I've just committed this.

          I opened WHIRR-111 for the larger issue.

          my 2p.

          Glad to see you're using the local currency!

          Show
          Tom White added a comment - I've just committed this. I opened WHIRR-111 for the larger issue. my 2p. Glad to see you're using the local currency!
          Hide
          Adrian Cole added a comment -

          I'm ok with this for a bug fix.

          Here's more for the more complete issue:

          it looks like alestic and rightscale images mount the first ephemeral disk onto /mnt. According to the amazon linux guide:
          "The ephemeral drive ephemeral0 is mounted in /media/ephmeral0 only on S3 backed AMIs."

          When we address this holistically, we may need to have a script that arranges the fstab and formats volumes accordingly, possibly using lvm or something to span disks together. Then, we can have a clear mount setup that whirr is authoritative for.

          my 2p.

          Show
          Adrian Cole added a comment - I'm ok with this for a bug fix. Here's more for the more complete issue: it looks like alestic and rightscale images mount the first ephemeral disk onto /mnt. According to the amazon linux guide: "The ephemeral drive ephemeral0 is mounted in /media/ephmeral0 only on S3 backed AMIs." When we address this holistically, we may need to have a script that arranges the fstab and formats volumes accordingly, possibly using lvm or something to span disks together. Then, we can have a clear mount setup that whirr is authoritative for. my 2p.
          Hide
          Tom White added a comment -

          Adrian - yes, this is definitely a better approach - thanks for pointing it out. I'm tempted to commit the current patch as an interim fix and open a new JIRA to take advantage of the Volume objects (this might tie in with adding EBS support too). Thoughts?

          Show
          Tom White added a comment - Adrian - yes, this is definitely a better approach - thanks for pointing it out. I'm tempted to commit the current patch as an interim fix and open a new JIRA to take advantage of the Volume objects (this might tie in with adding EBS support too). Thoughts?
          Hide
          Adrian Cole added a comment -

          there's another approach we could use.

          once the template is built, look at template.hardware.volumes. In ec2, this has the exact mount points for the ephemeral and ebs volumes attached

          Show
          Adrian Cole added a comment - there's another approach we could use. once the template is built, look at template.hardware.volumes. In ec2, this has the exact mount points for the ephemeral and ebs volumes attached
          Hide
          Tom White added a comment -

          Minor update to patch to pass cloud provider string to workers for installation script. Tested (with WHIRR-107 too) and manually checked that data was being stored on /mnt on EC2.

          Show
          Tom White added a comment - Minor update to patch to pass cloud provider string to workers for installation script. Tested (with WHIRR-107 too) and manually checked that data was being stored on /mnt on EC2.
          Hide
          Tom White added a comment -

          This should fix it, but I haven't been able to test it because I'm having problems connecting to EC2 instances once they start.

          Show
          Tom White added a comment - This should fix it, but I haven't been able to test it because I'm having problems connecting to EC2 instances once they start.

            People

            • Assignee:
              Tom White
              Reporter:
              Tom White
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development