Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.4.0
    • Fix Version/s: 0.6.0
    • Component/s: None
    • Labels:
      None

      Activity

      Hide
      Peter Linnell added a comment -

      +1 on the latest patch

      Show
      Peter Linnell added a comment - +1 on the latest patch
      Hide
      Roman Shaposhnik added a comment -

      +1

      Show
      Roman Shaposhnik added a comment - +1
      Hide
      Bruno Mahé added a comment -

      Here is an updated patch

      Show
      Bruno Mahé added a comment - Here is an updated patch
      Hide
      Roman Shaposhnik added a comment -

      Bruno, it would be nice to get an update on BIGTOP-863 so that we could also move forward on this one.

      Show
      Roman Shaposhnik added a comment - Bruno, it would be nice to get an update on BIGTOP-863 so that we could also move forward on this one.
      Hide
      Bruno Mahé added a comment -

      No, as I stated in several of my previous comments and BIGTOP-863:

      Somehow I found some yarn/mapreduce initscripts with typo into /etc/init.d and not belonging to any package (ex: /etc/init.d/hadoop-mapreduce-historyservere)

      Show
      Bruno Mahé added a comment - No, as I stated in several of my previous comments and BIGTOP-863 : Somehow I found some yarn/mapreduce initscripts with typo into /etc/init.d and not belonging to any package (ex: /etc/init.d/hadoop-mapreduce-historyservere)
      Hide
      Konstantin Boudnik added a comment -

      I think this is an attempt to prevent YARN from starting before hdfs-init happens. I don't think this is a good way of doing so, too.

      Show
      Konstantin Boudnik added a comment - I think this is an attempt to prevent YARN from starting before hdfs-init happens. I don't think this is a good way of doing so, too.
      Hide
      Roman Shaposhnik added a comment -

      I may be reviewing too many patches today, but why this?

      +    - "/bin/rm -f /etc/init.d/hadoop-mapreduce-historyservere"
      +    - "/bin/rm -f /etc/init.d/hadoop-yarn-nodemanagere"
      +    - "/bin/rm -f /etc/init.d/hadoop-yarn-resourcemanagere"
      
      Show
      Roman Shaposhnik added a comment - I may be reviewing too many patches today, but why this? + - "/bin/rm -f /etc/init.d/hadoop-mapreduce-historyservere" + - "/bin/rm -f /etc/init.d/hadoop-yarn-nodemanagere" + - "/bin/rm -f /etc/init.d/hadoop-yarn-resourcemanagere"
      Hide
      Bruno Mahé added a comment -

      Here is a new patch named 0003-BIGTOP-637.-Update-boxgrinder-appliance-for-the-comi.patch

      Show
      Bruno Mahé added a comment - Here is a new patch named 0003- BIGTOP-637 .-Update-boxgrinder-appliance-for-the-comi.patch
      Hide
      Bruno Mahé added a comment -

      I just created or converted existing tickets to sub tasks in order to track the remaining work to do.

      Show
      Bruno Mahé added a comment - I just created or converted existing tickets to sub tasks in order to track the remaining work to do.
      Hide
      Bruno Mahé added a comment -

      So here is the list of things to do in order to complete this ticket:

      • BIGTOP-861 needs to be committed
      • BIGTOP-852 - I just need yarn to be set up correctly. If no volunteer I will tackle this
      • I need to update the appliance for the location of the init-hdfs.sh script. It will not take long, but it is getting late here
      • Optionally, I would like to open a ticket and attach a patch to fix Apache Hadoop's daemons priorities as done by my boxgrinder appliance. All the services ought to start nicely and in order in any case.
      • I also need to verify the existence of the initscripts with typo
      Show
      Bruno Mahé added a comment - So here is the list of things to do in order to complete this ticket: BIGTOP-861 needs to be committed BIGTOP-852 - I just need yarn to be set up correctly. If no volunteer I will tackle this I need to update the appliance for the location of the init-hdfs.sh script. It will not take long, but it is getting late here Optionally, I would like to open a ticket and attach a patch to fix Apache Hadoop's daemons priorities as done by my boxgrinder appliance. All the services ought to start nicely and in order in any case. I also need to verify the existence of the initscripts with typo
      Hide
      Bruno Mahé added a comment -

      See also BIGTOP-861

      Show
      Bruno Mahé added a comment - See also BIGTOP-861
      Hide
      Bruno Mahé added a comment -

      I was too busy to touch it. But am back at it now.

      Show
      Bruno Mahé added a comment - I was too busy to touch it. But am back at it now.
      Hide
      Roman Shaposhnik added a comment -

      Bruno, thanks for taking care of this. Any updates so far on when do you think we can have at least the Boxgrinder part in a reasonable enough shape?

      Show
      Roman Shaposhnik added a comment - Bruno, thanks for taking care of this. Any updates so far on when do you think we can have at least the Boxgrinder part in a reasonable enough shape?
      Hide
      Mark Grover added a comment -

      All, thanks for your input and suggestions regarding the helper script. I created a new JIRA for that: BIGTOP-852. Feel free to comment/add if there is something I missed.

      Show
      Mark Grover added a comment - All, thanks for your input and suggestions regarding the helper script. I created a new JIRA for that: BIGTOP-852 . Feel free to comment/add if there is something I missed.
      Hide
      Bruno Mahé added a comment -

      Avoiding dups is great. But avoiding dups in any mkdir and chmod command in the project sounds plain insane if you ask me.

      We are not trying to avoid duplicating mkdir/chmod commands, we are trying to avoid duplicating the knowledge of which directories should exist where and with which rights and whose (whom?) ownership.
      Next time one of the usual suspect will add a new directory somewhere with some fancy rights, we won't have to maintain it in several places and the VM will not be the first thing to break.

      What BIGTOP-547 try to solve a much broader problem that structures HDFS for almost anything in the verse, except for that particular case that YARN depends on.

      No, BIGTOP-547 include that particular case that YARN depends on. It's just that there is a bug that needs to be fixed.

      But overall, what you qualify of plain insane or bulky and other names, is what I call awesome. And I call it awesome because we just killed a flock of birds with one stone.
      None of the issues we found is impossible to solve or a show stopper. Furthermore by dog fooding our own stuff, we discovered issues that will be fixed before users even see it.
      It's not because BIGTOP-547 is buggy and need some more work that we should throw the baby out with the bath water.

      So I am re-assigning this ticket to myself to wrap up this issue. I will also open tickets accordingly to the issues we found.

      Show
      Bruno Mahé added a comment - Avoiding dups is great. But avoiding dups in any mkdir and chmod command in the project sounds plain insane if you ask me. We are not trying to avoid duplicating mkdir/chmod commands, we are trying to avoid duplicating the knowledge of which directories should exist where and with which rights and whose (whom?) ownership. Next time one of the usual suspect will add a new directory somewhere with some fancy rights, we won't have to maintain it in several places and the VM will not be the first thing to break. What BIGTOP-547 try to solve a much broader problem that structures HDFS for almost anything in the verse, except for that particular case that YARN depends on. No, BIGTOP-547 include that particular case that YARN depends on. It's just that there is a bug that needs to be fixed. But overall, what you qualify of plain insane or bulky and other names, is what I call awesome. And I call it awesome because we just killed a flock of birds with one stone. None of the issues we found is impossible to solve or a show stopper. Furthermore by dog fooding our own stuff, we discovered issues that will be fixed before users even see it. It's not because BIGTOP-547 is buggy and need some more work that we should throw the baby out with the bath water. So I am re-assigning this ticket to myself to wrap up this issue. I will also open tickets accordingly to the issues we found.
      Hide
      Konstantin Boudnik added a comment -

      I have unassigned myself from the ticket - proceed as you wish guys.

      Show
      Konstantin Boudnik added a comment - I have unassigned myself from the ticket - proceed as you wish guys.
      Hide
      Konstantin Boudnik added a comment -

      Roman,

      it isn't about forks of "the code": the original patch was solving an exact issue that any user of the VM would face immediately.

      What BIGTOP-547 try to solve a much broader problem that structures HDFS for almost anything in the verse, except for that particular case that YARN depends on. The performance that script is another issue: I don't think it is acceptable UX to stretch VM boot for much longer than the Linux boot sequence takes. That said - I don't see how that ticket is related here. After all: the VM doesn't come with anything but plain Hadoop, so why bother? "Let's have it just in case" mentality, perhaps?

      Avoiding dups is great. But avoiding dups in any mkdir and chmod command in the project sounds plain insane if you ask me.

      Show
      Konstantin Boudnik added a comment - Roman, it isn't about forks of "the code": the original patch was solving an exact issue that any user of the VM would face immediately. What BIGTOP-547 try to solve a much broader problem that structures HDFS for almost anything in the verse, except for that particular case that YARN depends on. The performance that script is another issue: I don't think it is acceptable UX to stretch VM boot for much longer than the Linux boot sequence takes. That said - I don't see how that ticket is related here. After all: the VM doesn't come with anything but plain Hadoop, so why bother? "Let's have it just in case" mentality, perhaps? Avoiding dups is great. But avoiding dups in any mkdir and chmod command in the project sounds plain insane if you ask me.
      Hide
      Roman Shaposhnik added a comment -

      Any reason not to help make BIGTOP-547 better instead of having small forks of code in a few places?

      Show
      Roman Shaposhnik added a comment - Any reason not to help make BIGTOP-547 better instead of having small forks of code in a few places?
      Hide
      Konstantin Boudnik added a comment -

      Seriously? Let's use the solution that is slow and still doesn't solve the issue of prepping HDFS for YARN jobs. Am I hearing you right? Sorry, the logic of this evades me.

      Show
      Konstantin Boudnik added a comment - Seriously? Let's use the solution that is slow and still doesn't solve the issue of prepping HDFS for YARN jobs. Am I hearing you right? Sorry, the logic of this evades me.
      Hide
      Roman Shaposhnik added a comment -

      Konstantin Boudnik BIGTOP-547 is indeed pretty slow (I'll be looking into various ways of speeding it up this week). That said, I think we may be better off using it rather than embedding bits and piece of similar functionality as-is. IOW, if push comes to shove, I'd rather introduce a command line option for the implementation in BIGTOP-547 that would make sense for the VM-type of usage.

      Show
      Roman Shaposhnik added a comment - Konstantin Boudnik BIGTOP-547 is indeed pretty slow (I'll be looking into various ways of speeding it up this week). That said, I think we may be better off using it rather than embedding bits and piece of similar functionality as-is. IOW, if push comes to shove, I'd rather introduce a command line option for the implementation in BIGTOP-547 that would make sense for the VM-type of usage.
      Hide
      Konstantin Boudnik added a comment - - edited

      Also, BIGTOP-547 is pretty slow. Not sure we can do anything about it, but it would be great if we could find a way to speed it up.

      the reason of the slowness is a "brutal force" approach of directory structure creations coupled with needless checks. The script is clearly too bulky for the VM. What exactly was wrong with the script in the original patch?

      Show
      Konstantin Boudnik added a comment - - edited Also, BIGTOP-547 is pretty slow. Not sure we can do anything about it, but it would be great if we could find a way to speed it up. the reason of the slowness is a "brutal force" approach of directory structure creations coupled with needless checks. The script is clearly too bulky for the VM. What exactly was wrong with the script in the original patch?
      Hide
      Konstantin Boudnik added a comment - - edited

      Another issue of BIGTOP-547 is that even after running it, I could not run a simple pi job as root. It would complain /tmp/hadoop-yarn (something like that) is not accessible by root. Although jobs

      Bruno, the script I have introduced initially in this ticket allows to run jobs as root. BIGTOP-547 is incomplete.

      I don't see why you would include the whole bunch of the instructions from BIGTOP-547 - that is likely to make the startup of the VM painfully slow too - if the VM installs nothing but hadoop-conf-pseudo?

      Show
      Konstantin Boudnik added a comment - - edited Another issue of BIGTOP-547 is that even after running it, I could not run a simple pi job as root. It would complain /tmp/hadoop-yarn (something like that) is not accessible by root. Although jobs Bruno, the script I have introduced initially in this ticket allows to run jobs as root. BIGTOP-547 is incomplete. I don't see why you would include the whole bunch of the instructions from BIGTOP-547 - that is likely to make the startup of the VM painfully slow too - if the VM installs nothing but hadoop-conf-pseudo?
      Hide
      Bruno Mahé added a comment - - edited

      So here is the first pass of my take on this ticket (file 0001-BIGTOP-637.-Update-boxgrinder-appliance-for-the-comi.patch from 18/Feb/13 22:09)
      I went the way I outlined a few comments ago, meaning I reused the script Mark contributed in BIGTOP-547 and wrapped it into an init script which is ran between HDFS and YARN daemons at start up.
      And this works great.

      But I also note the following:

      • BIGTOP-547 was not checked in. So in order to get stuff done, this is included for now by boxgrinder. But I would like to sort out its situation so we can check this in and reuse it directly from the packages
      • BIGTOP-547 has a few issues. The first one being its use of sudo, which requires a login shell. Unfortunately login shells are not available from init scripts. So I had to replace sudo by su. This should be included in BIGTOP-547
      • Another issue of BIGTOP-547 is that even after running it, I could not run a simple pi job as root. It would complain /tmp/hadoop-yarn (something like that) is not accessible by root. Although jobs run fine as the user hdfs. So BIGTOP-547 probably need some tweaks to work
      • I had to adjust yarn daemon's priorities. They are set by default as the same as the HDFS ones. Which should also adjust them directly in the packages.
      • Somehow I found some yarn/mapreduce initscripts with typo into /etc/init.d and not belonging to any package (ex: /etc/init.d/hadoop-mapreduce-historyservere)
      • Also, BIGTOP-547 is pretty slow. Not sure we can do anything about it, but it would be great if we could find a way to speed it up.

      So would this approach resolve everyone's concerns?

      Show
      Bruno Mahé added a comment - - edited So here is the first pass of my take on this ticket (file 0001- BIGTOP-637 .-Update-boxgrinder-appliance-for-the-comi.patch from 18/Feb/13 22:09) I went the way I outlined a few comments ago, meaning I reused the script Mark contributed in BIGTOP-547 and wrapped it into an init script which is ran between HDFS and YARN daemons at start up. And this works great. But I also note the following: BIGTOP-547 was not checked in. So in order to get stuff done, this is included for now by boxgrinder. But I would like to sort out its situation so we can check this in and reuse it directly from the packages BIGTOP-547 has a few issues. The first one being its use of sudo, which requires a login shell. Unfortunately login shells are not available from init scripts. So I had to replace sudo by su. This should be included in BIGTOP-547 Another issue of BIGTOP-547 is that even after running it, I could not run a simple pi job as root. It would complain /tmp/hadoop-yarn (something like that) is not accessible by root. Although jobs run fine as the user hdfs. So BIGTOP-547 probably need some tweaks to work I had to adjust yarn daemon's priorities. They are set by default as the same as the HDFS ones. Which should also adjust them directly in the packages. Somehow I found some yarn/mapreduce initscripts with typo into /etc/init.d and not belonging to any package (ex: /etc/init.d/hadoop-mapreduce-historyservere) Also, BIGTOP-547 is pretty slow. Not sure we can do anything about it, but it would be great if we could find a way to speed it up. So would this approach resolve everyone's concerns?
      Hide
      Bruno Mahé added a comment - - edited

      Hold on. I am not done yet.

      Note also that

      files:
        "/root":
          - "start-mr.sh"
      

      with start-mr.sh in the same directory as the appliance, works perfectly fine as described in the documentation.

      Show
      Bruno Mahé added a comment - - edited Hold on. I am not done yet. Note also that files: "/root": - "start-mr.sh" with start-mr.sh in the same directory as the appliance, works perfectly fine as described in the documentation.
      Hide
      Roman Shaposhnik added a comment -

      Created BIGTOP-847 to track fixing the workaround that Cos has put in place.

      Show
      Roman Shaposhnik added a comment - Created BIGTOP-847 to track fixing the workaround that Cos has put in place.
      Hide
      Roman Shaposhnik added a comment -

      A couple of comments, I'm in general +1 on any kind of incremental improvements. IOW, if the settings make Cos's use case easier and they don't break the use case I care about – I'm +1.

      That said, here's my 2 rubles worth of feedback:

      1. I'd really suggest the following:
        -    - "/usr/bin/yes Y | su hdfs /bin/bash -c '/usr/bin/hadoop namenode -format'"
        +    - "/etc/init.d/hadoop-hdfs-namenode init"
        
      2. I think it would be better if we could avoid chkconfig'ing YARN services off, yet this looks like a reasonable workaround for the current situation I'm +1 on it for now, but I'd appreciate a new JIRA to track how are we going to initialize the state of services that require to be running before you can actually initialize their state
      3. I really think the manual starting of YARN should be added to /etc/rc.local

      I'm attaching a patch based on Cos's that takes care of the above.

      Show
      Roman Shaposhnik added a comment - A couple of comments, I'm in general +1 on any kind of incremental improvements. IOW, if the settings make Cos's use case easier and they don't break the use case I care about – I'm +1. That said, here's my 2 rubles worth of feedback: I'd really suggest the following: - - "/usr/bin/yes Y | su hdfs /bin/bash -c '/usr/bin/hadoop namenode -format'" + - "/etc/init.d/hadoop-hdfs-namenode init" I think it would be better if we could avoid chkconfig'ing YARN services off, yet this looks like a reasonable workaround for the current situation I'm +1 on it for now, but I'd appreciate a new JIRA to track how are we going to initialize the state of services that require to be running before you can actually initialize their state I really think the manual starting of YARN should be added to /etc/rc.local I'm attaching a patch based on Cos's that takes care of the above.
      Hide
      Bruno Mahé added a comment - - edited

      Roman> It's already done as of this afternoon.

      Cos> Sure. Let me take a look.

      I am not talking about "other uses". I am talking about bringing 6 HDFS + YARN daemons in say 2 GB and then trying to run almost anything on such a cluster. This is dreadful to say the least.

      So what is the alternative? Not starting them?
      I am not sure to follow what you are proposing. Whether the daemons start on their own or are brought up manually, they would still take the same amount of memory. Also, I am not sure what would be the point to have a VM with just HDFS running (except if you are bringing up a cluster, in which case you would still have the same daemons somewhere else (using the same amount of memory) and would need different set of VMs).

      As for the reference to Cloudera VM: this is exactly what I am referring to. It is literally painful to watch how nothing can get started in this thing. It crawls on all four.

      Could you define "nothing can get started". Last I checked, all the daemons were starting and running fine and the ui was reacting fine.

      Show
      Bruno Mahé added a comment - - edited Roman> It's already done as of this afternoon. Cos> Sure. Let me take a look. I am not talking about "other uses". I am talking about bringing 6 HDFS + YARN daemons in say 2 GB and then trying to run almost anything on such a cluster. This is dreadful to say the least. So what is the alternative? Not starting them? I am not sure to follow what you are proposing. Whether the daemons start on their own or are brought up manually, they would still take the same amount of memory. Also, I am not sure what would be the point to have a VM with just HDFS running (except if you are bringing up a cluster, in which case you would still have the same daemons somewhere else (using the same amount of memory) and would need different set of VMs). As for the reference to Cloudera VM: this is exactly what I am referring to. It is literally painful to watch how nothing can get started in this thing. It crawls on all four. Could you define " nothing can get started ". Last I checked, all the daemons were starting and running fine and the ui was reacting fine.
      Hide
      Konstantin Boudnik added a comment -

      and with the script in the same directory than the appliance definition file.

      As I said in my comment: boxgrinder preserves the hierarchy. Thus you will have to put start-mr.sh into the top directory of the workspace, instead of bigtop-deploy/vm/boxgrinder

      A script with a higher priority than the Apache Hadoop daemons

      The script you're referring to would have to start in the middle of Hadoop startup sequence. Checking HDFS structures requires live namenode.

      Well, this appliance only provides Apache Hadoop right now. So I am not sure for what a user would use it if not using Apache Hadoop

      I am not talking about "other uses". I am talking about bringing 6 HDFS + YARN daemons in say 2 GB and then trying to run almost anything on such a cluster. This is dreadful to say the least.

      As for the reference to Cloudera VM: this is exactly what I am referring to. It is literally painful to watch how nothing can get started in this thing. It crawls on all four.

      At any rate - if you see the better way to do this patch: go ahead and do it the way you want to. I did it the way I think is the best from UX and maintenance stand points.

      Show
      Konstantin Boudnik added a comment - and with the script in the same directory than the appliance definition file. As I said in my comment: boxgrinder preserves the hierarchy. Thus you will have to put start-mr.sh into the top directory of the workspace, instead of bigtop-deploy/vm/boxgrinder A script with a higher priority than the Apache Hadoop daemons The script you're referring to would have to start in the middle of Hadoop startup sequence. Checking HDFS structures requires live namenode. Well, this appliance only provides Apache Hadoop right now. So I am not sure for what a user would use it if not using Apache Hadoop I am not talking about "other uses". I am talking about bringing 6 HDFS + YARN daemons in say 2 GB and then trying to run almost anything on such a cluster. This is dreadful to say the least. As for the reference to Cloudera VM: this is exactly what I am referring to. It is literally painful to watch how nothing can get started in this thing. It crawls on all four. At any rate - if you see the better way to do this patch: go ahead and do it the way you want to. I did it the way I think is the best from UX and maintenance stand points.
      Hide
      Roman Shaposhnik added a comment -

      Guys, any chance you could actually hook it up back to our Jenkins so we can make sure it doesn't bitrot?

      Show
      Roman Shaposhnik added a comment - Guys, any chance you could actually hook it up back to our Jenkins so we can make sure it doesn't bitrot?
      Hide
      Bruno Mahé added a comment - - edited

      But that would means a one more external file to manage. Also, boxgrinder suggest to use relative paths to the external files because the path structures are preserved in the target VM. Hence, my approach is more straight forward

      That's ok, we have git
      It also makes maintenance easier by separating the content of the file with how it is being managed.

      The following should work, shouldn't it?

      files:
        "/root":
          - "start-mr.sh"
      

      and with the script in the same directory than the appliance definition file.

      these services shouldn't be started unless the HDFS has all needed directories in there.

      This requirement is addressable:

      • A script with a higher priority than the Apache Hadoop daemons that check if directories/users are initialized (optionally if it is first boot as well). The script that got recently checked in which initialize HDFS should help in that regard.
      • Having HDFS related daemons set with a higher priorities so they start before the yarn ones.

      Also, we don't know in advance how much memory a user would allot to the VM, thus it might be a dreadful sight trying to run all 6 of these heavy daemons to run in say 2GB of memory.

      Well, this appliance only provides Apache Hadoop right now. So I am not sure for what a user would use it if not using Apache Hadoop
      Also, Cloudera's demo vm has all the CDH's daemons starting on boot (so that includes, Apache HBase, Apache Oozie, Apache Zookeeper...) along with xfce and only recommend 3GB of ram. So by only having Apache Hadoop in this appliance, the memory requirements shouldn't be that big (I don't remember exactly so I don't want to say any number).

      Note also I intended to provide this appliance as a base and have some other appliances inheriting from that one to make it easy to build appliances for different purposes. For instance an appliance that build a datanode for AWS versus another appliance for a developer wishing to have Apache Bigtop in a box where (s)he can try his code.

      Show
      Bruno Mahé added a comment - - edited But that would means a one more external file to manage. Also, boxgrinder suggest to use relative paths to the external files because the path structures are preserved in the target VM. Hence, my approach is more straight forward That's ok, we have git It also makes maintenance easier by separating the content of the file with how it is being managed. The following should work, shouldn't it? files: "/root": - "start-mr.sh" and with the script in the same directory than the appliance definition file. these services shouldn't be started unless the HDFS has all needed directories in there. This requirement is addressable: A script with a higher priority than the Apache Hadoop daemons that check if directories/users are initialized (optionally if it is first boot as well). The script that got recently checked in which initialize HDFS should help in that regard. Having HDFS related daemons set with a higher priorities so they start before the yarn ones. Also, we don't know in advance how much memory a user would allot to the VM, thus it might be a dreadful sight trying to run all 6 of these heavy daemons to run in say 2GB of memory. Well, this appliance only provides Apache Hadoop right now. So I am not sure for what a user would use it if not using Apache Hadoop Also, Cloudera's demo vm has all the CDH's daemons starting on boot (so that includes, Apache HBase, Apache Oozie, Apache Zookeeper...) along with xfce and only recommend 3GB of ram. So by only having Apache Hadoop in this appliance, the memory requirements shouldn't be that big (I don't remember exactly so I don't want to say any number). Note also I intended to provide this appliance as a base and have some other appliances inheriting from that one to make it easy to build appliances for different purposes. For instance an appliance that build a datanode for AWS versus another appliance for a developer wishing to have Apache Bigtop in a box where (s)he can try his code.
      Hide
      Konstantin Boudnik added a comment -

      Thanks for the review Bruno. Addressing your comments above:

      Boxgrinder enables you to embed external files

      But that would means a one more external file to manage. Also, boxgrinder suggest to use relative paths to the external files because the path structures are preserved in the target VM. Hence, my approach is more straight forward

      you deactivate yarn/mapreduce on boot

      these services shouldn't be started unless the HDFS has all needed directories in there. Also, we don't know in advance how much memory a user would allot to the VM, thus it might be a dreadful sight trying to run all 6 of these heavy daemons to run in say 2GB of memory.

      Show
      Konstantin Boudnik added a comment - Thanks for the review Bruno. Addressing your comments above: Boxgrinder enables you to embed external files But that would means a one more external file to manage. Also, boxgrinder suggest to use relative paths to the external files because the path structures are preserved in the target VM. Hence, my approach is more straight forward you deactivate yarn/mapreduce on boot these services shouldn't be started unless the HDFS has all needed directories in there. Also, we don't know in advance how much memory a user would allot to the VM, thus it might be a dreadful sight trying to run all 6 of these heavy daemons to run in say 2GB of memory.
      Hide
      Bruno Mahé added a comment -

      Nice patch!

      Some notes:

      • Boxgrinder enables you to embed external files without going through packages. So the start-mr.sh script could be in a separate file
      • I see you deactivate yarn/mapreduce on boot. I also see your message to manually start the cluster. But why not starting it on boot? Wouldn't tweaking the init script boot priorities/dependencies do the trick?
      Show
      Bruno Mahé added a comment - Nice patch! Some notes: Boxgrinder enables you to embed external files without going through packages. So the start-mr.sh script could be in a separate file I see you deactivate yarn/mapreduce on boot. I also see your message to manually start the cluster. But why not starting it on boot? Wouldn't tweaking the init script boot priorities/dependencies do the trick?
      Hide
      Konstantin Boudnik added a comment -

      This seems to be doing the trick and guarantees that YARN layer doesn't try to start before all HDFS preparations are complete.

      In the interest of full disclosure, I have chosen to hint the user about running the MR startup script instead of doing this automatically.

      Apparently, this can be improved upon and made totally transparent, including re-enabling auto-start of YARN daemons.

      Show
      Konstantin Boudnik added a comment - This seems to be doing the trick and guarantees that YARN layer doesn't try to start before all HDFS preparations are complete. In the interest of full disclosure, I have chosen to hint the user about running the MR startup script instead of doing this automatically. Apparently, this can be improved upon and made totally transparent, including re-enabling auto-start of YARN daemons.
      Hide
      Konstantin Boudnik added a comment - - edited

      Oh well - never mind. Apparently it is because of piece of junk CentOS - boxgrinder doesn't really work on it. I took their Fedora meta appliance - everything looks normal now.

      Show
      Konstantin Boudnik added a comment - - edited Oh well - never mind. Apparently it is because of piece of junk CentOS - boxgrinder doesn't really work on it. I took their Fedora meta appliance - everything looks normal now.
      Hide
      Konstantin Boudnik added a comment -

      Trying to work on this ticket - description might be a big help, of course, as I might be doing something totally different - and see that namenode doesn't get formatted at all.
      It might be caused by my build machine misconfiguration - a junky centos after all ;( - as I see the following error diagnostic:

      F, 2013-02-13T14:19:55.525716 #19489 FATAL – : Guestfs::Error: command failed: LC_ALL=C '/usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.10.4/lib/boxgrinder-build/helpers/qemu.wrapper' -nographic -help

      If qemu is located on a non-standard path, try setting the LIBGUESTFS_QEMU

      I have stock CentOS6.3 with all bells and whistles installed, but there's no /usr/bin/qemu. And as far as I can tell - there's no such thing in the qemu-kvm package.

      Any ideas?

      Show
      Konstantin Boudnik added a comment - Trying to work on this ticket - description might be a big help, of course, as I might be doing something totally different - and see that namenode doesn't get formatted at all. It might be caused by my build machine misconfiguration - a junky centos after all ;( - as I see the following error diagnostic: F, 2013-02-13T14:19:55.525716 #19489 FATAL – : Guestfs::Error: command failed: LC_ALL=C '/usr/lib/ruby/gems/1.8/gems/boxgrinder-build-0.10.4/lib/boxgrinder-build/helpers/qemu.wrapper' -nographic -help If qemu is located on a non-standard path, try setting the LIBGUESTFS_QEMU I have stock CentOS6.3 with all bells and whistles installed, but there's no /usr/bin/qemu. And as far as I can tell - there's no such thing in the qemu-kvm package. Any ideas?

        People

        • Assignee:
          Bruno Mahé
          Reporter:
          Bruno Mahé
        • Votes:
          0 Vote for this issue
          Watchers:
          5 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development