Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12899

External distribution stitching scripts do not work correctly on Windows.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: build
    • Labels:
      None
    • Environment:

      Windows

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      In HADOOP-12850, we pulled the dist-layout-stitching and dist-tar-stitching scripts out of hadoop-dist/pom.xml and into external files. It appears this change is not working correctly on Windows.

      1. HADOOP-12899.001.patch
        4 kB
        Chris Nauroth
      2. HADOOP-12899.002.patch
        2 kB
        Chris Nauroth

        Issue Links

          Activity

          Hide
          cnauroth Chris Nauroth added a comment -
          mvn clean install -Pdist -Dtar -Dbundle.snappy -Dsnappy.lib=C:\snappy\lib -DskipTests -Dmaven.javadoc.skip=true
          
          [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.3.1:exec (dist) on project hadoop-dist: Command execution failed. Cannot run program "C:\hdc\hadoop-dist\..\dev-support\bin\dist-layout-stitching" (in directory "C:\hdc\hadoop-dist\target"): CreateProcess error=193, %1 is not a valid Win32 application -> [Help 1]
          
          Show
          cnauroth Chris Nauroth added a comment - mvn clean install -Pdist -Dtar -Dbundle.snappy -Dsnappy.lib=C:\snappy\lib -DskipTests -Dmaven.javadoc.skip= true [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.3.1:exec (dist) on project hadoop-dist: Command execution failed. Cannot run program "C:\hdc\hadoop-dist\..\dev-support\bin\dist-layout-stitching" (in directory "C:\hdc\hadoop-dist\target" ): CreateProcess error=193, %1 is not a valid Win32 application -> [Help 1]
          Hide
          aw Allen Wittenauer added a comment -

          Wow, weird. I wonder if the problem is using usr/bin/env as the bang path, given other exec-maven-plugin bits work?

          Show
          aw Allen Wittenauer added a comment - Wow, weird. I wonder if the problem is using usr/bin/env as the bang path, given other exec-maven-plugin bits work?
          Hide
          cnauroth Chris Nauroth added a comment -

          On Windows, it can't rely on the bang path or even really a .sh extension. Before HADOOP-12850, this worked by having exec-maven-plugin invoke bash directly in the executable attribute. I did a quick hack change locally to confirm that switching back to direct invocation of bash would work.

          Allen Wittenauer, I assume you want to retain the behavior of invoking through /usr/bin/env so that it can look up bash from the user environment, right? Can you confirm? If we want to keep that, then I think it puts me on the path of setting up a separate native-win profile for the special case in hadoop-dist/pom.xml.

          Show
          cnauroth Chris Nauroth added a comment - On Windows, it can't rely on the bang path or even really a .sh extension. Before HADOOP-12850 , this worked by having exec-maven-plugin invoke bash directly in the executable attribute. I did a quick hack change locally to confirm that switching back to direct invocation of bash would work. Allen Wittenauer , I assume you want to retain the behavior of invoking through /usr/bin/env so that it can look up bash from the user environment, right? Can you confirm? If we want to keep that, then I think it puts me on the path of setting up a separate native-win profile for the special case in hadoop-dist/pom.xml.
          Hide
          aw Allen Wittenauer added a comment -

          It would be good to lookup bash from the environment, but honestly it's an extreme edge case. So if it needs to be hard-coded, then so be it. My big goal was mainly to pull large chunks of shell out of the pom files so that they could get some love from shellcheck and to make them easier to debug. I'm in the process of doing the same thing with HADOOP-12892.

          Show
          aw Allen Wittenauer added a comment - It would be good to lookup bash from the environment, but honestly it's an extreme edge case. So if it needs to be hard-coded, then so be it. My big goal was mainly to pull large chunks of shell out of the pom files so that they could get some love from shellcheck and to make them easier to debug. I'm in the process of doing the same thing with HADOOP-12892 .
          Hide
          cnauroth Chris Nauroth added a comment - - edited

          I'm attaching patch v001. This goes back to the strategy of invoking the interpreter directly. I tested this successfully on Windows and Linux.

          I also changed the bang lines within the scripts. Technically, that's not really necessary for the fix, but I didn't want people reading it to mistakenly think that it's routing through env during the build.

          Show
          cnauroth Chris Nauroth added a comment - - edited I'm attaching patch v001. This goes back to the strategy of invoking the interpreter directly. I tested this successfully on Windows and Linux. I also changed the bang lines within the scripts. Technically, that's not really necessary for the fix, but I didn't want people reading it to mistakenly think that it's routing through env during the build.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 10s Docker mode activated.
          0 shelldocs 0m 3s Shelldocs was not available.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 7m 10s trunk passed
          +1 compile 0m 9s trunk passed with JDK v1.8.0_74
          +1 compile 0m 10s trunk passed with JDK v1.7.0_95
          +1 mvnsite 0m 12s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 javadoc 0m 9s trunk passed with JDK v1.8.0_74
          +1 javadoc 0m 10s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 11s the patch passed
          +1 compile 0m 8s the patch passed with JDK v1.8.0_74
          +1 javac 0m 8s the patch passed
          +1 compile 0m 8s the patch passed with JDK v1.7.0_95
          +1 javac 0m 8s the patch passed
          +1 mvnsite 0m 13s the patch passed
          +1 mvneclipse 0m 9s the patch passed
          +1 shellcheck 0m 8s The applied patch generated 0 new + 97 unchanged - 1 fixed = 97 total (was 98)
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 javadoc 0m 10s the patch passed with JDK v1.8.0_74
          +1 javadoc 0m 10s the patch passed with JDK v1.7.0_95
          +1 unit 0m 8s hadoop-dist in the patch passed with JDK v1.8.0_74.
          +1 unit 0m 8s hadoop-dist in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 18s Patch does not generate ASF License warnings.
          11m 5s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12791874/HADOOP-12899.001.patch
          JIRA Issue HADOOP-12899
          Optional Tests asflicense shellcheck shelldocs compile javac javadoc mvninstall mvnsite unit xml
          uname Linux 3b8f515db3b7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 391da36
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          shellcheck v0.4.3
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8814/testReport/
          modules C: hadoop-dist U: hadoop-dist
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8814/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. 0 shelldocs 0m 3s Shelldocs was not available. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 10s trunk passed +1 compile 0m 9s trunk passed with JDK v1.8.0_74 +1 compile 0m 10s trunk passed with JDK v1.7.0_95 +1 mvnsite 0m 12s trunk passed +1 mvneclipse 0m 12s trunk passed +1 javadoc 0m 9s trunk passed with JDK v1.8.0_74 +1 javadoc 0m 10s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 11s the patch passed +1 compile 0m 8s the patch passed with JDK v1.8.0_74 +1 javac 0m 8s the patch passed +1 compile 0m 8s the patch passed with JDK v1.7.0_95 +1 javac 0m 8s the patch passed +1 mvnsite 0m 13s the patch passed +1 mvneclipse 0m 9s the patch passed +1 shellcheck 0m 8s The applied patch generated 0 new + 97 unchanged - 1 fixed = 97 total (was 98) +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 0s The patch has no ill-formed XML file. +1 javadoc 0m 10s the patch passed with JDK v1.8.0_74 +1 javadoc 0m 10s the patch passed with JDK v1.7.0_95 +1 unit 0m 8s hadoop-dist in the patch passed with JDK v1.8.0_74. +1 unit 0m 8s hadoop-dist in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 18s Patch does not generate ASF License warnings. 11m 5s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12791874/HADOOP-12899.001.patch JIRA Issue HADOOP-12899 Optional Tests asflicense shellcheck shelldocs compile javac javadoc mvninstall mvnsite unit xml uname Linux 3b8f515db3b7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 391da36 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 shellcheck v0.4.3 JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8814/testReport/ modules C: hadoop-dist U: hadoop-dist Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8814/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          cnauroth Chris Nauroth added a comment -

          There are no tests, because this is a change in the build process only. As I mentioned earlier, I did manual testing on both Windows and Linux to confirm that bundling Snappy works correctly.

          Show
          cnauroth Chris Nauroth added a comment - There are no tests, because this is a change in the build process only. As I mentioned earlier, I did manual testing on both Windows and Linux to confirm that bundling Snappy works correctly.
          Hide
          andrew.wang Andrew Wang added a comment -

          Hey Chris, on my Ubuntu box bash is at /bin/bash not /usr/bin/bash. Shall we update the hash bangs for clarity?

          Show
          andrew.wang Andrew Wang added a comment - Hey Chris, on my Ubuntu box bash is at /bin/bash not /usr/bin/bash . Shall we update the hash bangs for clarity?
          Hide
          cnauroth Chris Nauroth added a comment -

          On CentOS, it's /usr/bin/bash. This sounds like good enough reason to keep the old bang line using /usr/bin/env. Here is patch v002 with just the pom.xml change. It's not necessary to change the bang lines to fix this issue.

          Show
          cnauroth Chris Nauroth added a comment - On CentOS, it's /usr/bin/bash. This sounds like good enough reason to keep the old bang line using /usr/bin/env. Here is patch v002 with just the pom.xml change. It's not necessary to change the bang lines to fix this issue.
          Hide
          andrew.wang Andrew Wang added a comment -

          I actually just checked a CentOS 6.4 box too, and it came up /bin/bash:

          # which bash
          /bin/bash
          # lsb_release -a
          LSB Version:	:base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch
          Distributor ID:	CentOS
          Description:	CentOS release 6.4 (Final)
          Release:	6.4
          Codename:	Final
          # /usr/bin/bash
          -bash: /usr/bin/bash: No such file or directory
          
          Show
          andrew.wang Andrew Wang added a comment - I actually just checked a CentOS 6.4 box too, and it came up /bin/bash: # which bash /bin/bash # lsb_release -a LSB Version: :base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch Distributor ID: CentOS Description: CentOS release 6.4 (Final) Release: 6.4 Codename: Final # /usr/bin/bash -bash: /usr/bin/bash: No such file or directory
          Hide
          cnauroth Chris Nauroth added a comment -

          Bottom line: let's please go with patch v002.

          Show
          cnauroth Chris Nauroth added a comment - Bottom line: let's please go with patch v002.
          Hide
          cnauroth Chris Nauroth added a comment -

          FWIW, mine is CentOS 7 and seems to be different. (See below.) Since patch v002 keeps the /usr/bin/env, it won't really matter.

          > lsb_release -a
          LSB Version:	:core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
          Distributor ID:	CentOS
          Description:	CentOS Linux release 7.1.1503 (Core) 
          Release:	7.1.1503
          Codename:	Core
          
          > rpmquery -f /bin/bash
          bash-4.2.46-12.el7.x86_64
          
          > rpmquery -f /usr/bin/bash
          bash-4.2.46-12.el7.x86_64
          
          Show
          cnauroth Chris Nauroth added a comment - FWIW, mine is CentOS 7 and seems to be different. (See below.) Since patch v002 keeps the /usr/bin/env, it won't really matter. > lsb_release -a LSB Version: :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch Distributor ID: CentOS Description: CentOS Linux release 7.1.1503 (Core) Release: 7.1.1503 Codename: Core > rpmquery -f /bin/bash bash-4.2.46-12.el7.x86_64 > rpmquery -f /usr/bin/bash bash-4.2.46-12.el7.x86_64
          Hide
          andrew.wang Andrew Wang added a comment -

          Sure, I'm +1. Thanks Chris!

          Show
          andrew.wang Andrew Wang added a comment - Sure, I'm +1. Thanks Chris!
          Hide
          aw Allen Wittenauer added a comment -

          This makes me wonder how shelldocs and releasedocsmaker are working on Windows. Hmm.

          Show
          aw Allen Wittenauer added a comment - This makes me wonder how shelldocs and releasedocsmaker are working on Windows. Hmm.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 9s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 6m 42s trunk passed
          +1 compile 0m 9s trunk passed with JDK v1.8.0_74
          +1 compile 0m 10s trunk passed with JDK v1.7.0_95
          +1 mvnsite 0m 14s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 javadoc 0m 9s trunk passed with JDK v1.8.0_74
          +1 javadoc 0m 11s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 10s the patch passed
          +1 compile 0m 6s the patch passed with JDK v1.8.0_74
          +1 javac 0m 6s the patch passed
          +1 compile 0m 8s the patch passed with JDK v1.7.0_95
          +1 javac 0m 8s the patch passed
          +1 mvnsite 0m 11s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 javadoc 0m 7s the patch passed with JDK v1.8.0_74
          +1 javadoc 0m 8s the patch passed with JDK v1.7.0_95
          +1 unit 0m 6s hadoop-dist in the patch passed with JDK v1.8.0_74.
          +1 unit 0m 8s hadoop-dist in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 18s Patch does not generate ASF License warnings.
          10m 14s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12792110/HADOOP-12899.002.patch
          JIRA Issue HADOOP-12899
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml
          uname Linux 543d17acd2db 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 0233d4e
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8825/testReport/
          modules C: hadoop-dist U: hadoop-dist
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8825/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 9s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 42s trunk passed +1 compile 0m 9s trunk passed with JDK v1.8.0_74 +1 compile 0m 10s trunk passed with JDK v1.7.0_95 +1 mvnsite 0m 14s trunk passed +1 mvneclipse 0m 12s trunk passed +1 javadoc 0m 9s trunk passed with JDK v1.8.0_74 +1 javadoc 0m 11s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 10s the patch passed +1 compile 0m 6s the patch passed with JDK v1.8.0_74 +1 javac 0m 6s the patch passed +1 compile 0m 8s the patch passed with JDK v1.7.0_95 +1 javac 0m 8s the patch passed +1 mvnsite 0m 11s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 0s The patch has no ill-formed XML file. +1 javadoc 0m 7s the patch passed with JDK v1.8.0_74 +1 javadoc 0m 8s the patch passed with JDK v1.7.0_95 +1 unit 0m 6s hadoop-dist in the patch passed with JDK v1.8.0_74. +1 unit 0m 8s hadoop-dist in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 18s Patch does not generate ASF License warnings. 10m 14s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12792110/HADOOP-12899.002.patch JIRA Issue HADOOP-12899 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml uname Linux 543d17acd2db 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0233d4e Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8825/testReport/ modules C: hadoop-dist U: hadoop-dist Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8825/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          cnauroth Chris Nauroth added a comment -

          Nope they don't. Let me see if I can get those fixed too within the scope of this patch.

          Show
          cnauroth Chris Nauroth added a comment - Nope they don't. Let me see if I can get those fixed too within the scope of this patch.
          Hide
          aw Allen Wittenauer added a comment -

          Argh. Thanks.

          I'll rework the other two patches that use the maven exec plugin then using this patch as a pattern.

          Thanks.

          Show
          aw Allen Wittenauer added a comment - Argh. Thanks. I'll rework the other two patches that use the maven exec plugin then using this patch as a pattern. Thanks.
          Hide
          cnauroth Chris Nauroth added a comment -

          I looked into shelldocs more. First, there is the same fix required for invoking the shell directly through maven-exec-plugin. However, that alone isn't sufficient to get shelldocs working on Windows. It looks like there are also going to be some requirements around translating paths using cygpath -w for the GPG commands. Even getting past that, I have some as-yet-undiagnosed error preventing it from working fully.

          At this point, I think I want to proceed with committing the existing patch v002 based on Andrew's +1. I'd like to defer shelldocs and releasedocmaker to a separate issue. I don't have time to dig into those further right now, and getting the distro build working (without the site) is still a helpful step for anyone who wants to do a distro build from trunk on Windows.

          I'll wait 24 hours before committing just in case anyone disagrees.

          Show
          cnauroth Chris Nauroth added a comment - I looked into shelldocs more. First, there is the same fix required for invoking the shell directly through maven-exec-plugin. However, that alone isn't sufficient to get shelldocs working on Windows. It looks like there are also going to be some requirements around translating paths using cygpath -w for the GPG commands. Even getting past that, I have some as-yet-undiagnosed error preventing it from working fully. At this point, I think I want to proceed with committing the existing patch v002 based on Andrew's +1. I'd like to defer shelldocs and releasedocmaker to a separate issue. I don't have time to dig into those further right now, and getting the distro build working (without the site) is still a helpful step for anyone who wants to do a distro build from trunk on Windows. I'll wait 24 hours before committing just in case anyone disagrees.
          Hide
          cnauroth Chris Nauroth added a comment -

          I have committed this to trunk. I filed HADOOP-12915 for follow-up on shelldocs and releasedocmaker. Thank you, Andrew and Allen.

          Show
          cnauroth Chris Nauroth added a comment - I have committed this to trunk. I filed HADOOP-12915 for follow-up on shelldocs and releasedocmaker. Thank you, Andrew and Allen.

            People

            • Assignee:
              cnauroth Chris Nauroth
              Reporter:
              cnauroth Chris Nauroth
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development