Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-453

external table metastore and execution codepaths use default FileSystem object instead of using one derived from the supplied path

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.4.0
    • Component/s: Metastore
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Use FileSystem derived from user supplied path when present instead of default.

      Description

      with fs.default.name set to file:///:

      hive> create external table kvu(key int, val string) location 's3n://data.s3ndemo.hive/kv1.txt'
      FAILED: Error in metadata: java.lang.IllegalArgumentException: Wrong FS: s3n://data.s3ndemo.hive/kv1.txt, expected: file:///

      stack trace:

      at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:322)
      at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem
      .java:52)
      at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.jav
      a:308)
      at org.apache.hadoop.fs.ChecksumFileSystem.mkdirs(ChecksumFileSystem.jav
      a:462)
      at org.apache.hadoop.hive.metastore.Warehouse.mkdirs(Warehouse.java:90)
      at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_tabl
      e(HiveMetaStore.java:293)
      at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(Hive
      MetaStoreClient.java:252)
      at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:256)

      1. hive-453.4.patch
        40 kB
        Prasad Chakka
      2. hive-453.4.tests.patch
        2 kB
        Prasad Chakka
      3. hive-453.br0.3.4.patch
        48 kB
        Prasad Chakka
      4. hive-453.patch.1
        31 kB
        Joydeep Sen Sarma
      5. hive-453.patch.2
        42 kB
        Joydeep Sen Sarma
      6. hive-453.patch.3
        42 kB
        Joydeep Sen Sarma

        Activity

        Hide
        jsensarma Joydeep Sen Sarma added a comment -

        generalizing - there seems to be issues in execution time code path as well. after patching metastore issues:

        hive> create external table kvu(key int, val string) location 's3n://data.s3ndemo.hive/kv1.txt';
        create external table kvu(key int, val string) location 's3n://data.s3ndemo.hive/kv1.txt';
        OK
        Time taken: 13.49 seconds
        hive> select * from kvu;
        select * from kvu;
        OK
        Failed with exception Wrong FS: s3n://data.s3ndemo.hive/kv1.txt, expected: file:///

        java.lang.IllegalArgumentException: Wrong FS: s3n://data.s3ndemo.hive/kv1.txt, e
        xpected: file:///
        at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:322)
        at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem
        .java:52)
        at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSys
        tem.java:416)
        at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.
        java:244)
        at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:651)
        at org.apache.hadoop.hive.ql.exec.FetchTask.getNextPath(FetchTask.java:1
        79)
        at org.apache.hadoop.hive.ql.exec.FetchTask.getRecordReader(FetchTask.ja
        va:223)
        at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:287)
        at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:322)

        Show
        jsensarma Joydeep Sen Sarma added a comment - generalizing - there seems to be issues in execution time code path as well. after patching metastore issues: hive> create external table kvu(key int, val string) location 's3n://data.s3ndemo.hive/kv1.txt'; create external table kvu(key int, val string) location 's3n://data.s3ndemo.hive/kv1.txt'; OK Time taken: 13.49 seconds hive> select * from kvu; select * from kvu; OK Failed with exception Wrong FS: s3n://data.s3ndemo.hive/kv1.txt, expected: file:/// java.lang.IllegalArgumentException: Wrong FS: s3n://data.s3ndemo.hive/kv1.txt, e xpected: file:/// at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:322) at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem .java:52) at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSys tem.java:416) at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem. java:244) at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:651) at org.apache.hadoop.hive.ql.exec.FetchTask.getNextPath(FetchTask.java:1 79) at org.apache.hadoop.hive.ql.exec.FetchTask.getRecordReader(FetchTask.ja va:223) at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:287) at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:322)
        Hide
        jsensarma Joydeep Sen Sarma added a comment -
        • the bulk of the changes are to not use the default filesystem object in many code paths and instead use the one derived from the path/conf.
        • the FetchTask.java changes are mostly indentation changes
        • can't think of how to add a test right now - in order to add a test - i need two functioning file systems - one the default one and the other for an external table. without a minimr cluster to run the tests - i don't have this available.
        • there was a small bug in DDLTask - addPartition that i realized while testing that has also been fixed. partition directory creation should be done before comitting metastore transaction - otherwise the transaction can be committed without the DDL statement succeeding.
        Show
        jsensarma Joydeep Sen Sarma added a comment - the bulk of the changes are to not use the default filesystem object in many code paths and instead use the one derived from the path/conf. the FetchTask.java changes are mostly indentation changes can't think of how to add a test right now - in order to add a test - i need two functioning file systems - one the default one and the other for an external table. without a minimr cluster to run the tests - i don't have this available. there was a small bug in DDLTask - addPartition that i realized while testing that has also been fixed. partition directory creation should be done before comitting metastore transaction - otherwise the transaction can be committed without the DDL statement succeeding.
        Hide
        jsensarma Joydeep Sen Sarma added a comment -

        would like to submit this to trunk and branch 0.3 (so that it can go into either 0.3 or 0.3.1)

        Show
        jsensarma Joydeep Sen Sarma added a comment - would like to submit this to trunk and branch 0.3 (so that it can go into either 0.3 or 0.3.1)
        Hide
        prasadc Prasad Chakka added a comment -

        > * there was a small bug in DDLTask - addPartition that i realized while testing that has also been fixed. partition directory creation should be done before comitting metastore transaction - otherwise the transaction can be committed without the DDL statement succeeding.
        What if the directory is created but the transaction fails? We need to delete the directory. I thought it is less of a problem if the metadata gets updated but not the directory since while loading the partition, that directory gets created. Otherwise there will be hdfs directories that might never get touched again.

        Show
        prasadc Prasad Chakka added a comment - > * there was a small bug in DDLTask - addPartition that i realized while testing that has also been fixed. partition directory creation should be done before comitting metastore transaction - otherwise the transaction can be committed without the DDL statement succeeding. What if the directory is created but the transaction fails? We need to delete the directory. I thought it is less of a problem if the metadata gets updated but not the directory since while loading the partition, that directory gets created. Otherwise there will be hdfs directories that might never get touched again.
        Hide
        jsensarma Joydeep Sen Sarma added a comment -

        true - directories need to be cleaned up. this is a pre-existing problem though - all other DDL tasks also first create directory and then commit transaction.

        Show
        jsensarma Joydeep Sen Sarma added a comment - true - directories need to be cleaned up. this is a pre-existing problem though - all other DDL tasks also first create directory and then commit transaction.
        Hide
        prasadc Prasad Chakka added a comment -

        create table part leaves the directory intact even if the transaction fails. i don't remember writing it that way but we should change that to create directory after the transaction has been committed. I don't think hive execution complains that directory doesn't exist.

        the drop table & drop partition try to delete after the transaction is committed only because we don't want to delete the data without deleting metadata.

        but, as you say we could try to drop the table/partitions directory if the transaction fails. then it should be fine.

        Show
        prasadc Prasad Chakka added a comment - create table part leaves the directory intact even if the transaction fails. i don't remember writing it that way but we should change that to create directory after the transaction has been committed. I don't think hive execution complains that directory doesn't exist. the drop table & drop partition try to delete after the transaction is committed only because we don't want to delete the data without deleting metadata. but, as you say we could try to drop the table/partitions directory if the transaction fails. then it should be fine.
        Hide
        athusoo Ashish Thusoo added a comment -

        Assigned.

        Show
        athusoo Ashish Thusoo added a comment - Assigned.
        Hide
        jsensarma Joydeep Sen Sarma added a comment -

        if the file systems operations are done afterwards and they fail (throw an exception) - then to the user it seems like the DDL statement failed (since we report the error). So it's a weird experience.

        will publish another patch that rolls back fs operations.

        Show
        jsensarma Joydeep Sen Sarma added a comment - if the file systems operations are done afterwards and they fail (throw an exception) - then to the user it seems like the DDL statement failed (since we report the error). So it's a weird experience. will publish another patch that rolls back fs operations.
        Hide
        jsensarma Joydeep Sen Sarma added a comment -
        • add rollback of fs operations if transaction fails
        • added a couple of negative tests that verify that metadata transactions don't succeed if fs operations fail.

        also added a cli option (by default false) to keep processing queries even if we hit errors. this was necessary for writing the negative tests and is likely useful more generally as well.

        Show
        jsensarma Joydeep Sen Sarma added a comment - add rollback of fs operations if transaction fails added a couple of negative tests that verify that metadata transactions don't succeed if fs operations fail. also added a cli option (by default false) to keep processing queries even if we hit errors. this was necessary for writing the negative tests and is likely useful more generally as well.
        Hide
        prasadc Prasad Chakka added a comment -

        +1

        running tests..

        Show
        prasadc Prasad Chakka added a comment - +1 running tests..
        Hide
        prasadc Prasad Chakka added a comment -

        2 Errors in TestCliNegativeDriver (error message is different if s3n is not configured on the test machine) and 2 in TestHiveMetaStoreChecker and 1 in TestCliDriver.

        diff -a -I (file:)\|(/tmp/.*) /data/users/pchakka/workspace/oshive3/build/ql/test/logs/clientnegative/external1.q.out /data/users/pchakka/workspace/oshive3/ql/src/test/results/clientnegative/external1.q.out
        [junit] 1c1
        < FAILED: Error in metadata: MetaException(message:Got exception: java.io.IOException No FileSystem for scheme: s3n)
        [junit]
        [junit] > FAILED: Error in metadata: java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).

        diff -a -I (file:&#41;\|(/tmp/.*) /data/users/pchakka/workspace/oshive3/build/ql/test/logs/clientnegative/external2.q.out /data/users/pchakka/workspace/oshive3/ql/src/test/results/clientnegative/external2.q.out
        [junit] 1c1
        [junit] Exception: Client execution results failed with error code = 1
        [junit] < FAILED: Error in metadata: MetaException(message:Got exception: java.io.IOException No FileSystem for scheme: s3n)
        [junit]
        [junit] > FAILED: Error in metadata: java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively).

        Running org.apache.hadoop.hive.ql.metadata.TestHiveMetaStoreChecker
        [junit] Tests run: 2, Failures: 0, Errors: 2, Time elapsed: 6.154 sec

        Show
        prasadc Prasad Chakka added a comment - 2 Errors in TestCliNegativeDriver (error message is different if s3n is not configured on the test machine) and 2 in TestHiveMetaStoreChecker and 1 in TestCliDriver. diff -a -I ( file:&#41;\ |(/tmp/.*) /data/users/pchakka/workspace/oshive3/build/ql/test/logs/clientnegative/external1.q.out /data/users/pchakka/workspace/oshive3/ql/src/test/results/clientnegative/external1.q.out [junit] 1c1 < FAILED: Error in metadata: MetaException(message:Got exception: java.io.IOException No FileSystem for scheme: s3n) [junit] — [junit] > FAILED: Error in metadata: java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively). diff -a -I ( file:&#41;\ |(/tmp/.*) /data/users/pchakka/workspace/oshive3/build/ql/test/logs/clientnegative/external2.q.out /data/users/pchakka/workspace/oshive3/ql/src/test/results/clientnegative/external2.q.out [junit] 1c1 [junit] Exception: Client execution results failed with error code = 1 [junit] < FAILED: Error in metadata: MetaException(message:Got exception: java.io.IOException No FileSystem for scheme: s3n) [junit] — [junit] > FAILED: Error in metadata: java.lang.IllegalArgumentException: AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively) of a s3n URL, or by setting the fs.s3n.awsAccessKeyId or fs.s3n.awsSecretAccessKey properties (respectively). Running org.apache.hadoop.hive.ql.metadata.TestHiveMetaStoreChecker [junit] Tests run: 2, Failures: 0, Errors: 2, Time elapsed: 6.154 sec
        Hide
        jsensarma Joydeep Sen Sarma added a comment -

        thanks for reviewing.

        strange that s3n is not being found. is this being tested against hadoop-19? (i made some changes to test classpath so that jets3t library was in classpath).

        regardless - putting s3n there was a bad idea - since the error message even in the best case would depend on whether the machine is connected or not. let me put something more deterministic and regenerate the patch.

        Show
        jsensarma Joydeep Sen Sarma added a comment - thanks for reviewing. strange that s3n is not being found. is this being tested against hadoop-19? (i made some changes to test classpath so that jets3t library was in classpath). regardless - putting s3n there was a bad idea - since the error message even in the best case would depend on whether the machine is connected or not. let me put something more deterministic and regenerate the patch.
        Hide
        prasadc Prasad Chakka added a comment -

        tested against 0.17.2.1

        Show
        prasadc Prasad Chakka added a comment - tested against 0.17.2.1
        Hide
        jsensarma Joydeep Sen Sarma added a comment -

        remove use of s3n filesystem from tests.

        Show
        jsensarma Joydeep Sen Sarma added a comment - remove use of s3n filesystem from tests.
        Hide
        prasadc Prasad Chakka added a comment -

        testing...

        Show
        prasadc Prasad Chakka added a comment - testing...
        Hide
        prasadc Prasad Chakka added a comment -

        new patch fixed s3n tests but not the other tests

        Tests run: 2, Failures: 0, Errors: 2, Time elapsed: 5.118 sec
        Test org.apache.hadoop.hive.ql.metadata.TestHiveMetaStoreChecker FAILED
        
            [junit] Begin query: input20.q
            [junit] plan = /tmp/plan2216.xml
            [junit] plan = /tmp/plan2217.xml
            [junit] diff -a -I \(file:\)\|\(/tmp/.*\) /Users/pchakka/workspace/hive1/build/ql/test/logs/clientpositive/input20.q.out /Users/pchakka/workspace/hive1/ql/src/test/results/clientpositiv\
        e/input20.q.out
            [junit] 83,391c83,391
            [junit] < NULL      0
            [junit] < NULL      10
            [junit] < NULL      100
            [junit] < NULL      103
            [junit] < NULL      104
            [junit] < NULL      105
            [junit] < NULL      11
            [junit] < NULL      111
            [junit] < NULL      113
        
        Tests run: 232, Failures: 1, Errors: 0, Time elapsed: 2,296.995 sec
        Test org.apache.hadoop.hive.cli.TestCliDriver FAILED
        
        Show
        prasadc Prasad Chakka added a comment - new patch fixed s3n tests but not the other tests Tests run: 2, Failures: 0, Errors: 2, Time elapsed: 5.118 sec Test org.apache.hadoop.hive.ql.metadata.TestHiveMetaStoreChecker FAILED [junit] Begin query: input20.q [junit] plan = /tmp/plan2216.xml [junit] plan = /tmp/plan2217.xml [junit] diff -a -I \(file:\)\|\(/tmp/.*\) /Users/pchakka/workspace/hive1/build/ql/test/logs/clientpositive/input20.q.out /Users/pchakka/workspace/hive1/ql/src/test/results/clientpositiv\ e/input20.q.out [junit] 83,391c83,391 [junit] < NULL 0 [junit] < NULL 10 [junit] < NULL 100 [junit] < NULL 103 [junit] < NULL 104 [junit] < NULL 105 [junit] < NULL 11 [junit] < NULL 111 [junit] < NULL 113 Tests run: 232, Failures: 1, Errors: 0, Time elapsed: 2,296.995 sec Test org.apache.hadoop.hive.cli.TestCliDriver FAILED
        Hide
        prasadc Prasad Chakka added a comment -

        TestHiveMetaStoreChecker log contains this. Looks like 'fs' is not initialized in testPartitionsCheck()

        <testcase classname="org.apache.hadoop.hive.ql.metadata.TestHiveMetaStoreChecker" name="testPartitionsCheck" time="0.572">
            <error type="java.lang.NullPointerException">java.lang.NullPointerException
                at org.apache.hadoop.hive.ql.metadata.TestHiveMetaStoreChecker.testPartitionsCheck(TestHiveMetaStoreChecker.java:187)
                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
                at java.lang.reflect.Method.invoke(Method.java:597)
                at junit.framework.TestCase.runTest(TestCase.java:154)
                at junit.framework.TestCase.runBare(TestCase.java:127)
                at junit.framework.TestResult$1.protect(TestResult.java:106)
                at junit.framework.TestResult.runProtected(TestResult.java:124)
                at junit.framework.TestResult.run(TestResult.java:109)
                at junit.framework.TestCase.run(TestCase.java:118)
                at junit.framework.TestSuite.runTest(TestSuite.java:208)
                at junit.framework.TestSuite.run(TestSuite.java:203)
                at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:421)
                at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:912)
                at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:766)
        </error>
          </testcase>
        
        Show
        prasadc Prasad Chakka added a comment - TestHiveMetaStoreChecker log contains this. Looks like 'fs' is not initialized in testPartitionsCheck() <testcase classname= "org.apache.hadoop.hive.ql.metadata.TestHiveMetaStoreChecker" name= "testPartitionsCheck" time= "0.572" > <error type= "java.lang.NullPointerException" >java.lang.NullPointerException at org.apache.hadoop.hive.ql.metadata.TestHiveMetaStoreChecker.testPartitionsCheck(TestHiveMetaStoreChecker.java:187) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at junit.framework.TestCase.runTest(TestCase.java:154) at junit.framework.TestCase.runBare(TestCase.java:127) at junit.framework.TestResult$1.protect(TestResult.java:106) at junit.framework.TestResult.runProtected(TestResult.java:124) at junit.framework.TestResult.run(TestResult.java:109) at junit.framework.TestCase.run(TestCase.java:118) at junit.framework.TestSuite.runTest(TestSuite.java:208) at junit.framework.TestSuite.run(TestSuite.java:203) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:421) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:912) at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:766) </error> </testcase>
        Hide
        prasadc Prasad Chakka added a comment -

        I fixed the TestHiveMetaStoreChecker issue and uploaded the patch

        i thinkinput20.q is failing because I ran the test on mac. i am running the tests on linux and see how it goes.

        code below seems to produce different results on mac and linux.

        # This script outputs a row of the following format                                                                                                                                           
        # <# of values for the key> <key>_<key>                                                                                                                                                       
        uniq -c | sed "s@^ *@@" | sed "s@\t@_@" | sed "s@ @\t@"
        
        Show
        prasadc Prasad Chakka added a comment - I fixed the TestHiveMetaStoreChecker issue and uploaded the patch i thinkinput20.q is failing because I ran the test on mac. i am running the tests on linux and see how it goes. code below seems to produce different results on mac and linux. # This script outputs a row of the following format # <# of values for the key> <key>_<key> uniq -c | sed "s@^ *@@" | sed "s@\t@_@" | sed "s@ @\t@"
        Hide
        prasadc Prasad Chakka added a comment -

        all tests passed. committed into trunk. it is patching into branch 0.3 properly. will upload a new patch and commit that one.

        Show
        prasadc Prasad Chakka added a comment - all tests passed. committed into trunk. it is patching into branch 0.3 properly. will upload a new patch and commit that one.
        Hide
        prasadc Prasad Chakka added a comment -

        committed to branch 0.30 and attached patch for it.

        Show
        prasadc Prasad Chakka added a comment - committed to branch 0.30 and attached patch for it.
        Hide
        jsensarma Joydeep Sen Sarma added a comment -

        looks like the new tests and outputs did not get added. svn add?

        Show
        jsensarma Joydeep Sen Sarma added a comment - looks like the new tests and outputs did not get added. svn add?
        Hide
        prasadc Prasad Chakka added a comment -

        tests that were not included in the previous patch

        Show
        prasadc Prasad Chakka added a comment - tests that were not included in the previous patch
        Hide
        namit Namit Jain added a comment -

        The tests external1.q and external2.q are failing in 0.3 branch

        Show
        namit Namit Jain added a comment - The tests external1.q and external2.q are failing in 0.3 branch
        Hide
        jsensarma Joydeep Sen Sarma added a comment -

        there seems to have been a merge problem with 0.3. Warehouse.java

        trunk:
        return f.getFileSystem(conf);

        0.3:
        return whRoot.getFileSystem(conf);

        also - the changes to clidriver to optionally proceed in spite of errors are missing. (on the bright side - the test is good )

        Actually - i am ok reverting these changes in 0.3. i don't think it makes sense to get 467 into it (too many changes) and without it - this will not help that much.

        Show
        jsensarma Joydeep Sen Sarma added a comment - there seems to have been a merge problem with 0.3. Warehouse.java trunk: return f.getFileSystem(conf); 0.3: return whRoot.getFileSystem(conf); also - the changes to clidriver to optionally proceed in spite of errors are missing. (on the bright side - the test is good ) Actually - i am ok reverting these changes in 0.3. i don't think it makes sense to get 467 into it (too many changes) and without it - this will not help that much.
        Hide
        prasadc Prasad Chakka added a comment -

        sorry, patching went through fine with some offset issues. so i didn't really look into code that much. but will check into 0.3 after fixing it.

        Show
        prasadc Prasad Chakka added a comment - sorry, patching went through fine with some offset issues. so i didn't really look into code that much. but will check into 0.3 after fixing it.
        Hide
        prasadc Prasad Chakka added a comment -

        merged properly and checked in (only for branch 0.3), trunk was fine.

        Show
        prasadc Prasad Chakka added a comment - merged properly and checked in (only for branch 0.3), trunk was fine.

          People

          • Assignee:
            jsensarma Joydeep Sen Sarma
            Reporter:
            jsensarma Joydeep Sen Sarma
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development