Hive
  1. Hive
  2. HIVE-2526

"lastAceesTime" is always zero when executed through "describe extended <table_name>" unlike "show table extende like <table_name>" where lastAccessTime is updated.

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: 0.9.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Environment:

      Linux : SuSE 11 SP1

      Description

      When the table is accessed(load),lastAccessTime is displaying updated accessTime in

      "show table extended like <table_name>".But "describe extended <table_name>" is always displaying zero.

      1. HIVE-2526.patch
        21 kB
        Priyadarshini

        Activity

        Hide
        Priyadarshini added a comment -

        Thanks for the comments Namit..

        In org.apache.hadoop.hive.metastore.api.Table, the field lastAccessTime has the datatype as int.
        This field should be of type long for capturing the time information. So thrift side changes has to be done for changing the type.
        Also "show table extended like <tname>" needs to be changed where it dynamically calculates the lastAccessTime from HDFS.
        Now lastAccessTime has to be updated into metadata also.

        Show
        Priyadarshini added a comment - Thanks for the comments Namit.. In org.apache.hadoop.hive.metastore.api.Table, the field lastAccessTime has the datatype as int. This field should be of type long for capturing the time information. So thrift side changes has to be done for changing the type. Also "show table extended like <tname>" needs to be changed where it dynamically calculates the lastAccessTime from HDFS. Now lastAccessTime has to be updated into metadata also.
        Hide
        Namit Jain added a comment -

        If we want to maintain the last access time, it should be at the metadata level (in the metastore),
        and not computed dynamically.

        Also, if you want to maintain the last access time, it should be done for both hive access and metastore access.

        Show
        Namit Jain added a comment - If we want to maintain the last access time, it should be at the metadata level (in the metastore), and not computed dynamically. Also, if you want to maintain the last access time, it should be done for both hive access and metastore access.
        Hide
        Priyadarshini added a comment -

        When "describe extended <tname>" is executed, lastAccessTime is always zero because its not gtting updated.
        Now we have updated the lastAccessTime using the HDFS FileStatus api.

        Show
        Priyadarshini added a comment - When "describe extended <tname>" is executed, lastAccessTime is always zero because its not gtting updated. Now we have updated the lastAccessTime using the HDFS FileStatus api.
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/2896/
        -----------------------------------------------------------

        Review request for hive.

        Summary
        -------

        When "describe extended <tname>" is executed, lastAccessTime is always zero because its not gtting updated.
        Now we have updated the lastAccessTime using the HDFS FileStatus api.

        This addresses bug HIVE-2526.
        https://issues.apache.org/jira/browse/HIVE-2526

        Diffs


        trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1202581
        trunk/ql/src/test/results/clientpositive/ppr_pushdown.q.out 1202581

        Diff: https://reviews.apache.org/r/2896/diff

        Testing
        -------

        All existing tests are passed.

        Thanks,

        Priyadarshini

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2896/ ----------------------------------------------------------- Review request for hive. Summary ------- When "describe extended <tname>" is executed, lastAccessTime is always zero because its not gtting updated. Now we have updated the lastAccessTime using the HDFS FileStatus api. This addresses bug HIVE-2526 . https://issues.apache.org/jira/browse/HIVE-2526 Diffs trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 1202581 trunk/ql/src/test/results/clientpositive/ppr_pushdown.q.out 1202581 Diff: https://reviews.apache.org/r/2896/diff Testing ------- All existing tests are passed. Thanks, Priyadarshini
        Hide
        Anoop Sam John added a comment -

        As part of meta data for table, there are fields named 'lastAccessTime' and 'createTime'.[ Both are of int type also]. But as of now we are not maintaining any values in these fields and always will be zero only.[ default value].
        When I checked the code DDLTask.writeFileSystemStats () contains code to display the lastaccesstime by getting the last access time from the underlying file system. So in case of command, show table extended like <TN>, this method is getting used and the last access time is getting displayed. In case of command describe exteneded <TN> the last access time is getting dispalyed from
        outStream.writeBytes(tbl.getTTable().toString());
        In org.apache.hadoop.hive.metastore.api.Table, the
        private int lastAccessTime; will always be default value ( 0 ) only and no one updates this also.

        I have 2 points here.
        1. Why we are not making use of the fields in tbls meta data table for storing the last access time, create time?
        2. Why is the dat type for lastAccessTime in org.apache.hadoop.hive.metastore.api.Table is int and not long? In order to capture the access time with values of System.currentTimeMillis() this type to be long. Also pls refer that in case of org.apache.hadoop.fs.FileStatus the access_time is having long data type rather than int.

        Show
        Anoop Sam John added a comment - As part of meta data for table, there are fields named 'lastAccessTime' and 'createTime'.[ Both are of int type also]. But as of now we are not maintaining any values in these fields and always will be zero only.[ default value]. When I checked the code DDLTask.writeFileSystemStats () contains code to display the lastaccesstime by getting the last access time from the underlying file system. So in case of command, show table extended like <TN>, this method is getting used and the last access time is getting displayed. In case of command describe exteneded <TN> the last access time is getting dispalyed from outStream.writeBytes(tbl.getTTable().toString()); In org.apache.hadoop.hive.metastore.api.Table, the private int lastAccessTime; will always be default value ( 0 ) only and no one updates this also. I have 2 points here. 1. Why we are not making use of the fields in tbls meta data table for storing the last access time, create time? 2. Why is the dat type for lastAccessTime in org.apache.hadoop.hive.metastore.api.Table is int and not long? In order to capture the access time with values of System.currentTimeMillis() this type to be long. Also pls refer that in case of org.apache.hadoop.fs.FileStatus the access_time is having long data type rather than int.

          People

          • Assignee:
            Priyadarshini
            Reporter:
            Rohith
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:

              Development