Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
0.94.24, 0.99.1, 0.98.6.1
-
Reviewed
Description
Currently table descriptors and tables are cached once they are accessed for the first time. Next calls to the master only require a trip to HDFS to lookup the modified time in order to reload the table descriptors if modified. However in clusters with a large number of tables or concurrent clients and this can be too aggressive to HDFS and the master causing contention to process other requests. A simple solution is to have a TTL based cached for FSTableDescriptors#getAll() and FSTableDescriptors#TableDescriptorAndModtime() that can allow the master to process those calls faster without causing contention without having to perform a trip to HDFS for every call. to listtables() or getTableDescriptor()
Attachments
Attachments
Issue Links
- links to