[HADOOP-13960] Initialize DynamoDBMetadataStore without associated S3AFileSystem

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: HADOOP-13345
• Component/s: fs/s3
• Target Version/s: HADOOP-13345
• Assignee: Mingliang Liu
• Reporter: Mingliang Liu
• Created: 09/Jan/17 00:23
• Updated: 12/Jan/17 13:19
• Resolved: 09/Jan/17 08:50

Description

Per the discussion in HADOOP-13650, it's helpful to initialize a DynamoDBMetadataStore object without S3AFileSystem. In the current code, we can achieve this via DynamoDBMetadataStore#initialize(Configuration). However, users still have to provide the associated S3AFileSystem URI in the configuration, by means of either setting the fs.defaultFS in configuration file or -fs s3://bucket command line parameter. Setting the default FileSystem in configuration seems not necessary as the command line is to manipulate metadata store, e.g. command line tools on an existing HDFS cluster.

This JIRA is to track the effort of initializing a DynamoDBMetadataStore without associating any S3 buckets, so that S3AFileSystem is not needed. Users have to specify in configuration the DynamoDB endpoints (for region) and table name along with credentials, AWS client settings.

See eddyxu and liuml07's comments in HADOOP-13650 for more details.