[GOBBLIN-531] Gobblin AWS Worker cannot start because of state store type and uri mismatch

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.12.0
• Component/s: gobblin-aws
• Labels: aws helix
• Assignee: Hung Tran
• Reporter: Joel Baranick
• Created: 12/Jul/18 16:46
• Updated: 13/Jul/18 13:34

Description

Something has changed from 0.10.0 to 0.12.0 which causes the StateStores class to be instantiated with a state.store.fs.uri which is mismatched with the state.store.type.  

The problem seems to be from: GobblinTaskRunner.java#L250

It create a new Config for like: 

 

Config stateStoreJobConfig = ConfigUtils.propertiesToConfig(properties)
  .withValue(ConfigurationKeys.STATE_STORE_FS_URI_KEY,
    ConfigValueFactory.fromAnyRef(rootPathUri.toString()));

Compare this to: GobblinHelixJobLauncher.java#L156

 

It creates a new Config like:

 

Config stateStoreJobConfig = ConfigUtils.propertiesToConfig(jobProps)    .withValue(ConfigurationKeys.STATE_STORE_FS_URI_KEY, ConfigValueFactory.fromAnyRef( new URI(appWorkDir.toUri().getScheme(), null, appWorkDir.toUri().getHost(), appWorkDir.toUri().getPort(), null, null, null).toString()));

The following screenshot shows the callstack and the overridden value.