[AURORA-1476] TaskConfig dedupe routine does not handle null fields properly

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 0.10.0
• Component/s: Scheduler
• Assignee: Bill Farner
• Reporter: Bill Farner
• Created: 02/Sep/15 22:01
• Updated: 24/Mar/16 17:09
• Resolved: 11/Sep/15 03:50

Description

When myBatis reads relations, it will create an empty collection when no relations exist (as opposed to leaving the fields null). In TaskConfigManager.insert, we try to re-use a relation that has already been inserted to save space. However, if a relation is null, it will be read back as an empty collection and considered non-equal. This can in turn cause an IllegalArgumentException:

        at com.google.common.collect.ImmutableMap.checkNoConflict(ImmutableMap.java:150)
        at com.google.common.collect.RegularImmutableMap.checkNoConflictInBucket(RegularImmutableMap.java:104)
        at com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:70)
        at com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:254)
        at com.google.common.collect.Maps.uniqueIndex(Maps.java:1166)
        at com.google.common.collect.Maps.uniqueIndex(Maps.java:1140)
        at org.apache.aurora.scheduler.storage.db.TaskConfigManager.getConfigRow(TaskConfigManager.java:47)
        at org.apache.aurora.scheduler.storage.db.TaskConfigManager.insert(TaskConfigManager.java:61)
        at org.apache.aurora.scheduler.storage.db.DbJobUpdateStore.saveJobUpdate(DbJobUpdateStore.java:110)
        at org.apache.aurora.scheduler.storage.log.SnapshotStoreImpl$7.restoreFromSnapshot(SnapshotStoreImpl.java:211)
        at org.apache.aurora.scheduler.storage.log.SnapshotStoreImpl$9.execute(SnapshotStoreImpl.java:278)
        at org.apache.aurora.scheduler.storage.Storage$MutateWork$NoResult.apply(Storage.java:137)
        at org.apache.aurora.scheduler.storage.Storage$MutateWork$NoResult.apply(Storage.java:132)
        at org.apache.aurora.scheduler.storage.db.DbStorage.transactionedWrite(DbStorage.java:152)
        at org.mybatis.guice.transactional.TransactionalMethodInterceptor.invoke(TransactionalMethodInterceptor.java:101)
        at org.apache.aurora.scheduler.storage.db.DbStorage.write(DbStorage.java:165)
        at org.apache.aurora.scheduler.storage.log.SnapshotStoreImpl.applySnapshot(SnapshotStoreImpl.java:272)
        at org.apache.aurora.scheduler.storage.log.SnapshotStoreImpl.applySnapshot(SnapshotStoreImpl.java:63)
        at org.apache.aurora.scheduler.storage.backup.TemporaryStorage$TemporaryStorageFactory.apply(TemporaryStorage.java:73)
        at org.apache.aurora.scheduler.storage.backup.TemporaryStorage$TemporaryStorageFactory.apply(TemporaryStorage.java:66)
        at org.apache.aurora.scheduler.storage.backup.Recovery$RecoveryImpl.stage(Recovery.java:151)
        at org.apache.aurora.scheduler.thrift.SchedulerThriftInterface.stageRecovery(SchedulerThriftInterface.java:759)
        at org.apache.aurora.scheduler.thrift.aop.ThriftStatsExporterInterceptor.invoke(ThriftStatsExporterInterceptor.java:47)
        at org.apache.aurora.scheduler.thrift.aop.FeatureToggleInterceptor.invoke(FeatureToggleInterceptor.java:38)
        at org.apache.aurora.scheduler.thrift.aop.UserCapabilityInterceptor.invoke(UserCapabilityInterceptor.java:92)
        at org.apache.aurora.scheduler.thrift.aop.LoggingInterceptor.invoke(LoggingInterceptor.java:101)
        at org.apache.aurora.scheduler.thrift.aop.ServerInfoInterceptor.invoke(ServerInfoInterceptor.java:30)
        at org.apache.aurora.scheduler.http.api.security.ShiroAuthorizingInterceptor.invoke(ShiroAuthorizingInterceptor.java:86)
        at org.apache.aurora.scheduler.http.api.security.ShiroAuthenticatingThriftInterceptor.invoke(ShiroAuthenticatingThriftInterceptor.java:55)
        at org.apache.aurora.gen.AuroraAdmin$Processor$stageRecovery.getResult(AuroraAdmin.java:1236)
        at org.apache.aurora.gen.AuroraAdmin$Processor$stageRecovery.getResult(AuroraAdmin.java:1221)

Issue Links

• relates to: AURORA-1650 I* entity objects have incorrect isSet* methods for collection types
• relates to: AURORA-1651 I* entity objects are lossy for primitive isSet-ness
• relates to: AURORA-647 Don't store serialized thrift objects in the database