Details
-
Bug
-
Status: Resolved
-
Critical
-
Resolution: Fixed
-
0.8-incubating
-
None
-
None
Description
I created a type File.v1 and then another type FileSet.v1 which contains an array of File.v1 type instances. The creation was using the AtlasClient.createType(TypeDef) API. This returned back successfully. However, post this Atlas server went into a funny state where simple DSL queries like hive_table started failing. Looking the application logs, noticed this error post the registration of the FileSet.v1 type:
2016-12-20 13:46:53,960 INFO - [pool-2-thread-4 - 8eaf688d-9cd4-46dc-92da-d84991512e9b:] ~ TypeSystem reset invoked by TypeRegistry changes (DefaultMetadataService:749) .... Many types restored and then .... 2016-12-20 13:46:54,266 ERROR - [pool-2-thread-4 - 8eaf688d-9cd4-46dc-92da-d84991512e9b:] ~ Failed to restore type-system after TypeRegistry changes (DefaultMetadataService:759) org.apache.atlas.typesystem.exception.TypeNotFoundException: Unknown datatype: array<HdfsFile.v1> at org.apache.atlas.typesystem.types.TypeSystem.getDataType(TypeSystem.java:192) at org.apache.atlas.typesystem.types.TypeSystem$TransientTypeSystem.getDataType(TypeSystem.java:669) at org.apache.atlas.typesystem.types.AttributeInfo.<init>(AttributeInfo.java:46) at org.apache.atlas.typesystem.types.TypeSystem$TransientTypeSystem.constructAttributeInfo(TypeSystem.java:496) at org.apache.atlas.typesystem.types.TypeSystem$TransientTypeSystem.constructHierarchicalType(TypeSystem.java:540) at org.apache.atlas.typesystem.types.TypeSystem$TransientTypeSystem.orderAndConstructTypes(TypeSystem.java:583) at org.apache.atlas.typesystem.types.TypeSystem$TransientTypeSystem.verifyTypes(TypeSystem.java:709) at org.apache.atlas.typesystem.types.TypeSystem.createTransientTypeSystem(TypeSystem.java:346) at org.apache.atlas.services.DefaultMetadataService.onChange(DefaultMetadataService.java:753) at org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore$TypeRegistryUpdateHook.notifyListeners(AtlasTypeDefGraphStore.java:940) at org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore$TypeRegistryUpdateHook.onComplete(AtlasTypeDefGraphStore.java:924)
DSL failures post this error had this trace:
2016-12-20 13:49:55,398 ERROR - [pool-2-thread-8 - 3d671edf-9dce-4656-a5e3-04e3d3bd3ade:] ~ Unable to get entity list for dslQuery hive_table (MetadataDiscoveryResource:141)
org.apache.atlas.discovery.DiscoveryException: Invalid expression : hive_table
at org.apache.atlas.discovery.graph.GraphBackedDiscoveryService.evaluate(GraphBackedDiscoveryService.java:139)
at org.apache.atlas.discovery.graph.GraphBackedDiscoveryService.searchByDSL(GraphBackedDiscoveryService.java:124)
at org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:51)
at org.apache.atlas.web.resources.MetadataDiscoveryResource.searchUsingQueryDSL(MetadataDiscoveryResource.java:135)
at sun.reflect.GeneratedMethodAccessor41.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
Restarting the server caused more issues because the types restoration failed as in the error in the first stack trace, which caused failures in initializations and so, effectively, the server did not come up. The workaround was the delete the HBase titan table and start over again.