I0522 13:28:39.918550 22029 resource-broker.cc:686] Received allocated resource for reservation id: b2486107e53609c7:98be50a18fd4fe97 I0522 13:28:39.918629 21449 resource-broker.cc:420] Getting allocated resource for reservation id b2486107e53609c7:98be50a18fd4fe97 and location localhost:31000 I0522 13:28:39.918653 21449 resource-broker.cc:420] Getting allocated resource for reservation id b2486107e53609c7:98be50a18fd4fe97 and location localhost:31001 I0522 13:28:39.918666 21449 resource-broker.cc:420] Getting allocated resource for reservation id b2486107e53609c7:98be50a18fd4fe97 and location localhost:31002 I0522 13:28:39.918689 21449 resource-broker.cc:596] Fulfilled reservation with id: b2486107e53609c7:98be50a18fd4fe97 Allocated Resource(reservation_id=b2486107e53609c7:98be50a18fd4fe97 client_resource_id=8145877aeb1ccc0a:6cffa68fd963c49d rm_resource_id=container_1432320689445_0003_01_000005,container_1432320689445_0003_01_000006,container_1432320689445_0003_01_000004 v_cpu_cores=2 memory_mb=1024 location=localhost:31001) I0522 13:28:39.919626 29358 query-resource-mgr.cc:167] Starting Vcore acquisition for: b2486107e53609c7:98be50a18fd4fe97 Allocated Resource(reservation_id=b2486107e53609c7:98be50a18fd4fe97 client_resource_id=8145877aeb1ccc0a:6cffa68fd963c49d rm_resource_id=container_1432320689445_0003_01_000005,container_1432320689445_0003_01_000006,container_1432320689445_0003_01_000004 v_cpu_cores=2 memory_mb=1024 location=localhost:31001) I0522 13:28:40.388375 21449 resource-broker.cc:633] Releasing all resources for reservation: b2486107e53609c7:98be50a18fd4fe97 I0522 13:28:40.388397 21449 resource-broker.cc:624] Releasing 3.00 GB and 6 cores for b2486107e53609c7:98be50a18fd4fe97 I0522 13:28:40.441159 29358 query-resource-mgr.cc:180] Expanding VCore allocation: b2486107e53609c7:98be50a18fd4fe97 I0522 13:28:40.567498 29358 status.cc:111] com.cloudera.llama.util.LlamaException: UNKNOWN_RESERVATION_FOR_EXPANSION - Unknown reservation 'b2486107e53609c7:98be50a18fd4fe97' for expansion, com.cloudera.llama.util.LlamaException: UNKNOWN_RESERVATION_FOR_EXPANSION - Unknown reservation 'b2486107e53609c7:98be50a18fd4fe97' for expansion, at com.cloudera.llama.am.impl.ExpansionReservationsLlamaAM.expand(ExpansionReservationsLlamaAM.java:249), at com.cloudera.llama.am.impl.APIContractLlamaAM.expand(APIContractLlamaAM.java:155), at com.cloudera.llama.am.LlamaAMServiceImpl.Expand(LlamaAMServiceImpl.java:205), at com.cloudera.llama.am.MetricLlamaAMService.Expand(MetricLlamaAMService.java:158), at com.cloudera.llama.thrift.LlamaAMService$Processor$Expand.getResult(LlamaAMService.java:532), at com.cloudera.llama.thrift.LlamaAMService$Processor$Expand.getResult(LlamaAMService.java:517), at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39), at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39), at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206), at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145), at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615), at java.lang.Thread.run(Thread.java:745) I0522 13:28:40.567524 29358 query-resource-mgr.cc:190] Fragment finished during Expand(): b2486107e53609c7:98be50a18fd4fe97 I0522 13:28:40.567535 29358 query-resource-mgr.cc:226] Leaving VCore acquisition thread: b2486107e53609c7:98be50a18fd4fe97 ------------ llama logs ------------ 2015-05-22 13:28:39,454 TRACE com.cloudera.llama.am.impl.APIContractLlamaAM: reserve(PlacedReservation[reservationId:null status:null placedOn:0 allocatedOn:0 expansionOf:null handle:211e27ba35d74365:8454169ebb47577a user:jenkins queue:root.jenkins gang:true queued:false resources:[PlacedResource[resourceId:e84d51dc39e70d80:4de15b21cfaed192 status:null locationAsk:localhost:31100 localityAsk:MUST cpuVCoresAsk:2 memoryMbsAsk:170 placedOn:0 reservationId:null handle:null user:null queue:null allocatedOn:0 location:null cpuVCores:0 memoryMbs:0 rmResourceId:null], PlacedResource[resourceId:8145877aeb1ccc0a:6cffa68fd963c49d status:null locationAsk:localhost:31101 localityAsk:MUST cpuVCoresAsk:2 memoryMbsAsk:170 placedOn:0 reservationId:null handle:null user:null queue:null allocatedOn:0 location:null cpuVCores:0 memoryMbs:0 rmResourceId:null], PlacedResource[resourceId:6744179d2d9ff25a:c11bd5ec96e4cc89 status:null locationAsk:localhost:31102 localityAsk:MUST cpuVCoresAsk:2 memoryMbsAsk:170 placedOn:0 reservationId:null handle:null user:null queue:null allocatedOn:0 location:null cpuVCores:0 memoryMbs:0 rmResourceId:null]]]): b2486107e53609c7:98be50a18fd4fe97 2015-05-22 13:28:40,388 TRACE com.cloudera.llama.am.impl.APIContractLlamaAM: releaseReservation(b2486107e53609c7:98be50a18fd4fe97, false) 2015-05-22 13:28:40,453 TRACE com.cloudera.llama.am.impl.APIContractLlamaAM: expand(Expansion[expansionOf:b2486107e53609c7:98be50a18fd4fe97 resource:PlacedResource[resourceId:6f4dcdb6cadbed72:f417b5182ddb01a0 status:null locationAsk:localhost:31100 localityAsk:MUST cpuVCoresAsk:1 memoryMbsAsk:0 placedOn:0 reservationId:null handle:null user:null queue:null allocatedOn:0 location:null cpuVCores:0 memoryMbs:0 rmResourceId:null]]): e04748af5832d6d5:71f7724f3cda2980 2015-05-22 13:28:40,453 TRACE com.cloudera.llama.am.impl.APIContractLlamaAM: expand(Expansion[expansionOf:b2486107e53609c7:98be50a18fd4fe97 resource:PlacedResource[resourceId:7a47769b72bbbc41:bf19678d2500eb83 status:null locationAsk:localhost:31100 localityAsk:MUST cpuVCoresAsk:1 memoryMbsAsk:0 placedOn:0 reservationId:null handle:null user:null queue:null allocatedOn:0 location:null cpuVCores:0 memoryMbs:0 rmResourceId:null]]): a34df920cf23172c:8caed08ff8d61393 2015-05-22 13:28:40,459 TRACE com.cloudera.llama.am.impl.APIContractLlamaAM: expand(Expansion[expansionOf:b2486107e53609c7:98be50a18fd4fe97 resource:PlacedResource[resourceId:594a515863c92747:b18392ec7f93d397 status:null locationAsk:localhost:31100 localityAsk:MUST cpuVCoresAsk:1 memoryMbsAsk:0 placedOn:0 reservationId:null handle:null user:null queue:null allocatedOn:0 location:null cpuVCores:0 memoryMbs:0 rmResourceId:null]]): c44f72f05fd824aa:3cc7c34d0094629c