[JCLOUDS-1120] Security groups don't work with non-default VPCs

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.0.0
• Component/s: jclouds-compute
• Labels: aws-ec2
• Assignee: Unassigned
• Reporter: Richard Janík
• Created: 01/Jun/16 06:37
• Updated: 15/May/17 12:00

Description

As per discussion on jclouds users list.

I've been trying to create nodes in EC2 in a non-default VPC, setting only subnetId on TemplateOptions (no security groups). Like so:

  AWSEC2TemplateOptions templateOptions = new AWSEC2TemplateOptions();
  ... some more template options setup that shouldn't be relevant
  templateOptions.subnetId("subnet-mysubnet");
  Template template = templateBuilder.
                .hardwareId(instanceType)
                .locationId(region)
                .imageId(imageId)
                .options(templateOptions)
                .build();
  Iterables.getOnlyElement(computeService.createNodesInGroup("cloudts-rjanik", 1, template));

I've been running into problems with that and also tried setting a security group via TemplateOptions.securityGroupIds(), but it seems the issue persists. The stacktrace is here:

Exception in thread "main" com.google.common.util.concurrent.UncheckedExecutionException: org.jclouds.aws.AWSResponseException: request POST https://ec2.us-east-1.amazonaws.com/ HTTP/1.1 failed with code 400, error: AWSError{requestId='54c91571-7082-4c3e-9ec2-fce52ebceb8e', requestToken='null', code='InvalidParameterValue', message='Invalid value 'jclouds#cloudts-rjanik' for groupName. You may not reference Amazon VPC security groups by name. Please use the corresponding id for this operation.', context='{Response=, Errors=}'}
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
	at com.google.common.cache.LocalCache.get(LocalCache.java:3934)
	at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3938)
	at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4821)
	at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4827)
	at org.jclouds.ec2.compute.strategy.CreateKeyPairAndSecurityGroupsAsNeededAndReturnRunOptions.getSecurityGroupsForTagAndOptions(CreateKeyPairAndSecurityGroupsAsNeededAndReturnRunOptions.java:181)
	at org.jclouds.aws.ec2.compute.strategy.CreateKeyPairPlacementAndSecurityGroupsAsNeededAndReturnRunOptions.addSecurityGroups(CreateKeyPairPlacementAndSecurityGroupsAsNeededAndReturnRunOptions.java:191)
	at org.jclouds.ec2.compute.strategy.CreateKeyPairAndSecurityGroupsAsNeededAndReturnRunOptions.execute(CreateKeyPairAndSecurityGroupsAsNeededAndReturnRunOptions.java:80)
	at org.jclouds.aws.ec2.compute.strategy.CreateKeyPairPlacementAndSecurityGroupsAsNeededAndReturnRunOptions.execute(CreateKeyPairPlacementAndSecurityGroupsAsNeededAndReturnRunOptions.java:88)
	at org.jclouds.aws.ec2.compute.strategy.CreateKeyPairPlacementAndSecurityGroupsAsNeededAndReturnRunOptions.execute(CreateKeyPairPlacementAndSecurityGroupsAsNeededAndReturnRunOptions.java:55)
	at org.jclouds.ec2.compute.strategy.EC2CreateNodesInGroupThenAddToSet.createKeyPairAndSecurityGroupsAsNeededThenRunInstances(EC2CreateNodesInGroupThenAddToSet.java:213)
	at org.jclouds.ec2.compute.strategy.EC2CreateNodesInGroupThenAddToSet.runInstancesAndWarnOnInvisible(EC2CreateNodesInGroupThenAddToSet.java:151)
	at org.jclouds.ec2.compute.strategy.EC2CreateNodesInGroupThenAddToSet.execute(EC2CreateNodesInGroupThenAddToSet.java:132)
	at org.jclouds.compute.internal.BaseComputeService.createNodesInGroup(BaseComputeService.java:217)
	at org.jclouds.ec2.compute.EC2ComputeService.createNodesInGroup(EC2ComputeService.java:148)
    at ... our stacktrace
Caused by: org.jclouds.aws.AWSResponseException: request POST https://ec2.us-east-1.amazonaws.com/ HTTP/1.1 failed with code 400, error: AWSError{requestId='54c91571-7082-4c3e-9ec2-fce52ebceb8e', requestToken='null', code='InvalidParameterValue', message='Invalid value 'jclouds#cloudts-rjanik' for groupName. You may not reference Amazon VPC security groups by name. Please use the corresponding id for this operation.', context='{Response=, Errors=}'}
	at org.jclouds.aws.handlers.ParseAWSErrorFromXmlContent.handleError(ParseAWSErrorFromXmlContent.java:75)
	at org.jclouds.http.handlers.DelegatingErrorHandler.handleError(DelegatingErrorHandler.java:65)
	at org.jclouds.http.internal.BaseHttpCommandExecutorService.shouldContinue(BaseHttpCommandExecutorService.java:136)
	at org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:105)
	at org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:90)
	at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:73)
	at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:44)
	at org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117)
	at com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:87)
	at com.sun.proxy.$Proxy57.describeSecurityGroupsInRegion(Unknown Source)
	at org.jclouds.ec2.compute.functions.EC2SecurityGroupIdFromName.apply(EC2SecurityGroupIdFromName.java:46)
	at org.jclouds.ec2.compute.functions.EC2SecurityGroupIdFromName.apply(EC2SecurityGroupIdFromName.java:30)
	at org.jclouds.aws.ec2.compute.loaders.AWSEC2CreateSecurityGroupIfNeeded.createSecurityGroupInRegion(AWSEC2CreateSecurityGroupIfNeeded.java:130)
	at org.jclouds.aws.ec2.compute.loaders.AWSEC2CreateSecurityGroupIfNeeded.load(AWSEC2CreateSecurityGroupIfNeeded.java:73)
	at org.jclouds.aws.ec2.compute.loaders.AWSEC2CreateSecurityGroupIfNeeded.load(AWSEC2CreateSecurityGroupIfNeeded.java:46)
	at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3524)
	at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2317)
	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2280)
	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2195)
	... 25 more

I found out that JClouds always (even if using non-default VPC and even if security groups have been provided) attempts to create some kind of marker security group in

CreateKeyPairAndSecurityGroupsAsNeededAndReturnRunOptions.getSecurityGroupsForTagAndOptions

When the security group is created, it tries to resolve the name into ID by calling

Iterables.getOnlyElement(api.getSecurityGroupApi().get().describeSecurityGroupsInRegion(region, name), null).getId();

That fails for security groups in non-default VPC, because to use the DescribeSecurityGroups action there, IDs have to be provided instead of names, see http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeSecurityGroups.html.