-
Notifications
You must be signed in to change notification settings - Fork 235
A sample that demonstrate how to deploy Custom config rules that created with RDK via ADF pipelines #451
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
A sample that demonstrate how to deploy Custom config rules that created with RDK via ADF pipelines #451
Changes from all commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
0276cd5
Added RDk sample
nadeen-aws 92c5e2a
Fixed typo
nadeen-aws 6d4d4ff
Merge branch 'master' into master
ntwobike d3f6cb5
Fixed typo in the Readme
nadeen-aws 2e03f7e
Merge branch 'master' of github.com:ntwobike/aws-deployment-framework
nadeen-aws ade9c34
Added new lines and removed unwanted comments
nadeen-aws 4f2e096
Removed commented tests
nadeen-aws b229f3e
Simplified the pipeline definition into one and readme update
nadeen-aws 5aabbc6
Review fixes
nadeen-aws ff7ae1e
Added utf-8 encoding on file opening; review fixes
nadeen-aws 8140817
lambda_helper.py functions prefixed by 2 empty lines; review fixes
nadeen-aws File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| *.swp | ||
| package-lock.json | ||
| __pycache__ | ||
| .pytest_cache | ||
| .venv | ||
| *.egg-info | ||
|
|
||
| # CDK asset staging directory | ||
| .cdk.staging | ||
| cdk.out |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,90 @@ | ||
| # Sample RDK Rules pipeline | ||
| This setup will allow you to deploy custom config rules created by the RDK via ADF pipeline. | ||
|
|
||
| ## Architecture | ||
|  | ||
| * As a first step it requires to have a Source code repository to store our code. In this pattern we are using CodeCommit repository. This repository created by as a part of the pipeline definition in the ADF's deployment_map.yml. Example of the pipeline definition is in the ADF setup section. | ||
| * ADF pipeline definition creates a pipeline that will deploy Lambda function(s) into the compliance account and Custom Config rule(s) to Target accounts. | ||
| * When a Custom Config rule get pushed into the CodeCommit repository; | ||
| - CodeBuild will find the RDK rule(s) recursively in the `config-rules` directory then zip each single rule one by one and upload into ADF bucket. Buildspec is utilising a helper script called lambda_helper.py to achieve this task. ADF populates bucket names into SSM Parameter store on the Installation. lambda_helper.py fetches the bucket name from the SSM Parameter Store. Parameter name looks like /cross_region/s3_regional_bucket/{region}. | ||
| - Then CodeBuild will generate 2 CloudFormation templates one for Lambda function(s) deployment and other for the Custom Config rule(s) deployment. | ||
|
|
||
| * When a Lambda function get invokes by a Target account Custom config rule; it will assume the Config role in Target account then put config Evaluations into the Target account's Config rule. | ||
|
|
||
| ### ADF setup | ||
| Sample pipeline definition looks like below: | ||
|
|
||
| ``` | ||
| - name: custom-config-rules-pipeline ## repo name | ||
| default_providers: | ||
| source: | ||
| provider: codecommit | ||
| properties: | ||
| account_id: <compliance-account-id> | ||
| build: | ||
| provider: codebuild | ||
| properties: | ||
| image: "STANDARD_5_0" | ||
| deploy: | ||
| provider: cloudformation | ||
| targets: | ||
| - name: LambdaDeployment | ||
| regions: <regions> | ||
| target: <compliance-account-id> | ||
| properties: | ||
| template_filename: "template-lambda.json" | ||
| - name: ConfigRulesDeployment | ||
| regions: <regions> | ||
| target: | ||
| - <target-accounts-to-deploy-custom-config-rules> | ||
| properties: | ||
| template_filename: "template-config-rules.json" | ||
| ``` | ||
|
|
||
|
|
||
| ## Development setup | ||
| After you clone the repo following file/folder structure will be there; | ||
| | Name | Purpose | | ||
| |---|---| | ||
| | config-rules | This folder will contain all the custom config rules created by `rdk create ...`. Make sure to setup correct details in the `parameters.json` file(ex: SourceRuntime) | | ||
| | params | Contains parameters we need for the generated CloudFormation templates. You must set the account id of the Compliance account in `LambdaAccountId` and Target accounts Config role arn as a pattern in `ConfigRoleArnToAssume`. These will be used as parameters when it deploys config-rule into Target accounts to refer Lambda function from the Compliance account. [Refer this link](https://github.com/awslabs/aws-deployment-framework/blob/master/docs/user-guide.md#cloudformation-parameters-and-tagging) | | ||
| | templates | This folder contains all the cloudformation template pieces that required to build cfn template for the lambda function deployment. | | ||
| | buildspec.yml | Buildspec file to generate Cloudformation templates for the Lambda and Custom Config rules | | ||
| | lambda_helper.py | This is the helper file that pack and upload the lambda code recursively in the config-rules folder | | ||
| | requirements.txt| Requirements for the lambda_helper.py script.| | ||
|
|
||
| ## Lambda function implementation requirements | ||
| In Lambda functions when you want to refer boto3 client or resource make sure | ||
| - Set `ASSUME_ROLE_MODE` constant to `True` | ||
| - Use `get_client` method for client. | ||
| - Duplicate `get_client` and create the `get_resource` method. | ||
|
|
||
| ``` | ||
| def get_resource(service, event, region=None): | ||
| """Return the service boto resource. It should be used instead of directly calling the resource. | ||
|
|
||
| Keyword arguments: | ||
| service -- the service name used for calling the boto.resource() | ||
| event -- the event variable given in the lambda handler | ||
| region -- the region where the resource is called (default: None) | ||
| """ | ||
| if not ASSUME_ROLE_MODE: | ||
| return boto3.resource(service, region) | ||
| credentials = get_assume_role_credentials(get_execution_role_arn(event), region) | ||
| return boto3.resource(service, aws_access_key_id=credentials['AccessKeyId'], | ||
| aws_secret_access_key=credentials['SecretAccessKey'], | ||
| aws_session_token=credentials['SessionToken'], | ||
| region_name=region | ||
| ) | ||
| ``` | ||
| These methods use STS and config payload to assume the IAM role in the target account. If not lambda execution will be failed. | ||
|
|
||
|
|
||
| [More info](https://aws.amazon.com/blogs/mt/aws-config-rdk-multi-account-and-multi-region-deployment/) | ||
|
|
||
| ## Prerequisites/ Important bits | ||
| - This solution does not setup config or config recorder. | ||
| - When this solution deploys the config-rule to a target account; it expectes config is enabled in the target account. | ||
| - Each target account's config role should be able assume by `<account-that-has-the-lambda-function>` to put evaluations into each target account's config. AKA config role in target account(2222222222) should have the lambda-function-account-id(1111111111) as trusted entity as below. | ||
|
|
||
|  | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| version: 0.2 | ||
| phases: | ||
| install: | ||
| runtime-versions: | ||
| python: 3.8 | ||
| nodejs: 16 | ||
| commands: | ||
| - aws s3 cp s3://$S3_BUCKET_NAME/adf-build/ adf-build/ --recursive --quiet | ||
| - pip install -r adf-build/requirements.txt -q | ||
| - python adf-build/generate_params.py | ||
|
|
||
| build: | ||
| commands: | ||
| - pip install rdk | ||
| - cd config-rules | ||
| - rdk create-rule-template --rules-only -a -o ../template-config-rules.json | ||
| - cd .. | ||
| - pip install -r requirements.txt | ||
| - python lambda_helper.py --template-name template-lambda.json | ||
|
|
||
| artifacts: | ||
| files: '**/*' |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.