Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions .chloggen/awscloudwatchexporter_external_id.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Use this changelog template to create an entry for release notes.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: enhancement

# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
component: awscloudwatchlogsexporter, awsemfexporter, awsxrayexporter

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Adding external id support when assuming a role for AWS credentials.

# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
issues: [36725]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext: AWS IAM Docs https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_common-scenarios_third-party.html#id_roles_third-party_external-id

# If your change doesn't affect end users or the exported elements of any package,
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
# Optional: The change log or logs in which this entry should be included.
# e.g. '[user]' or '[user, api]'
# Include 'user' if the change is relevant to end users.
# Include 'api' if there is a change to a library API.
# Default: '[user]'
change_logs: [user]
43 changes: 34 additions & 9 deletions exporter/awscloudwatchlogsexporter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,17 +21,19 @@ NOTE: OpenTelemetry Logging support is experimental, hence this exporter is subj

The following settings are required:

- `log_group_name`: The group name of the CloudWatch Logs. If it does not exist it will be created automatically.
- `log_group_name`: The group name of the CloudWatch Logs. If it does not exist it will be created automatically.
- `log_stream_name`: The stream name of the CloudWatch Logs. If it does not exist it will be created automatically.

The following settings can be optionally configured:

- `region`: The AWS region where the log stream is in. Region must be specified if it is not already set in the default credential chain.
- `endpoint`: The CloudWatch Logs service endpoint which the requests are forwarded to. [See the CloudWatch Logs endpoints](https://docs.aws.amazon.com/general/latest/gr/cwl_region.html) for a list.
- `log_retention`: LogRetention is the option to set the log retention policy for only newly created CloudWatch Log Groups. Defaults to Never Expire if not specified or set to 0. Possible values for retention in days are 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, 2192, 2557, 2922, 3288, or 3653.
- `tags`: Tags is the option to set tags for the CloudWatch Log Group. If specified, please add at most 50 tags. Input is a string to string map like so: { 'key': 'value' }. Keys must be between 1-128 characters and follow the regex pattern: `^([\p{L}\p{Z}\p{N}_.:/=+\-@]+)$`(alphanumerics, whitespace, and _.:/=+-!). Values must be between 1-256 characters and follow the regex pattern: `^([\p{L}\p{Z}\p{N}_.:/=+\-@]*)$`(alphanumerics, whitespace, and _.:/=+-!). [Link to tagging restrictions](https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_CreateLogGroup.html#:~:text=Required%3A%20Yes-,tags,-The%20key%2Dvalue)
- `log_retention`: LogRetention is the option to set the log retention policy for only newly created CloudWatch Log Groups. Defaults to Never Expire if not specified or set to 0. Possible values for retention in days are 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, 2192, 2557, 2922, 3288, or 3653.
- `tags`: Tags is the option to set tags for the CloudWatch Log Group. If specified, please add at most 50 tags. Input is a string to string map like so: { 'key': 'value' }. Keys must be between 1-128 characters and follow the regex pattern: `^([\p{L}\p{Z}\p{N}_.:/=+\-@]+)$`(alphanumerics, whitespace, and _.:/=+-!). Values must be between 1-256 characters and follow the regex pattern: `^([\p{L}\p{Z}\p{N}_.:/=+\-@]\*)$`(alphanumerics, whitespace, and \_.:/=+-!). [Link to tagging restrictions](https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_CreateLogGroup.html#:~:text=Required%3A%20Yes-,tags,-The%20key%2Dvalue)
- `raw_log`: Boolean default false. If set to true, only the log message will be exported to CloudWatch Logs. This needs to be set to true for [EMF logs](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format_Specification.html).
- `sending_queue`: [Parameters for the sending queue](https://github.com/open-telemetry/opentelemetry-collector/blob/main/exporter/exporterhelper/README.md), where you can control parallelism and the size of the sending buffer. Obs.: this component will always have a sending queue enabled.
- `role_arn`: IAM role to upload logs to a different account.
- `external_id`: Shared identitier used when assuming an IAM role in an external AWS account. [See AWS IAM Guide](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_common-scenarios_third-party.html#id_roles_third-party_external-id)
- `sending_queue`: [Parameters for the sending queue](https://github.com/open-telemetry/opentelemetry-collector/blob/main/exporter/exporterhelper/README.md), where you can control parallelism and the size of the sending buffer. Obs.: this component will always have a sending queue enabled.
- `num_consumers`: Number of consumers that will consume from the sending queue. This parameter controls how many consumers will consume from the sending queue in parallel.
- `queue_size`: Maximum number of batches kept in memory before dropping; ignored if enabled is false

Expand All @@ -57,15 +59,38 @@ exporters:
region: "us-east-1"
endpoint: "logs.us-east-1.amazonaws.com"
log_retention: 365
tags: { 'sampleKey': 'sampleValue'}
tags: { "sampleKey": "sampleValue" }
```

## Additional Notes
## Additional Notes

- If the log group and/or log stream are specified in an EMF log, that EMF log will be exported to that log group and/or log stream (i.e. ignores the log group and log stream defined in the configuration)
- The log group and log stream will also be created automatically if they do not already exist.
- Example of an EMF log with log group and log stream:
- Example of an EMF log with log group and log stream:

```json
{"_aws":{"Timestamp":1574109732004,"LogGroupName":"Foo", "LogStreamName": "Bar", "CloudWatchMetrics":[{"Namespace":"MyApp","Dimensions":[["Operation"]],"Metrics":[{"Name":"ProcessingLatency","Unit":"Milliseconds","StorageResolution":60}]}]},"Operation":"Aggregator","ProcessingLatency":100}
```
{
"_aws": {
"Timestamp": 1574109732004,
"LogGroupName": "Foo",
"LogStreamName": "Bar",
"CloudWatchMetrics": [
{
"Namespace": "MyApp",
"Dimensions": [["Operation"]],
"Metrics": [
{
"Name": "ProcessingLatency",
"Unit": "Milliseconds",
"StorageResolution": 60
}
]
}
]
},
"Operation": "Aggregator",
"ProcessingLatency": 100
}
```

- Resource ARNs (Amazon Resource Name (ARN) of the AWS resource running the collector) are currently not supported with the CloudWatch Logs Exporter.
1 change: 1 addition & 0 deletions exporter/awsemfexporter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ The following exporter configuration parameters are supported.
| `proxy_address` | Upload Structured Logs to AWS CloudWatch through a proxy. | |
| `region` | Send Structured Logs to AWS CloudWatch in a specific region. If this field is not present in config, environment variable "AWS_REGION" can then be used to set region. | determined by metadata |
| `role_arn` | IAM role to upload segments to a different account. | |
| `external_id` | Shared identitier used when assuming an IAM role in an external AWS account. | |
| `max_retries` | Maximum number of retries before abandoning an attempt to post data. | 1 |
| `dimension_rollup_option` | DimensionRollupOption is the option for metrics dimension rollup. Three options are available: `NoDimensionRollup`, `SingleDimensionRollupOnly` and `ZeroAndSingleDimensionRollup`. The default value is `ZeroAndSingleDimensionRollup`. Enabling feature gate `awsemf.nodimrollupdefault` will set default to `NoDimensionRollup`. |"ZeroAndSingleDimensionRollup" (Enable both zero dimension rollup and single dimension rollup)|
| `resource_to_telemetry_conversion` | "resource_to_telemetry_conversion" is the option for converting resource attributes to telemetry attributes. It has only one config option- `enabled`. For metrics, if `enabled=true`, all the resource attributes will be converted to metric labels by default. See `Resource Attributes to Metric Labels` section below for examples. | `enabled=false` |
Expand Down
1 change: 1 addition & 0 deletions exporter/awsxrayexporter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ comparable AWS X-Ray Daemon configuration values.
| `local_mode` | Local mode to skip EC2 instance metadata check. | false |
| `resource_arn` | Amazon Resource Name (ARN) of the AWS resource running the collector. | |
| `role_arn` | IAM role to upload segments to a different account. | |
| `external_id` | Shared identitier used when assuming an IAM role in an external AWS account. | |
| `indexed_attributes` | List of attribute names to be converted to X-Ray annotations. | |
| `index_all_attributes` | Enable or disable conversion of all OpenTelemetry attributes to X-Ray annotations. | false |
| `aws_log_groups` | List of log group names for CloudWatch. | [] |
Expand Down
2 changes: 2 additions & 0 deletions internal/aws/awsutil/awsconfig.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ type AWSSessionSettings struct {
ResourceARN string `mapstructure:"resource_arn"`
// IAM role to upload segments to a different account.
RoleARN string `mapstructure:"role_arn"`
// External ID to verify third party role assumption
ExternalID string `mapstructure:"external_id"`
}

func CreateDefaultSessionConfig() AWSSessionSettings {
Expand Down
Loading