Skip to content

Add semantic conventions for cloud provider-specific resource attributes #1099

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Nov 25, 2020
Merged

Add semantic conventions for cloud provider-specific resource attributes #1099

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
b158045
proposal for cloud provider-specific resource semantic conventions
Oct 15, 2020
9b6c00d
addressed feedback
Oct 19, 2020
b8d4c0d
changed log attribs to be string arrays
Oct 19, 2020
931024a
pluralized array attribs
Oct 21, 2020
ffb068e
Update specification/resource/semantic_conventions/README.md
willarmiros Oct 22, 2020
c66f5b1
switched cluster to cluster.arn
Oct 28, 2020
8525215
Merge branch 'aws-conventions' of https://github.com/willarmiros/open…
Oct 28, 2020
4024bb0
lowercased launch types
Oct 29, 2020
bcbd99e
Merge branch 'master' into aws-conventions
willarmiros Nov 24, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion semantic_conventions/resource/cloud.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ groups:
brief: 'Amazon Web Services'
- id: Azure
value: 'azure'
brief: 'Amazon Web Services'
brief: 'Microsoft Azure'
Copy link
Contributor

@anuraaga anuraaga Oct 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😄

- id: GCP
value: 'gcp'
brief: 'Google Cloud Platform'
Expand All @@ -37,3 +37,12 @@ groups:
note: >
In AWS, this is called availability-zone.
examples: ['us-central1-a']
- id: infrastructure.service
Copy link
Contributor

@anuraaga anuraaga Oct 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's probably good to separate this into a separate PR since it's broader reaching than AWS-specific conventions.

Copy link
Contributor Author

@willarmiros willarmiros Oct 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok will separate it into a new PR with your feedback included.

type: string
brief: >
Description of cloud resource in use.
note: >
The first entry should generally be a cloud provider, followed by a product
category, then finally a particular piece of compute infrastructure. Each entry
is delimited by a double colon (::).
examples: ['AWS::EC2::Instance', 'Azure::Compute::VM', 'GCP::ComputeEngine::VM']
Copy link
Contributor

@anuraaga anuraaga Oct 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AWS, etc are already in cloud.provider so this should only need something like ec2, aks, cloud run. Of course we'll want insight from a GCP and Azure member for some examples :)

X-Ray uses these extended names but it's the X-Ray exporter that can format the string correctly but not what we'd require in the semantic convention which should just be a precise semantic string.

Copy link
Contributor Author

@willarmiros willarmiros Oct 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, I sorta figured as much. I'll get rid of the delimiters and just specify the compute platform.

34 changes: 34 additions & 0 deletions semantic_conventions/resource/cloud_provider/aws/aws.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
groups:
Copy link
Member

@thisthat thisthat Oct 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems more related to logging than AWS in the general sense.

Copy link
Contributor Author

@willarmiros willarmiros Oct 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you prefer I move this to somewhere in the logs spec? The loggroup and logstream fields are specific terms to AWS CloudWatch, but if there are more generic terms that would apply to other logging backends but still convey the same meaning, I'd be happy to switch the wording.

Copy link
Member

@thisthat thisthat Oct 19, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not familiar with logs, but I guess there will be semantic conventions for it as well!
For the moment, I would suggest changing the id to aws.log to avoid a naming clash with other more general aws semantic conventions.

- id: aws
prefix: aws
brief: >
Resources specific to Amazon Web Services.
attributes:
- id: loggroup.name
type: string
brief: >
The name of the AWS log group an application is writing to.
examples: ['/aws/lambda/my-function', 'opentelemetry-service']
- id: loggroup.arn
type: string
brief: >
The Amazon Resource Name (ARN) of an AWS log group.
examples: ['arn:aws:logs:us-west-1:123456789012:log-group:/aws/my/group:*']
note: >
See the
[log group ARN format documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/iam-access-control-overview-cwl.html#CWL_ARN_Format).
- id: logstream.name
type: string
brief: >
The name of the AWS log stream an application is writing to.
examples: ['logs/main/10838bed-421f-43ef-870a-f43feacbbb5b']
- id: logstream.arn
type: string
brief: >
The ARN of the AWS log stream.
examples: ['arn:aws:logs:us-west-1:123456789012:log-group:/aws/my/group:log-stream:logs/main/10838bed-421f-43ef-870a-f43feacbbb5b']
note: >
See the
[log stream ARN format documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/iam-access-control-overview-cwl.html#CWL_ARN_Format).
One log group can contain several log streams, so this ARN necessarily identifies both a log
group and a log stream.
38 changes: 38 additions & 0 deletions semantic_conventions/resource/cloud_provider/aws/ecs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
groups:
- id: aws.ecs
prefix: aws.ecs
brief: >
Resources used by AWS Elastic Container Service (ECS).
attributes:
- id: container.arn
type: string
brief: >
The ARN of an [ECS container instance](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ECS_instances.html).
examples: ['arn:aws:ecs:us-west-1:123456789123:container/32624152-9086-4f0e-acae-1a75b14fe4d9']
- id: cluster
type: string
brief: >
The name of an [ECS cluster](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/clusters.html).
examples: ['opentelemetry-cluster']
- id: launchtype
type:
allow_custom_values: false
members:
- id: EC2
value: "EC2"
- id: Fargate
value: "Fargate"
brief: >
The [launch type](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/launch_types.html) for an ECS task.
examples: ['EC2', 'Fargate']
- id: task.arn
type: string
brief: >
The ARN of an [ECS task definition](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definitions.html).
examples: ['arn:aws:ecs:us-west-1:123456789123:task/10838bed-421f-43ef-870a-f43feacbbb5b']
- id: task.family
type: string
brief: >
The task definition family this task definition is a member of.
examples: ['opentelemetry-family']

6 changes: 6 additions & 0 deletions specification/resource/semantic_conventions/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -135,3 +135,9 @@ Attributes defining a running environment (e.g. Operating System, Cloud, Data Ce
Version attributes, such as `service.version`, are values of type `string`. They are
the exact version used to identify an artifact. This may be a semantic version, e.g., `1.2.3`, git hash, e.g.,
`8ae73a`, or an arbitrary version string, e.g., `0.1.2.20210101`, whatever was used when building the artifact.

## Provider-Specific Attributes

Attributes that are only applicable to resources from a specific cloud provider. Currently, these
resources can only be defined for providers listed as a valid `cloud.provider` in
[Cloud](./cloud.md). Provider-specific attributes all reside in the `cloud_provider` directory.
Copy link
Member

@thisthat thisthat Oct 16, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add the list of providers for which we have some semantic convention here

Copy link
Contributor Author

@willarmiros willarmiros Oct 16, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I'll include a list of valid cloud providers more explicitly. I'll also include the README in the aws directory :)

5 changes: 4 additions & 1 deletion specification/resource/semantic_conventions/cloud.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,17 @@
| `cloud.account.id` | string | The cloud account ID used to identify different entities. | `opentelemetry` | No |
| `cloud.region` | string | A specific geographical location where different entities can run. | `us-central1` | No |
| `cloud.zone` | string | Zones are a sub set of the region connected through low-latency links. [1] | `us-central1-a` | No |
| `cloud.infrastructure.service` | string | Description of cloud resource in use. [2] | `AWS::EC2::Instance`<br>`Azure::Compute::VM`<br>`GCP::ComputeEngine::VM` | No |

**[1]:** In AWS, this is called availability-zone.

**[2]:** The first entry should generally be a cloud provider, followed by a product category, then finally a particular piece of compute infrastructure. Each entry is delimited by a double colon (::).

`cloud.provider` MUST be one of the following or, if none of the listed values apply, a custom value:

| Value | Description |
|---|---|
| `aws` | Amazon Web Services |
| `azure` | Amazon Web Services |
| `azure` | Microsoft Azure |
| `gcp` | Google Cloud Platform |
<!-- endsemconv -->
18 changes: 18 additions & 0 deletions specification/resource/semantic_conventions/cloud_provider/aws/aws.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# AWS

**Type:** `aws`

**Description:** Resources specific to Amazon Web Services.

<!-- semconv aws -->
| Attribute | Type | Description | Example | Required |
|---|---|---|---|---|
| `aws.loggroup.name` | string | The name of the AWS log group an application is writing to. | `/aws/lambda/my-function`<br>`opentelemetry-service` | No |
| `aws.loggroup.arn` | string | The Amazon Resource Name (ARN) of an AWS log group. [1] | `arn:aws:logs:us-west-1:123456789012:log-group:/aws/my/group:*` | No |
| `aws.logstream.name` | string | The name of the AWS log stream an application is writing to. | `logs/main/10838bed-421f-43ef-870a-f43feacbbb5b` | No |
| `aws.logstream.arn` | string | The ARN of the AWS log stream. [2] | `arn:aws:logs:us-west-1:123456789012:log-group:/aws/my/group:log-stream:logs/main/10838bed-421f-43ef-870a-f43feacbbb5b` | No |

**[1]:** See the [log group ARN format documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/iam-access-control-overview-cwl.html#CWL_ARN_Format).

**[2]:** See the [log stream ARN format documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/iam-access-control-overview-cwl.html#CWL_ARN_Format). One log group can contain several log streams, so this ARN necessarily identifies both a log group and a log stream.
<!-- endsemconv -->
22 changes: 22 additions & 0 deletions specification/resource/semantic_conventions/cloud_provider/aws/ecs.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# AWS ECS

**type:** `aws.ecs`

**Description:** Resources used by AWS Elastic Container Service (ECS).

<!-- semconv aws.ecs -->
| Attribute | Type | Description | Example | Required |
|---|---|---|---|---|
| `aws.ecs.container.arn` | string | The ARN of an [ECS container instance](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ECS_instances.html). | `arn:aws:ecs:us-west-1:123456789123:container/32624152-9086-4f0e-acae-1a75b14fe4d9` | No |
| `aws.ecs.cluster` | string | The name of an [ECS cluster](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/clusters.html). | `opentelemetry-cluster` | No |
| `aws.ecs.launchtype` | string enum | The [launch type](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/launch_types.html) for an ECS task. | `EC2`<br>`Fargate` | No |
| `aws.ecs.task.arn` | string | The ARN of an [ECS task definition](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definitions.html). | `arn:aws:ecs:us-west-1:123456789123:task/10838bed-421f-43ef-870a-f43feacbbb5b` | No |
| `aws.ecs.task.family` | string | The task definition family this task definition is a member of. | `opentelemetry-family` | No |

`aws.ecs.launchtype` MUST be one of the following:

| Value | Description |
|---|---|
| `EC2` | EC2 |
| `Fargate` | Fargate |
<!-- endsemconv -->