Skip to content

docs: webhook ci documentation updates #2637

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Dec 2, 2022
Merged

docs: webhook ci documentation updates #2637

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
2dbfc9b
add webhook ci api
SNe789 Nov 15, 2022
f3e53d6
minor updates
SNe789 Nov 15, 2022
b3a9477
minor updates
SNe789 Nov 15, 2022
2eaead4
minor updates
SNe789 Nov 15, 2022
79efe53
minor updates
SNe789 Nov 15, 2022
cb9dd31
existing error
SNe789 Nov 21, 2022
38aae75
existing error
SNe789 Nov 21, 2022
e7c4566
Update ci-pipeline.md
prakarsh-dt Nov 22, 2022
f6013bc
updates
SNe789 Nov 28, 2022
f65f966
updates
SNe789 Nov 28, 2022
fc983aa
updates
SNe789 Dec 2, 2022
b75dac1
Update ci-pipeline.md
SNe789 Dec 2, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
148 changes: 118 additions & 30 deletions docs/user-guide/creating-application/workflow/ci-pipeline.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,27 +1,27 @@
# CI Pipeline

> **Info:**
> **Note:**
>
> For Devtron version older than v0.4.0, please refer the [CI Pipeline (legacy)](./ci-pipeline-legacy.md) page.

A CI Pipeline can be created in one of the three ways:
A CI Workflow can be created in one of the following ways:

* [Continuous Integration](#1-continuous-integration)
* [Linked CI Pipeline](#2.-linked-ci-pipeline)
* [Incoming Webhook](#3.-incoming-webhook)
* [Build and Deploy from Source Code](#1-build-and-deploy-from-source-code)
* [Linked Build Pipeline](#2-linked-build-pipeline)
* [Deploy Image from External Service](#3-deploy-image-from-external-service)

![](https://devtron-public-asset.s3.us-east-2.amazonaws.com/images/creating-application/workflow-ci-pipeline/workflow-ci.jpg)

Each of these methods has different use-cases that can be tailored to the needs of the organization.
Each method has different use-cases that can be tailored according the needs of the organization.

## 1. Continuous Integration
## 1. Build and Deploy from Source Code

Continuous Integration Pipeline allows you to build the container image from a source code repository.
`Build and Deploy from Source Code` workflow allows you to build the container image from a source code repository.

1. From the **Applications** menu, select your application.
2. On the **App Configuration** page, select **Workflow Editor**.
3. Select **+ New Build Pipeline**.
4. Select **Continuous Integration**.
3. Select **+ New Workflow**.
4. Select **Build and deploy from source code**.
5. Enter the following fields on the **Create build pipeline** screen:

![](../../../.gitbook/assets/ci-pipeline-2.png)
Expand Down Expand Up @@ -173,14 +173,14 @@ Select **Create Pipeline**.
>
> **(b)** The total timeout for the execution of the CI pipeline is by default set as 3600 seconds. This default timeout is configurable according to the use case. The timeout can be edited in the configmap of the orchestrator service in the env variable as `env:"DEFAULT_TIMEOUT" envDefault:"3600"`

## 2. Linked CI Pipeline
## 2. Linked Build Pipeline

If one code is shared across multiple applications, `Linked CI Pipeline` can be used, and only one image will be built for multiple applications because if there is only one build, it is not advisable to create multiple CI Pipelines.
If one code is shared across multiple applications, `Linked Build Pipeline` can be used, and only one image will be built for multiple applications because if there is only one build, it is not advisable to create multiple CI Pipelines.

1. From the **Applications** menu, select your application.
2. On the **App Configuration** page, select **Workflow Editor**.
3. Select **+ New Build Pipeline**.
4. Select **Linked CI Pipeline**.
3. Select **+ New Workflow**.
4. Select **Linked Build Pipeline**.
5. Enter the following fields on the **Create linked build pipeline** screen:

![](../../../.gitbook/assets/ca-workflow-linked.png)
Expand All @@ -189,33 +189,121 @@ If one code is shared across multiple applications, `Linked CI Pipeline` can be
* Select the source CI pipeline from the application that you selected above.
* Enter a name for the linked CI pipeline.

Select **Create Linked CI Pipeline**.
Click **Create Linked CI Pipeline**.

After creating a linked CI pipeline, you can create a CD pipeline.
Builds cannot be triggered from a linked CI pipeline; they can only be triggered from the source CI pipeline. There will be no images to deploy in the CD pipeline created from the 'linked CI pipeline' at first. To see the images in the CD pipeline of the linked CI pipeline, trigger build in the source CI pipeline. The build images will now be listed in the CD pipeline of the 'linked CI pipeline' whenever you trigger a build in the source CI pipeline.

## 3. Incoming Webhook
## 3. Deploy Image from External Service

The CI pipeline receives container images from an external source via a webhook service.
The CI pipeline can receive container images from an external source via webhook API.

You can use Devtron for deployments on Kubernetes while using your CI tool such as Jenkins. External CI features can be used when the CI tool is hosted outside the Devtron platform.
You can use Devtron for deployments on Kubernetes while using an external CI tool such as Jenkins or CircleCI. External CI feature can be used when the CI tool is hosted outside the Devtron platform. However, by using an external CI, you will not be able to use some of the Devtron features such as Image scanning and security policies, configuring pre-post CI stages etc.

1. From the **Applications** menu, select your application.
2. On the **App Configuration** page, select **Workflow Editor**.
3. Select **+ New Build Pipeline**.
4. Select **Incoming Webhook**.

![](../../../.gitbook/assets/ca-workflow-external.png)
* Create a [new](https://docs.devtron.ai/usage/applications/create-application) or [clone](https://docs.devtron.ai/usage/applications/cloning-application) an application.
* To configure `Git Repository`, you can add any Git repository account (e.g., dummy account) and click **Next**.
* To configure the `Container Registry` and `Container Repository`, you can leave the fields blank or simply add any test repository and click **Save & Next**.
* On the `Base Deployment Template` page, select the `Chart type` from the drop-down list and configure as per your [requirements](https://docs.devtron.ai/usage/applications/creating-application/deployment-template) and click **Save & Next**.
* On the **Workflow Editor** page, click **New Workflow** and select **Deploy image from external service**.

![](https://devtron-public-asset.s3.us-east-2.amazonaws.com/images/creating-application/webhook-ci/click-new-workflow.png)

![](https://devtron-public-asset.s3.us-east-2.amazonaws.com/images/creating-application/webhook-ci/select-deploy-image-from-external-service.png)

* On the **Deploy image from external source** page, provide the information in the following fields:

![](https://devtron-public-asset.s3.us-east-2.amazonaws.com/images/creating-application/webhook-ci/deploy-image-from-external-service.png)

| Fields | Description |
| --- | --- |
| **Deploy to environment** | <ul><li>`Environment`: Provide the name of the [environment](https://docs.devtron.ai/getting-started/global-configurations/cluster-and-environments/namespaces-and-environments).</ul></li><ul><li>`Namepsace`: Provide the [namespace](https://docs.devtron.ai/getting-started/global-configurations/cluster-and-environments/namespaces-and-environments).</ul></li> |
| **When do you want to deploy** | You can deploy either in one of the following ways: <ul><li>`Automatic`: If you select automatic, your application will be deployed automatically everytime a new image is received.</ul></li> <ul><li>`Manual`: In case of manual, you have to select the image and deploy manually. </ul></li>|
| **Deployment Strategy** | Configure the deployment preferences for this pipeline. |

* Click **Create Pipeline**.
A new CI pipeline will be created for the external source.
To get the webhook URL and JSON sample payload to be used in external CI pipeline, click **Show webhook details**.

![](https://devtron-public-asset.s3.us-east-2.amazonaws.com/images/creating-application/webhook-ci/show-webhook-details.jpg)

* On the **Webhook Details** page, you have to authenticate via `API token` to allow requests from an external service (e.g. Jenkins or CircleCI).

* For authentication, only users with `super-admin` permissions can select or generate an API token:
* You can either use **Select API Token** if you have generated an [API Token](https://docs.devtron.ai/getting-started/global-configurations/authorization/api-tokens) under **Global Configurations**.

![](https://devtron-public-asset.s3.us-east-2.amazonaws.com/images/creating-application/webhook-ci/select-api-token-webhook-details.png)


* Or use **Auto-generate token** to generate the API token with the required permissions. Make sure to enter the token name in the **Token name** field.

![](https://devtron-public-asset.s3.us-east-2.amazonaws.com/images/creating-application/webhook-ci/auto-generate-token-webhook-details.png)

* To allow requests from the external source, you can request the API by using:
* **Webhook URL**
* **cURL Request**

### Webhook URL

HTTP Method: `POST`

API Endpoint: `https://{domain-name}/orchestrator/webhook/ext-ci/{pipeline-id}`

JSON Payload:

```bash
{
"dockerImage": "445808685819.dkr.ecr.us-east-2.amazonaws.com/orch:23907713-2"
}
```

You can also select metadata to send to Devtron. Sample JSON will be generated accordingly.
You can send the Payload script to your CI tools such as Jenkins and Devtron will receive the build image every time the CI pipeline is triggered or you can use the Webhook URL, which will build an image every time CI pipeline is triggered using Devtron Dashboard.


### Sample cURL Request

```bash
curl --location --request POST \
'https://{domain-name}/orchestrator/webhook/ext-ci/{pipeline-id}' \
--header 'Content-Type: application/json' \
--header 'token: {token}' \
--data-raw '{
"dockerImage": "445808685819.dkr.ecr.us-east-2.amazonaws.com/orch:23907713-2"
}'
```

### Response Codes

| Code | Description |
| --- | --- |
| `200` | `app detail page url` |
| `400` | `Bad request` |
| `401` | `Unauthorized` |


### Integrate with External Sources - Jenkins or CircleCI

* On the Jenkins dashboard, select the Jenkins job which you want to integrate with the Devtron dashboard.
* Go to the **Configuration** > **Build Steps**, click **Add build step**, and then click **Execute Shell**.

![](https://devtron-public-asset.s3.us-east-2.amazonaws.com/images/creating-application/webhook-ci/add-build-step-jenkins.png)

* Enter the cURL request command.
* Make sure to enter the `API token` and `dockerImage` in your cURL command and click **Save**.

![](https://devtron-public-asset.s3.us-east-2.amazonaws.com/images/creating-application/webhook-ci/execute-shell-jenkins.jpg)

Now, you can access the images on the Devtron dashboard and deploy manually. In case, if you select **Automatic** deployment option, then your application will be deployed automatically everytime a new image is received.

Similarly, you can also integrate with external source such as **CircleCI** by:

* Select the job on the `CircleCI` dashboard and click `Configuration File`.
* On the respective job, enter the `cURL` command and update the `API token` and `dockerImage` in your cURL command.


| Field Name | Required/Optional | Description |
| :--- | :--- | :--- |
| **Pipeline Name** | Required | Name of the pipeline |
| **Source Type** | Required | ‘Branch Fixed’ or ‘Tag Regex’ |
| **Branch Name** | Required | Name of the branch |

* Select **Save and Generate URL**. This generates the Payload format and Webhook URL.

You can send the Payload script to your CI tools such as Jenkins and Devtron will receive the build image every time the CI Service is triggered or you can use the Webhook URL which will build an image every time CI Service is triggered using Devtron Dashboard.

## Update CI Pipeline

Expand Down
16 changes: 8 additions & 8 deletions docs/user-guide/global-configurations/authorization/api-tokens.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,16 +1,17 @@
# API Tokens
API tokens are like ordinary OAuth access tokens. They can be used instead of username and password for programmatic access to API.
API token allows users to generate API tokens with the desired access. Only super admin users can generate tokens and see generated tokens.

API tokens are the access tokens for authentication. Instead of using username and password, it can be used for programmatic access to API. It allows users to generate API tokens with the desired access. Only super admin users can generate API tokens and see the generated tokens.

## Generate API Token

To generate API tokens, go to global configurations -> Authorizations -> API tokens and click on Generate New Token.
To generate API tokens, go to Global Configurations -> Authorization -> API tokens and click `Generate New Token`.

![](https://devtron-public-asset.s3.us-east-2.amazonaws.com/images/global-configurations/api-token/api-token-1.png)

1. Enter a name for the token
1. Enter a name for the token.
2. Add Description.

3. Select an Expiration date for the token(7 days, 30 days, 60 days, 90 days, custom and no expiration)
3. Select an Expiration date for the token (7 days, 30 days, 60 days, 90 days, custom and no expiration).

![](https://devtron-public-asset.s3.us-east-2.amazonaws.com/images/global-configurations/api-token/api-token-2.png)

Expand All @@ -29,7 +30,6 @@ Or select specific permission if you want to generate a token with a specific ro
5. Now click on Generate Token.



A pop-up window will appear over the screen from where you can copy the API token.

![](https://devtron-public-asset.s3.us-east-2.amazonaws.com/images/global-configurations/api-token/api-token-5.png)
Expand Down Expand Up @@ -68,7 +68,7 @@ Select a new expiration date and click on regenerated token.

![](https://devtron-public-asset.s3.us-east-2.amazonaws.com/images/global-configurations/api-token/api-token-11.png)

This will generated a new token with new expiration date.
This will generated a new token with a new expiration date.


To update API token permissions, give the permissions as you want to and click on update token.
Expand All @@ -79,7 +79,7 @@ To update API token permissions, give the permissions as you want to and click o

## Delete API Token

To delete an API token, click on the delete icon. Any applications or scripts using this token will no longer be able to access the Devtron API.
To delete an API token, click `delete` icon. Any applications or scripts using this token will no longer be able to access the Devtron API.

![](https://devtron-public-asset.s3.us-east-2.amazonaws.com/images/global-configurations/api-token/api-token-14.png)

Expand Down