Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
171343d
Update Changelog, temporarily remove Case Sensitivity testing (#174)
gwenwindflower Jun 4, 2024
e532bd4
Update testing structure for dbt Labs testing support - postgres (#181)
emmyoop Oct 18, 2024
7acc8c0
Restore CI test for case-sensitive identifiers when generating source…
dbeatty10 Nov 19, 2024
4668e22
Changlogs for 0.12.0, 0.12.1, and 0.13.0-b1 (#196)
dbeatty10 Nov 20, 2024
c959e4d
add snowflake (#198)
emmyoop Nov 20, 2024
b444bf8
Fix quoted identifiers in the `generate_base_model` macro for BigQuer…
dbeatty10 Nov 21, 2024
696c9f0
Try removing Redshift-specific logic (#208)
dbeatty10 Nov 21, 2024
5caf2e3
Use the `cimg/postgres` Docker image created by CircleCI with continu…
dbeatty10 Nov 21, 2024
6388e32
Independent workflow job for dbt-postgres (#215)
dbeatty10 Nov 22, 2024
f41d1b9
Simplify environment variables for BigQuery in CircleCI (#216)
dbeatty10 Nov 22, 2024
1cc3090
Stop installing prereleases from PyPI in favor of stable releases onl…
dbeatty10 Nov 22, 2024
fdc998c
Upgrade to Python 3.11 in CircleCI (#222)
dbeatty10 Nov 22, 2024
b28ab7d
Use dynamic schema names rather than hardcoded ones (#224)
dbeatty10 Nov 22, 2024
972aa3f
add support for redshift testing (#204)
emmyoop Nov 22, 2024
a88bc12
Add support for bigquery testing in GitHub CI via tox (#203)
emmyoop Nov 22, 2024
f5ec206
Update changelog for 0.13.0 release (#227)
dbeatty10 Dec 3, 2024
a79f28f
Revert "Restore CI test for case-sensitive identifiers when generatin…
dbeatty10 Dec 4, 2024
314908f
Update changelog for 0.13.1 release (#232)
dbeatty10 Dec 4, 2024
a3b6979
Upgrade from Postgres 9 to 17 (#234)
dbeatty10 Dec 5, 2024
2f20e94
Update ci.yml (#235)
emmyoop Dec 6, 2024
5a31c62
Bigquery repeated data type (#236)
Thrasi Dec 13, 2024
6137ca6
Remove "I have added an entry to CHANGELOG.md" from the PR template (…
dbeatty10 Dec 13, 2024
3169bda
Contributors shouldn't edit the `CHANGELOG.md` directly anymore (#240)
dbeatty10 Dec 13, 2024
af4e2c0
Remove the PR checklist items related to the type of change (#243)
dbeatty10 Dec 13, 2024
70b4ecb
Align the PR description with dbt-core, dbt-adapters, etc. (#244)
dbeatty10 Dec 13, 2024
4f04af0
Align the pull request template with `dbt-utils` (#246)
dbeatty10 Dec 13, 2024
95bb093
Update CODEOWNERS file (#248)
security-dbtlabs Feb 28, 2025
f6666ca
adding `generate_unit_test_template` macro (#251)
bruno-szdl Apr 3, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
125 changes: 78 additions & 47 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
@@ -1,98 +1,129 @@
version: 2

jobs:
build:

integration-postgres:
docker:
- image: cimg/python:3.9.9
- image: circleci/postgres:9.6.5-alpine-ram
- image: cimg/python:3.11
- image: cimg/postgres:17.0
environment:
POSTGRES_USER: root
environment:
POSTGRES_HOST: localhost
POSTGRES_USER: root
DBT_ENV_SECRET_POSTGRES_PASS: ''
POSTGRES_PORT: 5432
POSTGRES_DATABASE: circle_test
POSTGRES_SCHEMA: codegen_integration_tests_postgres

steps:
- checkout

- run:
run: setup_creds
command: |
echo $BIGQUERY_SERVICE_ACCOUNT_JSON > ${HOME}/bigquery-service-key.json

- restore_cache:
key: deps1-{{ .Branch }}

- run:
name: "Setup dbt"
command: |
python3 -m venv dbt_venv
. dbt_venv/bin/activate

python -m pip install --upgrade pip setuptools
python -m pip install --pre dbt-core dbt-postgres dbt-redshift dbt-snowflake dbt-bigquery

- run: pip install dbt-core dbt-postgres
- run:
name: "Run Tests - Postgres"
environment:
POSTGRES_TEST_HOST: localhost
POSTGRES_TEST_USER: root
POSTGRES_TEST_PASS: ""
POSTGRES_TEST_PORT: 5432
POSTGRES_TEST_DBNAME: circle_test
command: |
. dbt_venv/bin/activate
cd integration_tests
dbt --warn-error deps --target postgres
dbt --warn-error run-operation create_source_table --target postgres
dbt --warn-error seed --target postgres --full-refresh
dbt --warn-error run --target postgres
dbt --warn-error test --target postgres
- store_artifacts:
path: integration_tests/logs
- store_artifacts:
path: integration_tests/target

# The resource_class feature allows configuring CPU and RAM resources for each job. Different resource classes are available for different executors. https://circleci.com/docs/2.0/configuration-reference/#resourceclass
resource_class: large

integration-redshift:
docker:
- image: cimg/python:3.11
steps:
- checkout
- run: pip install dbt-core dbt-redshift
- run:
name: "Run Tests - Redshift"
command: |
. dbt_venv/bin/activate
echo `pwd`
cd integration_tests
dbt --warn-error deps --target redshift
dbt --warn-error run-operation create_source_table --target redshift
dbt --warn-error seed --target redshift --full-refresh
dbt --warn-error run --target redshift
dbt --warn-error test --target redshift
- store_artifacts:
path: integration_tests/logs
- store_artifacts:
path: integration_tests/target
# The resource_class feature allows configuring CPU and RAM resources for each job. Different resource classes are available for different executors. https://circleci.com/docs/2.0/configuration-reference/#resourceclass
resource_class: large

integration-snowflake:
docker:
- image: cimg/python:3.11
steps:
- checkout
- run: pip install dbt-core dbt-snowflake
- run:
name: "Run Tests - Snowflake"
command: |
. dbt_venv/bin/activate
echo `pwd`
cd integration_tests
dbt --warn-error deps --target snowflake
dbt --warn-error run-operation create_source_table --target snowflake
dbt --warn-error seed --target snowflake --full-refresh
dbt --warn-error run --target snowflake
dbt --warn-error test --target snowflake
- store_artifacts:
path: integration_tests/logs
- store_artifacts:
path: integration_tests/target
# The resource_class feature allows configuring CPU and RAM resources for each job. Different resource classes are available for different executors. https://circleci.com/docs/2.0/configuration-reference/#resourceclass
resource_class: large

integration-bigquery:
environment:
BIGQUERY_SERVICE_KEY_PATH: "/home/circleci/bigquery-service-key.json"
docker:
- image: cimg/python:3.11
steps:
- checkout
- run: pip install dbt-core dbt-bigquery
- run:
name: Setup Environment Variables
command: |
echo $BIGQUERY_SERVICE_ACCOUNT_JSON > $BIGQUERY_SERVICE_KEY_PATH
echo 'export BIGQUERY_KEYFILE_JSON="$BIGQUERY_SERVICE_ACCOUNT_JSON"' >> "$BASH_ENV"
- run:
name: "Run Tests - BigQuery"
environment:
BIGQUERY_SERVICE_KEY_PATH: "/home/circleci/bigquery-service-key.json"

command: |
. dbt_venv/bin/activate
echo `pwd`
cd integration_tests
dbt --warn-error deps --target bigquery
dbt --warn-error run-operation create_source_table --target bigquery
dbt --warn-error seed --target bigquery --full-refresh
dbt --warn-error run --target bigquery
dbt --warn-error test --target bigquery

- save_cache:
key: deps1-{{ .Branch }}
paths:
- "dbt_venv"
- store_artifacts:
path: integration_tests/logs
- store_artifacts:
path: integration_tests/target
# The resource_class feature allows configuring CPU and RAM resources for each job. Different resource classes are available for different executors. https://circleci.com/docs/2.0/configuration-reference/#resourceclass
resource_class: large

workflows:
version: 2
test-all:
jobs:
- build:
context:
- profile-redshift
- profile-snowflake
- profile-bigquery
- integration-postgres:
context: profile-postgres
- integration-redshift:
context: profile-redshift
requires:
- integration-postgres
- integration-snowflake:
context: profile-snowflake
requires:
- integration-postgres
- integration-bigquery:
context: profile-bigquery
requires:
- integration-postgres
2 changes: 1 addition & 1 deletion .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -1 +1 @@
* @clrcrl
* @dbt-labs/dbt-package-owners
26 changes: 14 additions & 12 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,23 @@
resolves #

This is a:
- [ ] documentation update
- [ ] bug fix with no breaking changes
- [ ] new functionality
- [ ] a breaking change
### Problem

All pull requests from community contributors should target the `main` branch (default).
<!---
Describe the problem this PR is solving. What is the application state
before this PR is merged?
-->

### Solution

## Description & motivation
<!---
Describe your changes, and why you're making them.
Describe the way this PR solves the above problem. Add as much detail as you
can to help reviewers understand your changes. Include any alternatives and
tradeoffs you considered.
-->

## Checklist
- [ ] This code is associated with an issue which has been triaged and [accepted for development](https://docs.getdbt.com/docs/contributing/oss-expectations#pull-requests).
- [ ] I have verified that these changes work locally
- [ ] This code is associated with an [issue](https://github.com/dbt-labs/dbt-codegen/issues) which has been triaged and [accepted for development](https://docs.getdbt.com/docs/contributing/oss-expectations#pull-requests).
- [ ] I have read [the contributing guide](https://github.com/dbt-labs/dbt-codegen/blob/main/CONTRIBUTING.md) and understand what's expected of me
- [ ] I have run this code in development and it appears to resolve the stated issue
- [ ] This PR includes tests, or tests are not required/relevant for this PR
- [ ] I have updated the README.md (if applicable)
- [ ] I have added tests & descriptions to my models (and macros if applicable)
- [ ] I have added an entry to CHANGELOG.md
45 changes: 45 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# **what?**
# Run tests for dbt-codegen against supported adapters

# **why?**
# To ensure that dbt-codegen works as expected with all supported adapters

# **when?**
# On every PR, and every push to main and when manually triggered

name: Package Integration Tests

on:
push:
branches:
- main
pull_request_target:
workflow_dispatch:

jobs:
run-tests:
uses: dbt-labs/dbt-package-testing/.github/workflows/run_tox.yml@v1
# this just tests with postgres so no variables need to be passed through.
# When it's time to add more adapters you will need to pass through inputs for
# the other adapters as shown in the below example for redshift
with:
# snowflake
SNOWFLAKE_USER: ${{ vars.SNOWFLAKE_USER }}
SNOWFLAKE_ROLE: ${{ vars.SNOWFLAKE_ROLE }}
SNOWFLAKE_DATABASE: ${{ vars.SNOWFLAKE_DATABASE }}
SNOWFLAKE_WAREHOUSE: ${{ vars.SNOWFLAKE_WAREHOUSE }}
SNOWFLAKE_SCHEMA: "integration_tests_snowflake_${{ github.run_number }}"
# bigquery
BIGQUERY_PROJECT: ${{ vars.BIGQUERY_PROJECT }}
BIGQUERY_SCHEMA: "integration_tests_bigquery_${{ github.run_number }}"
# redshift
REDSHIFT_HOST: ${{ vars.REDSHIFT_HOST }}
REDSHIFT_USER: ${{ vars.REDSHIFT_USER }}
REDSHIFT_DATABASE: ${{ vars.REDSHIFT_DATABASE }}
REDSHIFT_SCHEMA: "integration_tests_redshift_${{ github.run_number }}"
REDSHIFT_PORT: ${{ vars.REDSHIFT_PORT }}
secrets:
SNOWFLAKE_ACCOUNT: ${{ secrets.SNOWFLAKE_ACCOUNT }}
DBT_ENV_SECRET_SNOWFLAKE_PASS: ${{ secrets.SNOWFLAKE_PASS }}
DBT_ENV_SECRET_REDSHIFT_PASS: ${{ secrets.REDSHIFT_PASS }}
BIGQUERY_KEYFILE_JSON: ${{ secrets.BIGQUERY_KEYFILE_JSON }}
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ target/
dbt_modules/
dbt_packages/
logs/
env/
env*/
.venv/
.env/
venv/
Loading