Add quote_identifiers parameter to unpivot to handle case-sensitive column names

[error418]

resolves #216

Description & motivation

Added quote_identifiers parameter to unpivot to handle case-sensitive column names.

This is like #135 was for pivot.

Checklist

• This code is associated with an Issue which has been triaged and accepted for development.
• I have verified that these changes work locally on the following warehouses (Note: it's okay if you do not have access to all warehouses, this helps us understand what has been covered)
  • BigQuery (tested by @Ishankoradia - see PR comments)
  • Postgres
  • Redshift
  • Snowflake
• I followed guidelines to ensure that my changes will work on "non-core" adapters by:
  • dispatching any new macro(s) so non-core adapters can also use them (e.g. the star() source)
  • using the limit_zero() macro in place of the literal string: limit 0
  • using dbt.type_* macros instead of explicit datatypes (e.g. dbt.type_timestamp() instead of TIMESTAMP
• I have updated the README.md (if applicable)
• I have added tests & descriptions to my models (and macros if applicable)
• I have added an entry to CHANGELOG.md

[joellabes]

Thanks @error418! The changes to the macro look good; I have a few nitpicks around the tests though ⛏️

[error418]

rebased to current main and resolved merge conflicts in CHANGELOG.md

[error418]

Anything missing to get this merged?

[github-actions]

This PR has been marked as Stale because it has been open with no activity as of late. If you would like the PR to remain open, please comment on the PR or else it will be closed in 7 days.

[error418]

ping

[error418]

@Shadowsong27 @Ishankoradia @Abhishek-N at the moment I am waiting for a review by @joellabes (or any maintainer of dbt-utils).

@Ishankoradia thank you for testing the changes for BigQuery

Should there be any remaining issues with this PR I am happy to provide the necessary fixes.

[pnd-mnicolau]

bump

[alejobs]

What's missing here? I need this fix to be deployed. Thanks.

[MiguelMadero]

What's missing here? I need this fix to be deployed. Thanks.

[error418]

@joellabes could you please provide some information about the status of this pull request?

This is stalling for a while now without any feedback from the maintainers, which sadly becomes a bit frustrating.

Thanks

[dbeatty10]

Hi all,

This is our highest priority PR to review right now, and I'll be diving deeper into it this week.

After taking an initial look, the heart of the implementation looks good.

The main thing I'd like to follow-up on prior to approving + merging is the testing:

• It looks like there is special configuration in the CI tests so that Snowflake and Redshift pass

So I'd like to:

• Make sure that the test is running for Redshift.
• Align how Snowflake is handled across similar tests. i.e., either standardize on prior art or standardize on the adapter.quote() approach in this PR

[dbeatty10]

@error418 I think I figured out the root cause of the Redshift test failures in CI.

As you noted, the default behavior in Redshift is to lowercase all column names even if a table is created with quoted mixed-case column names.

It looks like the solution is to set enable_case_sensitive_identifier to true. I'm going to try adding that configuration to the CI cluster or workgroup's parameter group and see if that allows this test to pass for Redshift.

Addressed the tests ⚒️

[dbeatty10]

In an ideal world, I would rather have the default be quote_identifiers=True so that it matches with pivot. But agree with your choice here to default it to False instead so we don't have any breaking behavior change.

[dbeatty10]

We're ready to rock @error418 🎸

The tests in Redshift are passing after setting enable_case_sensitive_identifier to true in the cluster’s parameter group via the Amazon Redshift console.