Skip to content

fix(dbt): fix DataSource typo and enum definition #1977

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
douenergy merged 4 commits into from
Sep 26, 2025
Merged

fix(dbt): fix DataSource typo and enum definition #1977

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
1ac83d3
update schema
goldmedal Sep 26, 2025
c9b31c5
fix typo and enum definition
goldmedal Sep 26, 2025
e79c129
update data source validation
goldmedal Sep 26, 2025
af727e2
fix lint
goldmedal Sep 26, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 11 additions & 7 deletions wren-launcher/commands/dbt/converter.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -341,14 +341,15 @@ func ConvertDbtCatalogToWrenMDL(catalogPath string, dataSource DataSource, manif
// --- 2. Initialize Wren Manifest and Pre-process Metadata ---

manifest := &WrenMDLManifest{
JsonSchema: "https://raw.githubusercontent.com/Canner/WrenAI/main/wren-mdl/mdl.schema.json",
Catalog: "wren",
Schema: "public",
EnumDefinitions: []EnumDefinition{},
Models: []WrenModel{},
Relationships: []Relationship{},
Metrics: []Metric{},
Views: []View{},
DataSources: dataSource.GetType(),
DataSource: dataSource.GetType(),
}

// Create lookup maps to store pre-processed information for quick access.
Expand Down Expand Up @@ -622,9 +623,11 @@ func extractRelationshipsFromTests(fromModelName, fromColumnName string, tests [
// createOrLinkEnum is a helper to de-duplicate and manage enum creation based on 'accepted_values' tests.
func createOrLinkEnum(modelName, columnName, columnKey string, values []interface{}, allEnums *[]EnumDefinition, enumValueToNameMap, columnToEnumNameMap map[string]string) {
var strValues []string
var enumValues []EnumValue
for _, v := range values {
if s, ok := v.(string); ok {
strValues = append(strValues, s)
enumValues = append(enumValues, EnumValue{Name: s})
}
Comment on lines +628 to 633
Copy link
Contributor

@coderabbitai coderabbitai bot Sep 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

accepted_values: non-string enum values are silently dropped.

createOrLinkEnum only captures strings, ignoring numbers/booleans. This changes behavior for valid dbt accepted_values lists containing non-strings.

Apply this diff to include all simple types by stringifying:

 func createOrLinkEnum(modelName, columnName, columnKey string, values []interface{}, allEnums *[]EnumDefinition, enumValueToNameMap, columnToEnumNameMap map[string]string) {
   var strValues []string
-  var enumValues []EnumValue
-  for _, v := range values {
-    if s, ok := v.(string); ok {
-      strValues = append(strValues, s)
-      enumValues = append(enumValues, EnumValue{Name: s})
-    }
-  }
+  var enumValues []EnumValue
+  for _, v := range values {
+    s := fmt.Sprint(v)
+    strValues = append(strValues, s)
+    enumValues = append(enumValues, EnumValue{Name: s})
+  }
   if len(strValues) == 0 {
     return
   }
@@
-    *allEnums = append(*allEnums, EnumDefinition{
-      Name:   enumName,
-      Values: enumValues,
-    })
+    *allEnums = append(*allEnums, EnumDefinition{
+      Name:   enumName,
+      Values: enumValues,
+    })

Also applies to: 652-653

🤖 Prompt for AI Agents 
In wren-launcher/commands/dbt/converter.go around lines 628-633 (and similarly
at 652-653), the loop only appends values when they are strings, silently
dropping numeric/boolean/etc. Modify the logic to accept all simple scalar types
by converting non-string simple values to their string representation (e.g.,
fmt.Sprint or similar) before appending to strValues and enumValues, while still
ignoring complex types (maps/slices); apply the same change to the other
occurrence at lines 652-653 so all accepted_values entries are preserved as
stringified EnumValue{Name: ...}.

}
if len(strValues) == 0 {
Expand All @@ -644,7 +647,7 @@ func createOrLinkEnum(modelName, columnName, columnKey string, values []interfac
}
*allEnums = append(*allEnums, EnumDefinition{
Name: enumName,
Values: strValues,
Values: enumValues,
})
enumValueToNameMap[valueKey] = enumName
}
Expand Down Expand Up @@ -954,11 +957,6 @@ func buildWrenColumn(colMap map[string]interface{}, nodeKey string, dataSource D
NotNull: columnToNotNullMap[columnKey], // Defaults to false if not found
}

// Assign an enum if one was derived from dbt tests
if enumName, ok := columnToEnumNameMap[columnKey]; ok {
column.Enum = enumName
}

// Use a temporary map to build the properties
properties := make(map[string]string)
if description, exists := columnDescriptions[column.Name]; exists && description != "" {
Expand All @@ -968,6 +966,12 @@ func buildWrenColumn(colMap map[string]interface{}, nodeKey string, dataSource D
properties["comment"] = comment
}

// Assign an enum if one was derived from dbt tests
// TODO: enum isn't implemented in Wren yet, putting this here for future use
if enumName, ok := columnToEnumNameMap[columnKey]; ok {
properties["enumDefinition"] = enumName
}

// Assign the properties map only if it's not empty
if len(properties) > 0 {
column.Properties = properties
Expand Down
13 changes: 9 additions & 4 deletions wren-launcher/commands/dbt/wren_mdl.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,26 @@ package dbt

// WrenMDLManifest represents the complete Wren MDL structure
type WrenMDLManifest struct {
JsonSchema string `json:"$schema"`
Catalog string `json:"catalog"`
Schema string `json:"schema"`
EnumDefinitions []EnumDefinition `json:"enumDefinitions,omitempty"`
Models []WrenModel `json:"models"`
Relationships []Relationship `json:"relationships"`
Metrics []Metric `json:"metrics,omitempty"`
Views []View `json:"views"`
DataSources string `json:"dataSources,omitempty"`
DataSource string `json:"dataSource,omitempty"`
}

// EnumDefinition represents a named list of values that can be used by columns.
type EnumDefinition struct {
Name string `json:"name"`
Values []string `json:"values"`
Name string `json:"name"`
Values []EnumValue `json:"values"`
}

type EnumValue struct {
Name string `json:"name"`
Value string `json:"value,omitempty"`
}

// WrenModel represents a model in the Wren MDL format
Expand All @@ -41,7 +47,6 @@ type WrenColumn struct {
Name string `json:"name"`
DisplayName string `json:"displayName,omitempty"`
Type string `json:"type"`
Enum string `json:"enum,omitempty"`
Relationship string `json:"relationship,omitempty"`
IsCalculated bool `json:"isCalculated,omitempty"`
NotNull bool `json:"notNull,omitempty"`
Expand Down
28 changes: 8 additions & 20 deletions wren-mdl/mdl.schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://github.com/Canner/WrenAI/tree/main/wren-mdl/mdl.schema.json",
"$id": "https://raw.githubusercontent.com/Canner/WrenAI/main/wren-mdl/mdl.schema.json",
"title": "WrenMDL Manifest Schema",
"description": "A schema for WrenMDL manifest file",
"$defs": {
Expand Down Expand Up @@ -124,6 +124,11 @@
},
"type": "object",
"properties": {
"$schema": {
"description": "the schema of WrenMDL",
"type": "string",
"const": "https://raw.githubusercontent.com/Canner/WrenAI/main/wren-mdl/mdl.schema.json"
},
"catalog": {
"description": "the catalog name of WrenMDL",
"type": "string",
Expand All @@ -140,26 +145,9 @@
"minLength": 1
},
"dataSource": {
"description": "the data source type",
"description": "the data source type (case insensitive)",
"type": "string",
"enum": [
"BIGQUERY",
"CLICKHOUSE",
"CANNER",
"TRINO",
"MSSQL",
"MYSQL",
"POSTGRES",
"SNOWFLAKE",
"DUCKDB",
"LOCAL_FILE",
"S3_FILE",
"GCS_FILE",
"MINIO_FILE",
"ORACLE",
"ATHENA",
"REDSHIFT"
],
"pattern": "^(?i)(BIGQUERY|CLICKHOUSE|CANNER|TRINO|MSSQL|MYSQL|POSTGRES|SNOWFLAKE|DUCKDB|LOCAL_FILE|S3_FILE|GCS_FILE|MINIO_FILE|ORACLE|ATHENA|REDSHIFT)$",
"minLength": 1
},
"models": {
Expand Down