[Fix][Connector-V2][MySQL-CDC] Add support for MYSQL_SET_UNSIGNED type in MySqlTypeConverter#10453
[Fix][Connector-V2][MySQL-CDC] Add support for MYSQL_SET_UNSIGNED type in MySqlTypeConverter#10453wgzhao wants to merge 7 commits intoapache:devfrom
Conversation
…r CDC source connector
…e in MySqlTypeConverter fix apache#10451
Issue 1: The added MYSQL_SET_UNSIGNED type does not exist in MySQLLocation: static final String MYSQL_SET_UNSIGNED = "SET UNSIGNED";Issue Description: Related Context:
Potential Risks:
Impact Scope:
Severity: MAJOR Improvement Suggestions: // Option 1: If truly necessary, add detailed comments explaining the source
/**
* Handle edge case where Debezium may report type as "SET UNSIGNED"
* even though MySQL doesn't officially support this type.
* See issue #10451 for details.
*/
static final String MYSQL_SET_UNSIGNED = "SET UNSIGNED";
// Option 2: Normalize at the type conversion entry point (recommended)
// In MySqlTypeUtils.convertToSeaTunnelColumn():
String dataType = column.typeName().toUpperCase();
if ("SET UNSIGNED".equals(dataType)) {
log.warn("Normalizing unexpected type name 'SET UNSIGNED' to 'SET' for column {}",
column.name());
dataType = "SET";
}
builder.dataType(dataType);Rationale:
Issue 2: Missing corresponding unit testsLocation: Issue Description: Related Context:
Potential Risks:
Impact Scope:
Severity: MAJOR Improvement Suggestions: @Test
public void testConvertSetUnsigned() {
BasicTypeDefine<Object> typeDefine =
BasicTypeDefine.builder()
.name("test")
.columnType("SET UNSIGNED")
.dataType("SET UNSIGNED")
.length(100L)
.build();
Column column = MySqlTypeConverter.DEFAULT_INSTANCE.convert(typeDefine);
Assertions.assertEquals(typeDefine.getName(), column.getName());
Assertions.assertEquals(BasicType.STRING_TYPE, column.getDataType());
Assertions.assertEquals(100, column.getColumnLength());
Assertions.assertEquals(typeDefine.getColumnType(), column.getSourceType());
// Test with default length
typeDefine = BasicTypeDefine.builder()
.name("test")
.columnType("SET UNSIGNED")
.dataType("SET UNSIGNED")
.length(0L)
.build();
column = MySqlTypeConverter.DEFAULT_INSTANCE.convert(typeDefine);
Assertions.assertEquals(100, column.getColumnLength());
}Rationale:
Issue 3: PR description does not match actual changesLocation: PR description section Issue Description: Related Context:
Potential Risks:
Impact Scope:
Severity: MINOR Improvement Suggestions: Rationale:
Issue 4: Missing warning logs for exceptional typesLocation: case MYSQL_ENUM:
case MYSQL_SET:
case MYSQL_SET_UNSIGNED:
builder.dataType(BasicType.STRING_TYPE);
// ...Issue Description: Related Context:
Potential Risks:
Impact Scope:
Severity: MINOR Improvement Suggestions: case MYSQL_ENUM:
case MYSQL_SET:
builder.dataType(BasicType.STRING_TYPE);
if (typeDefine.getLength() == null || typeDefine.getLength() <= 0) {
builder.columnLength(100L);
} else {
builder.columnLength(typeDefine.getLength());
}
break;
case MYSQL_SET_UNSIGNED:
log.warn("Unexpected type '{}' encountered for column '{}'. MySQL does not officially "
+ "support SET UNSIGNED. This may indicate a Debezium reporting issue. "
+ "Treating as SET type.", MYSQL_SET_UNSIGNED, typeDefine.getName());
builder.dataType(BasicType.STRING_TYPE);
if (typeDefine.getLength() == null || typeDefine.getLength() <= 0) {
builder.columnLength(100L);
} else {
builder.columnLength(typeDefine.getLength());
}
break;Rationale:
Issue 5: OceanBaseMySqlTypeConverter may need synchronized changesLocation: Issue Description: Related Context:
Potential Risks:
Impact Scope:
Severity: MINOR (needs confirmation if OceanBase supports SET) Improvement Suggestions:
// If OceanBase supports SET
static final String MYSQL_SET = "SET";
// Add in switch
case MYSQL_ENUM:
case MYSQL_SET:
builder.dataType(BasicType.STRING_TYPE);
// ...Rationale:
|
Purpose of this pull request
This pull request try to fix #10451
Does this PR introduce any user-facing change?
No
How was this patch tested?
I tested in my production env
Check list
New License Guide
incompatible-changes.mdto describe the incompatibility caused by this PR.