Skip to content

Conversation

@njnu-seafish
Copy link
Contributor

Purpose of the pull request

close #17413

Brief change log

The Hive and Spark data source information can be displayed normally, and the tests pass.

Verify this pull request

This pull request is code cleanup without any test coverage.

Setting up an end-to-end test for the Kerberos environment is somewhat challenging. Let's fix the issue first, and we can look into adding the E2E test later when time permits.

Pull Request Notice

Pull Request Notice

If your pull request contains incompatible change, you should also add it to docs/docs/en/guide/upgrade/incompatible.md

@njnu-seafish
Copy link
Contributor Author

njnu-seafish commented Sep 11, 2025

for my env, hive version is 3.1.2 and enable kerberos, the following error appears:

Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/hive/thrift/TFilterTransport
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:493)
at java.net.URLClassLoader.access$100(URLClassLoader.java:75)
at java.net.URLClassLoader$1.run(URLClassLoader.java:389)
at java.net.URLClassLoader$1.run(URLClassLoader.java:383)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at org.apache.hive.jdbc.HiveConnection.createBinaryTransport(HiveConnection.java:601)
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:341)
at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:228)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at org.apache.dolphinscheduler.plugin.datasource.hive.param.HiveDataSourceProcessor.getConnection(HiveDataSourceProcessor.java:139)

Please add the following dependencies:

        <dependency>
            <groupId>org.apache.hive</groupId>
            <artifactId>hive-exec</artifactId>
            <version>3.1.2</version>
        </dependency>

I'm uncertain if Hive 2.x or other versions need this in kerberos env, so I'll skip adding the dependency for now.

Copy link
Member

@ruanwenjun ruanwenjun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ruanwenjun ruanwenjun added the bug Something isn't working label Sep 11, 2025
@ruanwenjun ruanwenjun added this to the 3.3.2 milestone Sep 11, 2025
Copy link
Member

@SbloodyS SbloodyS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@sonarqubecloud
Copy link

Quality Gate Failed Quality Gate failed

Failed conditions
50.0% Coverage on New Code (required ≥ 60%)

See analysis details on SonarQube Cloud

@SbloodyS SbloodyS merged commit b5e7398 into apache:dev Sep 17, 2025
70 of 71 checks passed
davidzollo pushed a commit to davidzollo/dolphinscheduler that referenced this pull request Oct 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] [DataSource] Hive and Spark not work properly in kerberos env(Principal field is not displayed and not used correctly)

3 participants