[WIP based on PR #16032] - Issue #16003: add kafka to no-error testing by dejan2609 · Pull Request #16395 · checkstyle/checkstyle

dejan2609 · 2025-02-22T16:32:37Z

See this PR for prologue: #16032 (comment)

Related issue: #16003

romani · 2025-02-22T17:29:51Z

@dejan2609 , I appreciate your help.

dejan2609 · 2025-03-22T14:57:21Z

I made some limited progress; will try to compare build environment configurations (i.e. Apache Kafka Github Actions vs. Checkstyle Travis CI).

Herewith an error from Travis CI:

> Task :coordinator-common:classes
Skipping task ':coordinator-common:classes' as it has no actions.
Resolve mutations for :coordinator-common:checkstyleMain (Thread[#86,included builds,5,main]) started.
:coordinator-common:checkstyleMain (Thread[#86,included builds,5,main]) started.

> Task :coordinator-common:checkstyleMain
Caching disabled for task ':coordinator-common:checkstyleMain' because:
  Build cache is disabled
Task ':coordinator-common:checkstyleMain' is not up-to-date because:
  No history is available.
Starting process 'Gradle Worker Daemon 2'. Working directory: /home/circleci/.gradle/workers Command: /usr/local/jdk-21.0.5/bin/java -Dcheckstyle.enableExternalDtdLoad=false --add-opens java.prefs/java.util.prefs=ALL-UNNAMED @/home/circleci/.gradle/.tmp/gradle-worker-classpath6644201704756195266txt -Xms200m -Xmx1g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Worker Daemon 2'
Successfully started process 'Gradle Worker Daemon 2'
Started Gradle worker daemon (1.305 secs) with fork options DaemonForkOptions{executable=/usr/local/jdk-21.0.5/bin/java, minHeapSize=200m, maxHeapSize=1g, jvmArgs=[--add-opens, java.prefs/java.util.prefs=ALL-UNNAMED], keepAliveMode=SESSION}.

This JVM does not support getting OS memory, so no OS memory status updates will be broadcast

> Task :coordinator-common:checkstyleMain
Running checkstyle with toolchain '/usr/local/jdk-21.0.5'.
[ant:checkstyle] Running Checkstyle 10.21.1-SNAPSHOT on 27 files

The message received from the daemon indicates that the daemon has disappeared.
Build request sent: Build{id=0f700024-4129-4544-9e94-197aff8c8cd7, currentDir=/home/circleci/project/.ci-temp/kafka}
Attempting to read last messages from the daemon log...
Daemon pid: 430
  log file: /home/circleci/.gradle/daemon/8.13/daemon-430.out.log
----- Last 20 lines from daemon log file - daemon-430.out.log -----
Compiling with toolchain '/usr/local/jdk-21.0.5'.
Compiling with JDK Java compiler API.
Class dependency analysis for incremental compilation took 0.063 secs.
Created classpath snapshot for incremental compilation in 0.074 secs.
work action resolve main (project :coordinator-common) (Thread[#86,included builds,5,main]) started.
Resolve mutations for :coordinator-common:classes (Thread[#86,included builds,5,main]) started.
:coordinator-common:classes (Thread[#86,included builds,5,main]) started.
Skipping task ':coordinator-common:classes' as it has no actions.
Resolve mutations for :coordinator-common:checkstyleMain (Thread[#86,included builds,5,main]) started.
:coordinator-common:checkstyleMain (Thread[#86,included builds,5,main]) started.
Caching disabled for task ':coordinator-common:checkstyleMain' because:
  Build cache is disabled
Task ':coordinator-common:checkstyleMain' is not up-to-date because:
  No history is available.
Starting process 'Gradle Worker Daemon 2'. Working directory: /home/circleci/.gradle/workers Command: /usr/local/jdk-21.0.5/bin/java -Dcheckstyle.enableExternalDtdLoad=false --add-opens java.prefs/java.util.prefs=ALL-UNNAMED @/home/circleci/.gradle/.tmp/gradle-worker-classpath6644201704756195266txt -Xms200m -Xmx1g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant worker.org.gradle.process.internal.worker.GradleWorkerMain 'Gradle Worker Daemon 2'
Successfully started process 'Gradle Worker Daemon 2'
Started Gradle worker daemon (1.305 secs) with fork options DaemonForkOptions{executable=/usr/local/jdk-21.0.5/bin/java, minHeapSize=200m, maxHeapSize=1g, jvmArgs=[--add-opens, java.prefs/java.util.prefs=ALL-UNNAMED], keepAliveMode=SESSION}.
This JVM does not support getting OS memory, so no OS memory status updates will be broadcast
Running checkstyle with toolchain '/usr/local/jdk-21.0.5'.
[ant:checkstyle] Running Checkstyle 10.21.1-SNAPSHOT on 27 files----- End of the daemon log -----


FAILURE: Build failed with an exception.

* What went wrong:
Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)

* Try:
> Run with --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.

* Exception is:
org.gradle.launcher.daemon.client.DaemonDisappearedException: Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)
	at org.gradle.launcher.daemon.client.DaemonClient.handleDaemonDisappearance(DaemonClient.java:283)
	at org.gradle.launcher.daemon.client.DaemonClient.monitorBuild(DaemonClient.java:256)

error_stacktrace.txt

dejan2609 · 2025-03-22T14:59:58Z

@romani just for the recored: is Travis CI usage mandatory for Checkstyle team?
Or build can be spawned on Github Actions also ?

romani · 2025-03-22T22:26:54Z

we can use any CI , github action is ok.
CircleCI is very quick CI too.
@dejan2609

dejan2609 · 2025-03-24T16:58:44Z

Ok then, let's wait with moving to GA or Circle CI (I think I can solve this on Travis CI).
Will send some more updates soon.

dejan2609 · 2025-04-23T15:05:25Z

Some updates: it seems that CircleCI somehow fails to honor Kafka Gradle settings/options/memory limits (and hence Gradle build on CircleCI fails due to lack of resources).

I'm experimenting with replicating Kafka Gradle settings onto Circle CI (here is the resource: https://circleci.com/docs/java-oom/#using-java-environment-variables-to-set-memory-limits).

dejan2609 · 2025-10-10T11:03:09Z

@romani I'm back, hopefully will be able to solve this in the foreseeable future 🛠️ 🙂

romani · 2025-10-10T12:37:08Z

I will have a bit more time in winter too, so we can collaborate.
If you be a bit verbose in comments, I might suggest something on the go.

We can make github action for this task, if circleci is so capricious.
Keep in mind we are on jdk17 , and all CI are moved to jdk21 as preparation to moving checkstyle min jdk to version 21 in January 2026

dejan2609 · 2025-10-10T13:09:15Z

We have a deal @romani ! Github Actions it is then 🤝

Incoming shameless plug 😎: I spent some time camping in Kafka's corner and scored a few wins there (Gradle 9 upgrade and Java 25 build).
So, Kafka builds on GA and against Java 17/25 matrix and hence I don't expect this issue to be that hard (but one never knows).
Let's nail this during the winter 🔨 ❄️ ☃️ 🙂

stoyanK7 · 2025-12-02T18:42:02Z

Some updates: it seems that CircleCI somehow fails to honor Kafka Gradle settings/options/memory limits (and hence Gradle build on CircleCI fails due to lack of resources).

@dejan2609 Do you remember if the message mentioned what the memory tipping point was?

dejan2609 · 2025-12-03T18:10:21Z

@stoyanK7 Здравей, Стоян, поздрави от Белград !

Things have changed on Kafka side quite a bit: both Gradle and Java versions are upgraded and you also have to factor in that my attempt was made on CircleCI (and CheckStyle team now opts exclusively for Github Actions) so those errors are kind of obsolete.

Also: given a fact that Kafka CI build is also using Github Actions we can be 💯 percent sure that build should (must) work fine in CheckStyle setup.

From where I see this one should execute following (or similar) Gradle command via CheckStyle GA worklow:

  ./gradlew --stacktrace --info --console=plain --no-daemon checkstyleMain checkstyleTest -x test \
    -PcheckstyleVersion="${CS_POM_VERSION}" \
    -I localRepo.gradle

Related notable comments:

⚠️ potential pitfall: Gradle daemon issues. I don't expect them now, but we will sort that out somehow (if they occur, God forbid).

dejan2609 · 2025-12-03T18:31:13Z

One more thing to add: I will put this stalled PR to rest 🙂

@stoyanK7 feel free to add me to your future PR (I can provide some non-binding review).

stoyanK7 · 2025-12-08T17:35:30Z

@dejan2609 Поздрави от София, Деян!

Thanks for the detailed response.

Things have changed on Kafka side quite a bit: both Gradle and Java versions are upgraded and you also have to factor in that my attempt was made on CircleCI (and CheckStyle team now opts exclusively for Github Actions) so those errors are kind of obsolete.

I'll first do my best to make it work on CircleCI. If I see it's a hassle, I'll opt for GitHub Actions. I'm curious to find out what's wrong with the memory usage 🤔

feel free to add me to your future PR (I can provide some non-binding review).

Of course! I'll open a PR in the following days and tag you 🚀

dejan2609 · 2025-12-08T18:33:10Z

@stoyanK7 for all I know (please read this: for all I can remember): we (both @romani and me) had issues with CircleCI Gradle build that would just stop (best bet: due to lack of memory): #16032 (comment)

Question is: do you want to utilize Gradle daemon build in a first place ?

Now, given a fact that Kafka is building against two Java versions (17 and 25, at the moment) answer is probably yes.
On the other hand: if CheckStyle team wants to perform checks with Kafka trunk (and latest CheckStyle snapshot) only against one Java version (and hence only single Gradle execution is required) maybe you could completely drop Gradle daemon usage in your Circle CI setup ?

Here me out: if you want to try your luck with Circle CI my suggestion would be to:

disable Gradle daemon
setup CircleCI job properly
and only then re-enable Gradle daemon (if you opt for a daemon at all, that is).

⚠️ Pitfall: Circle CI has this matrix, you may ~~have to wrap your had around here few times~~ 🙂 find it useful: https://circleci.com/docs/guides/optimize/java-oom/#using-java-environment-variables-to-set-memory-limits

That would be my 2 cents on this 🪙 🪙

Off topic: Levski or CSKA ? ⚽ 😃

romani · 2025-12-09T05:36:53Z

Now, given a fact that Kafka is building against two Java versions (17 and 25, at the moment) answer is probably yes.

On January I will merge jdk21 migration. So better to target jdk25 execution for Kafka.

Off topic: не знал что ЦСКА существует в других странах. :)

romani added 5 commits December 21, 2024 07:34

minor: DO NOT MERGE, disable all CIs

591e0f2

minor: DO NOT MERGE, disable all circleci except for kafka

9c63a1c

Issue checkstyle#16003: add kafka to no-error testing

ae773b2

minor: try

7f2709d

minor: try2

edc963d

bundling all commits into one (just to restart debugging)

4ea7405

dejan2609 force-pushed the i16003-kafka branch from 33abc65 to 4ea7405 Compare April 23, 2025 15:38

dejan2609 closed this Dec 3, 2025

stoyanK7 mentioned this pull request Dec 9, 2025

Issue #16003: Add Kafka to no-error testing #18263

Merged

Uh oh!

Conversation

dejan2609 commented Feb 22, 2025

Uh oh!

romani commented Feb 22, 2025

Uh oh!

dejan2609 commented Mar 22, 2025

Uh oh!

dejan2609 commented Mar 22, 2025

Uh oh!

romani commented Mar 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dejan2609 commented Mar 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dejan2609 commented Apr 23, 2025

Uh oh!

dejan2609 commented Oct 10, 2025

Uh oh!

romani commented Oct 10, 2025

Uh oh!

dejan2609 commented Oct 10, 2025

Uh oh!

stoyanK7 commented Dec 2, 2025

Uh oh!

dejan2609 commented Dec 3, 2025

Uh oh!

dejan2609 commented Dec 3, 2025

Uh oh!

stoyanK7 commented Dec 8, 2025

Uh oh!

dejan2609 commented Dec 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

romani commented Dec 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

romani commented Mar 22, 2025 •

edited

Loading

dejan2609 commented Mar 24, 2025 •

edited

Loading

dejan2609 commented Dec 8, 2025 •

edited

Loading

romani commented Dec 9, 2025 •

edited

Loading