Use docker compose for kafka integration tests

[hellspawn679]

Which problem is this PR solving?

Part of #5485

Description of the changes

• In script/kafka-integration-test.sh initialized kafka by docker compose
• updated .github/workflow/ci-kafka.yml to take logs from docker compose
• added comments in script/kafka-integration-test.sh and fixed a typo
• add new locations to dependabot config to allow it to upgrade versions in the new docker compose files

How was this change tested?

• CI

Checklist

• I have read https://github.com/jaegertracing/jaeger/blob/master/CONTRIBUTING_GUIDELINES.md
• I have signed all commits
• I have added unit tests for the new functionality
• I have run lint and test steps successfully
  • for jaeger: make lint test
  • for jaeger-ui: yarn lint and yarn test

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.18%. Comparing base (629c47e) to head (59e1434).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #5500   +/-   ##
=======================================
  Coverage   96.18%   96.18%           
=======================================
  Files         327      327           
  Lines       16012    16012           
=======================================
  Hits        15401    15401           
  Misses        436      436           
  Partials      175      175           

Flag	Coverage Δ
badger_v1	8.05% <ø> (ø)
badger_v2	1.93% <ø> (ø)
cassandra-3.x-v1	16.43% <ø> (ø)
cassandra-3.x-v2	1.85% <ø> (ø)
cassandra-4.x-v1	16.43% <ø> (ø)
cassandra-4.x-v2	1.85% <ø> (ø)
elasticsearch-7-v1	18.88% <ø> (?)
elasticsearch-8-v1	19.08% <ø> (?)
elasticsearch-8-v2	19.08% <ø> (?)
elasticsearch-8.x-v2	?
grpc_v1	9.48% <ø> (ø)
grpc_v2	7.53% <ø> (ø)
kafka	9.78% <ø> (ø)
opensearch-1-v1	18.92% <ø> (?)
opensearch-1.x-v1	?
opensearch-2-v1	18.94% <ø> (?)
opensearch-2-v2	18.92% <ø> (?)
opensearch-2.x-v1	?
opensearch-2.x-v2	?
unittests	93.87% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[yurishkuro]

please write informative PR title and description

[hellspawn679]

please have a look at dependabot then I will add it for both elaticsearch/opensearch and kafka

[hellspawn679]

@yurishkuro as you can see when the elasticsearch 8.x v2failed the output elastic container logs job was not complete to fix this we need to use
docker compose -f docker-compose/elasticsearch/v${{matrix.version.major}}.yml logs
in place of
docker log <GITHUB_OUTPUT>(Cid)
and we probably need to change
the jobs matrix
from {{matrix.version.major}}-->7.x to {{matrix.version.major}}-->7
and we won't be needing GITHUB_OUTPUT any more

[hellspawn679]

[yurishkuro]

neat, I didn't know this was possible, and it addresses one of my concerns in the ticket (to avoid it auto-upgrade 7x -> 8x)

[hellspawn679]

@yurishkuro as you can see when the elasticsearch 8.x v2failed the output elastic container logs job was not complete to fix this we need to use docker compose -f docker-compose/elasticsearch/v${{matrix.version.major}}.yml logs in place of docker log <GITHUB_OUTPUT>(Cid) and we probably need to change the jobs matrix from {{matrix.version.major}}-->7.x to {{matrix.version.major}}-->7 and we won't be needing GITHUB_OUTPUT any more

@yurishkuro can you have a look at this should I include this issue in this pr only ?

[yurishkuro]

can you have a look at this should I include this issue in this pr only ?

Yeah, you can add that here. I think it would be better to keep the encapsulation and not require the workflow to know the exact file name - similar to cid >> $GITHUB_OUTPUT, you can store the compose file name in the output and have the next step use that

[yurishkuro]

Thanks!