Skip to content

Conversation

@matthewmcneely
Copy link
Contributor

@matthewmcneely matthewmcneely commented Aug 15, 2025

Description

This PR significantly improves performance of "t"-based testing in Dgraph

  1. Removes Docker container address resolution from the testutil init() function (any package that imports would block waiting for container addresses that weren't established yet). Instead, the PR introduces a lazy-loading concept that sync.Once's the address resolution. This saves about 20 seconds per test invocation from t
  2. Parallel-izes the resolution of expected container addresses
  3. Parallel-izes the health and login flows
  4. Suppresses health-check and login warnings until after a reasonable threshold of failures (these checks constantly pollute the logs)
  5. On non-Linux system, informs the user about DGRAPH_BINARY settings that can let tests proceed.
  6. Upgrades the warp runner for the core tests to the 16x instance (core tests are the most comprehensive tests)

In general (on my M4) many tests improve time to complete by a factor of 3.

Checklist

  • Code compiles correctly and linting passes locally

@github-actions github-actions bot added area/testing Testing related issues area/graphql Issues related to GraphQL support on Dgraph. area/live-loader Issues related to live loading. area/acl Related to Access Control Lists area/core internal mechanisms go Pull requests that update Go code labels Aug 15, 2025
@matthewmcneely matthewmcneely marked this pull request as ready for review August 15, 2025 18:24
@matthewmcneely matthewmcneely requested review from a team and Copilot August 15, 2025 18:24
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces lazy initialization of Docker container addresses to improve performance of test harness execution. The changes move from global variable initialization to a thread-safe, on-demand address resolution pattern that supports concurrent address lookups and provides fallback values when Docker isn't available.

  • Replaces direct global variable access with getter functions that ensure addresses are initialized
  • Implements concurrent address resolution using goroutines and sync.WaitGroup for faster startup
  • Adds fallback default addresses when Docker containers are not available

Reviewed Changes

Copilot reviewed 54 out of 54 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
testutil/client.go Core refactoring: converts global variables to private with lazy initialization, adds getter functions, implements concurrent address resolution
Multiple test files Updates all test files to use new getter functions instead of direct global variable access
testutil/minio.go Adds MaxRetries configuration for improved reliability
t/t.go Adds concurrent health checking for cluster startup and GOPATH validation
dgraph/docker-compose.yml Improves GOPATH environment variable validation with error messages

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@matthewmcneely matthewmcneely force-pushed the matthewmcneely/improve-testing-performance branch from ef42d4e to 424ab74 Compare August 21, 2025 23:34
@trunk-io
Copy link

trunk-io bot commented Aug 22, 2025

Static BadgeStatic BadgeStatic BadgeStatic Badge

View Full Report ↗︎Docs

@github-actions github-actions bot added the area/integrations Related to integrations with other projects. label Aug 22, 2025
@matthewmcneely matthewmcneely changed the title chore(test): WIP. Increase performance of t-harness tests chore(test): Increase performance of t-harness tests Aug 22, 2025
@matthewmcneely matthewmcneely enabled auto-merge (squash) August 22, 2025 21:52
@matthewmcneely matthewmcneely merged commit 5d6928f into main Aug 22, 2025
14 of 15 checks passed
@matthewmcneely matthewmcneely deleted the matthewmcneely/improve-testing-performance branch August 22, 2025 23:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/acl Related to Access Control Lists area/core internal mechanisms area/graphql Issues related to GraphQL support on Dgraph. area/integrations Related to integrations with other projects. area/live-loader Issues related to live loading. area/testing Testing related issues go Pull requests that update Go code

Development

Successfully merging this pull request may close these issues.

2 participants