Skip to content

[Tooling] Returning failed tests full output#542

Merged
deblasis merged 2 commits intomainfrom
issue/improve_test_failure_reporting
Feb 23, 2023
Merged

[Tooling] Returning failed tests full output#542
deblasis merged 2 commits intomainfrom
issue/improve_test_failure_reporting

Conversation

@deblasis
Copy link
Copy Markdown
Contributor

@deblasis deblasis commented Feb 23, 2023

Description

This PR updated our Github workflow to return the full output of failed tests for easier debugging/troubleshooting.

It was raised by @bryanchriswhite here:
#534 (comment)

image

And also probably related:

image

Issue

Type of change

Please mark the relevant option(s):

  • New feature, functionality or library
  • Bug fix
  • Code health or cleanup
  • Major breaking change
  • Documentation
  • Other

List of changes

  • updated jq 🪄 to return all the rows that match a failed test

Testing

  • make develop_test
  • LocalNet w/ all of the steps outlined in the README

Required Checklist

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have tested my changes using the available tooling
  • I have updated the corresponding CHANGELOG

If Applicable Checklist

  • I have updated the corresponding README(s); local and/or global
  • I have added tests that prove my fix is effective or that my feature works
  • I have added, or updated, mermaid.js diagrams in the corresponding README(s)
  • I have added, or updated, documentation and mermaid.js diagrams in shared/docs/* if I updated shared/*README(s)

@deblasis deblasis added infra Core infrastructure - not protocol related tooling tooling to support development, testing et al labels Feb 23, 2023
@deblasis deblasis self-assigned this Feb 23, 2023
@deblasis deblasis requested a review from okdas February 23, 2023 15:06
@deblasis
Copy link
Copy Markdown
Contributor Author

Outputting the results of failed tests:

test_outputs.mp4

@Olshansk
Copy link
Copy Markdown
Collaborator

@deblasis Per my comment in discord, do you think it's possible to aggregate the outputs for one failing test in one output field?

@Olshansk Olshansk self-requested a review February 23, 2023 17:28
@deblasis
Copy link
Copy Markdown
Contributor Author

@deblasis Per my comment in discord, do you think it's possible to aggregate the outputs for one failing test in one output field?

image

I basically obtained the initial output with some jq and bash

Copy link
Copy Markdown
Contributor

@okdas okdas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love this, thank you for this change! It makes sense to include the whole stack of error.

@deblasis deblasis merged commit 6ef866b into main Feb 23, 2023
@Olshansk Olshansk deleted the issue/improve_test_failure_reporting branch February 23, 2023 20:35
bryanchriswhite added a commit that referenced this pull request Feb 24, 2023
* pokt/main:
  [Tooling] Returning failed tests full output (#542)
  [Tooling] Update CLI to support key management (Issue #489) (#524)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

infra Core infrastructure - not protocol related tooling tooling to support development, testing et al

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants