info message not printed by TestDirectory

[bh11]

The wiki page Status of standard tests in GAP packages states that it's sufficient to use TestDirectory in a package test file. However, as far as I can see, the required info message is only printed if TestDirectory is used with the exitGAP option. Am I overlooking something?

[olexandr-konovalov]

@bh11 you're right in this diagnosis - I've just discovered that trying to figure out why we can't detect pass/fail for @Stefan-Kohl's ResClasses.

If there will be no objections, for GAP 4.8.3 I will modify TestDirectory to print the required info message at any case, independently of the option used.

[fingolfin]

My guess for the reason why it does not printed the status code when exitGap is omitted: Because this way, you can add more tests aftere this, and only print the status code at the end.

But perhaps it would be better to make that explict, by adding a suppressStatusMessage option or so, which defaults to false.

Perhaps @ChrisJefferson has a comment?

[ChrisJefferson]

Overall, TestDirectory is designed to make testing the library and packages easier -- it's still in evolution, and we should make any changes that make people's lives easier :)

I think switching to always printing a message, and then having a suppressStatusMessage, seems like a sensible option.

[Stefan-Kohl]

A problem is also that the package author has no way to check whether the automated tests run through with the information provided in a package -- or is there any possibility to do such test?

[olexandr-konovalov]

@ChrisJefferson this sounds like a good choice - a single run of TestDirectory would then produce the info message, and multiple runs are normally called from some script, so there should be no problem to put suppressStatusMessage there. Assigned to myself for GAP 4.8.3

[olexandr-konovalov]

@ChrisJefferson I've just submitted #670 to add suppressStatusMessage option.

I've tested it with testinstall.g and with ResClasses from GAP 4.8.2 release. I now detect that ResClasses passed the test correctly (also, enjoyed new ways of testing a package with TestPackage("resclasses"); and with make testpackage PKGNAME=resclasses implemented in #647 to run the tests only for a single package).

[olexandr-konovalov]

Resolved in PR #670.