Skip to content

Clam 1288 Signature regex filter search optimization#837

Merged
val-ms merged 1 commit intoCisco-Talos:mainfrom
val-ms:CLAM-1288-regex-filter_search
Feb 22, 2023
Merged

Clam 1288 Signature regex filter search optimization#837
val-ms merged 1 commit intoCisco-Talos:mainfrom
val-ms:CLAM-1288-regex-filter_search

Conversation

@val-ms
Copy link
Contributor

@val-ms val-ms commented Feb 14, 2023

PDB, WDB, and CDB signatures can use a regex feature to match domain names. At one time in the process we ran a filter search to speed up filtering out non-matching static patterns but were accidentally discarding the result. When tested, it turns out it wasn't working correctly anyways.

Since then, we've fixed some bugs and upgraded the regex implementation to the latest version. After re-testing, I have found that the filter_search() appears to be working correctly now, both in the unit tests and with the existing CDB signatures.

PDB, WDB, and CDB signatures can use a regex feature to match
domain names. At one time in the process we ran a filter search
to speed up filtering out non-matching static patterns but were
accidentally discarding the result. When tested, it turns out
it wasn't working correctly anyways.

Since then, we've fixed some bugs and upgraded the regex
implementation to the latest version. After re-testing, I have
found that the filter_search() appears to be working correctly
now, both in the unit tests and with the existing CDB signatures.
@val-ms val-ms force-pushed the CLAM-1288-regex-filter_search branch from 590cc25 to a28d7f3 Compare February 15, 2023 20:21
@val-ms
Copy link
Contributor Author

val-ms commented Feb 22, 2023

I went back to figure out when the test was fixed. I didn't spent a lot of time testing, but I did find that with this change version 0.104 has the following test failure:

2: Running suite(s): cl_suite
2:  cli
2:  jsnorm
2:  str
2:  regex
2:  disasm
2:  unique
2:  matchers
2:  htmlnorm
2:  bytecode
2: 99%: Checks: 1176, Failures: 1, Errors: 0
2: /home/micasnyd/workspace/clamav-micah/unit_tests/check_regex.c:295:F:regex_list:regex_list_match_test:13: regex_list_match

That failure is gone as of 0.105, and of course is not present in main either.

Outside of some expected failures in the CI pipelines, everything looks good.

@val-ms val-ms merged commit d7317ca into Cisco-Talos:main Feb 22, 2023
@val-ms val-ms deleted the CLAM-1288-regex-filter_search branch February 22, 2023 01:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant