Question mark character not treated as special in path segment

[dlong500]

What were you expecting to happen?

The string base/folder?/file1.txt should return base as the non-magic parent path

What actually happened?

globParent('base/folder?/file1.txt') returns base/folder?

Please provide the following information:

• OS & version: Windows 10 20H2, also Ubuntu 16.04
• node version (run node -v): 15.10
• yarn version: 1.22.5

Additional information

I've been trying to track down the source of my issue where the del library wasn't deleting expected folders based on a pattern like the one mentioned above (after del switched to using fast-glob). This led me to follow the dependency tree from del to fast-glob to glob-parent.

Unless I'm completely mistaken about how this library works I would expect a question mark to be treated like other special characters. For example, globParent('base/folder*/file1.txt') returns base.

[phated]

@dlong500 thanks for investigating this! It doesn't seem like we have a test for this case. If you'd want to send a test and possibly a fix, I can get that in our next major release. We are actually making a few fixes related to the globs not catching certain things, and we need to bump a major just in case someone relying on the mistaken behavior.

[dlong500]

@phated I can take a stab at it. So are you in agreement that my expected behavior is correct? It looks like the issue stems from is-glob returning false, and that module does have a strict flag that can be disabled. If I replace isGlob(str) with isGlob(str, { strict: false }) I get the expected result.

[phated]

@dlong500 I'm not exactly sure. The question mark seems to be an "extglob" which usually only matters when prefixed to a glob type pattern with parens.

Why do you think that question mark should be considered a globbing pattern when not part of an extglob? I'd need something to reference to determine.

I don't think we should turn off strict mode.

[dlong500]

I'm going based off of the syntax specified in the fast-glob documentation where a question mark is supposed to "match any single character except slashes". The del module that I had been using for a while was working fine to delete files based on a similar pattern (something like base/folder?/file?.txt), and then they switched from glob to fast-glob in del v5 at which point things broke in my app.

Maybe there is another way I can specify the same type of pattern using extglobs or something, but all I know is it was working with del version previous to v5, and the fast-glob documentation also seems to indicate being able to use question marks as a single character wildcard. You can see more details on my initial issue in the del repo that I linked to up above.

[phated]

Hmm, well major versions are supposed to contain breaking changes and if they weren't following actual glob "specs" (for as much as there is one) then I definitely could see how that wouldn't be compatible between different libraries. We've tried to stick to libraries that follow as closely to bash as possible.

[phated]

According to the is-glob tests at https://github.com/micromatch/is-glob/blob/master/test.js#L54-L56, they don't expect questionmark to be a glob. However, in the picomatch docs at https://github.com/micromatch/picomatch#basic-globbing, they do treat it as you described.

I think there could be a case for getting that matcher added to is-glob's regexp. Can you open a PR adding the test and fix to the strict regexp and we can discuss then I can get it merged and released?

[dlong500]

Are you saying I should open a PR in the is-glob repo? I can try that, though I'm not sure how people would feel about changing the strict behavior since they have a flag to disable strict mode (which is specifically there to allow for flexibility since there isn't really an actual glob spec).

Maybe there should be a strict option in glob-parent that corresponds to the is-glob strict option? Of course that means modules like fast-glob would need to also add such an option so it could be cascaded down through the dependencies...

[phated]

I'm a maintainer on is-glob. I checked the strict regexp and it just seems to be missing the generic questionmark path.

[dlong500]

I've had a tiny bit of time to work on this and it's easy to add the question mark character to the strict regex but that trips up a lot of the tests related to invalid regex patterns and capture groups. Without knowing the history of why all the existing tests are there I don't want to be messing with a bunch of other tests. What would you recommend is the best path forward?

[phated]

Yeah, I'd guess it would be a breaking change and the tests would need to be updated. We are in the process of making a breaking release to this library, too, so it could be upstreamed here.

I was mostly asking for the implementation/breaking tests proposed as a discussion point. I can update all the broken tests if everyone agrees that ? should be treated as a glob over there.

[jonschlinkert]

But that result is correct. I assume that if you want ? to work the way it works in regular expressions, you are trying to conditionally match the r in folder in this example. The correct way to achieve this is with a brace expression or an extglob:

'base/folde{r,}/file1.txt'
'base/folde?(r)/file1.txt'

This should also return the correct result with is-glob. The decision to not treat ? as a special character was based on bash glob, and other glob implementations, and the fact that question marks are valid path characters in most file systems besides Windows.

I don't usually have strong opinions on these kinds of decisions, I just follow the logic and read up on prior work to see where that leads me. If the consensus is that ? should be a special character, then we can start implementing that. However, it's worth considering that whenever we decide to go against established conventions we're potentially taking on lots of other edge cases that other people have dealt with or avoided over the years.

Let me know your thoughts.

[phated]

@jonschlinkert I'm fine with whatever we decide, but I'm curious why picomatch says that ? is a globbing pattern.

[dlong500]

I expect there can be some other way to accomplish what I'm after and I'm perfectly happy to leave things as-is if I can adjust my patterns/regex to work. What's confusing is the del library was working fine with the question mark representing a simple single character (non-extglob) wildcard prior to v5.

Here is a more detailed example of the type of thing I've been doing that works with del v4:
del('data/100-123?_files/@(0|1|2|3)')

This would delete folders like:

data/100-123A_files/0/
data/100-123A_files/1/
data/100-123A_files/2/
data/100-123A_files/3/
data/100-123B_files/0/
data/100-123B_files/1/
data/100-123B_files/2/
data/100-123B_files/3/

but not folders like:

data/100-123A_files/4/
data/100-123AX_files/0/

If I'm reading the docs correctly for extglobs, maybe I can achieve the same thing using a pattern like:
data/100-123@(?)_files/@(0|1|2|3)

Is that correct?