Skip to content
Open
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
a5822ec
[Fix][FTP]Fix 'file_filterpattern' cannot filter directories
Nov 9, 2025
bbdd493
[Fix][FTP]Fix 'file_filterpattern' cannot filter directories
Nov 10, 2025
46a2787
[Fix][FTP]Fix 'file_filterpattern' cannot filter directories
Nov 10, 2025
2e352bc
[Fix][FTP]Fix 'file_filterpattern' cannot filter directories
Nov 10, 2025
f64eede
[Fix][FTP]Fix 'file_filterpattern' cannot filter directories
Nov 10, 2025
f1aff6e
[Fix][FTP]Fix 'file_filterpattern' cannot filter directories
Nov 11, 2025
0bd638d
[Fix][FTP]Fix 'file_filterpattern' cannot filter directories
Nov 12, 2025
b70136c
[Fix][FTP]Fix 'file_filterpattern' cannot filter directories
Nov 13, 2025
768c0be
[Fix][FTP]Fix 'file_filterpattern' cannot filter directories
Nov 13, 2025
75c81f2
[Fix][FTP]Fix 'file_filterpattern' cannot filter directories
Nov 14, 2025
3028d99
[Fix][FTP]Fix 'file_filterpattern' cannot filter directories
Nov 14, 2025
6d6995a
[Fix][FTP]Fix 'file_filterpattern' cannot filter directories
Nov 14, 2025
15e15e2
[Fix][FTP]Fix 'file_filterpattern' cannot filter directories
Nov 14, 2025
9928688
[Fix][FTP]Fix 'file_filterpattern' cannot filter directories
Nov 15, 2025
a589d60
[Fix][FTP]Fix 'file_filterpattern' cannot filter directories
Nov 15, 2025
d96a651
[Fix][FTP]Fix 'file_filterpattern' cannot filter directories
Nov 19, 2025
d375235
[Fix][FTP]Fix 'file_filterpattern' cannot filter directories
Nov 22, 2025
08fd273
[Fix][FTP]Fix 'file_filterpattern' cannot filter directories
Nov 22, 2025
d7b4aff
[Fix][FTP]Fix 'file_filterpattern' cannot filter directories
Nov 22, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions docs/en/connector-v2/source/CosFile.md
Original file line number Diff line number Diff line change
Expand Up @@ -313,12 +313,12 @@ Whether to use the header line to parse the file, only used when the file_format

### file_filter_pattern [string]

Filter pattern, which used for filtering files.
Filter pattern, which used for filtering files. If you only want to filter based on file names, simply write the regular file names; If you want to filter based on the file directory at the same time, the expression needs to start with `path`.

The pattern follows standard regular expressions. For details, please refer to https://en.wikipedia.org/wiki/Regular_expression.
There are some examples.

File Structure Example:
If the `path` is `/data/setunnel`, and the file structure example is:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If the `path` is `/data/setunnel`, and the file structure example is:
If the `path` is `/data/seatunnel`, and the file structure example is:

```
/data/seatunnel/20241001/report.txt
/data/seatunnel/20241007/abch202410.csv
Expand All @@ -330,22 +330,22 @@ Matching Rules Example:

**Example 1**: *Match all .txt files*,Regular Expression:
```
/data/seatunnel/20241001/.*\.txt
.*.txt
```
The result of this example matching is:
```
/data/seatunnel/20241001/report.txt
```
**Example 2**: *Match all file starting with abc*,Regular Expression:
```
/data/seatunnel/20241002/abc.*
abc.*
```
The result of this example matching is:
```
/data/seatunnel/20241007/abch202410.csv
/data/seatunnel/20241002/abcg202410.csv
```
**Example 3**: *Match all file starting with abc,And the fourth character is either h or g*, the Regular Expression:
**Example 3**: *Match all files starting with abc in folder 20241007,And the fourth character is either h or g*, the Regular Expression:
```
/data/seatunnel/20241007/abc[h,g].*
```
Expand All @@ -355,7 +355,7 @@ The result of this example matching is:
```
**Example 4**: *Match third level folders starting with 202410 and files ending with .csv*, the Regular Expression:
```
/data/seatunnel/202410\d*/.*\.csv
/data/seatunnel/202410\d*/.*.csv
```
The result of this example matching is:
```
Expand Down
13 changes: 6 additions & 7 deletions docs/en/connector-v2/source/FtpFile.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,12 +106,12 @@ Whether to enable remote host verification for FTP data channels, default is `tr

### file_filter_pattern [string]

Filter pattern, which used for filtering files.
Filter pattern, which used for filtering files. If you only want to filter based on file names, simply write the regular file names; If you want to filter based on the file directory at the same time, the expression needs to start with `path`.

The pattern follows standard regular expressions. For details, please refer to https://en.wikipedia.org/wiki/Regular_expression.
There are some examples.

File Structure Example:
If the `path` is `/data/setunnel`, and the file structure example is:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto


```
/data/seatunnel/20241001/report.txt
Expand All @@ -126,7 +126,7 @@ Matching Rules Example:
**Example 1**: *Match all .txt files*,Regular Expression:

```
/data/seatunnel/20241001/.*\.txt
.*.txt
```

The result of this example matching is:
Expand All @@ -138,7 +138,7 @@ The result of this example matching is:
**Example 2**: *Match all file starting with abc*,Regular Expression:

```
/data/seatunnel/20241002/abc.*
abc.*
```

The result of this example matching is:
Expand All @@ -147,8 +147,7 @@ The result of this example matching is:
/data/seatunnel/20241007/abch202410.csv
/data/seatunnel/20241002/abcg202410.csv
```

**Example 3**: *Match all file starting with abc,And the fourth character is either h or g*, the Regular Expression:
**Example 3**: *Match all files starting with abc in folder 20241007,And the fourth character is either h or g*, the Regular Expression:

```
/data/seatunnel/20241007/abc[h,g].*
Expand All @@ -163,7 +162,7 @@ The result of this example matching is:
**Example 4**: *Match third level folders starting with 202410 and files ending with .csv*, the Regular Expression:

```
/data/seatunnel/202410\d*/.*\.csv
/data/seatunnel/202410\d*/.*.csv
```

The result of this example matching is:
Expand Down
12 changes: 6 additions & 6 deletions docs/en/connector-v2/source/HdfsFile.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,12 +118,12 @@ default `\n`

### file_filter_pattern [string]

Filter pattern, which used for filtering files.
Filter pattern, which used for filtering files. If you only want to filter based on file names, simply write the regular file names; If you want to filter based on the file directory at the same time, the expression needs to start with `path`.

The pattern follows standard regular expressions. For details, please refer to https://en.wikipedia.org/wiki/Regular_expression.
There are some examples.

File Structure Example:
If the `path` is `/data/setunnel`, and the file structure example is:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

```
/data/seatunnel/20241001/report.txt
/data/seatunnel/20241007/abch202410.csv
Expand All @@ -135,22 +135,22 @@ Matching Rules Example:

**Example 1**: *Match all .txt files*,Regular Expression:
```
/data/seatunnel/20241001/.*\.txt
.*.txt
```
The result of this example matching is:
```
/data/seatunnel/20241001/report.txt
```
**Example 2**: *Match all file starting with abc*,Regular Expression:
```
/data/seatunnel/20241002/abc.*
abc.*
```
The result of this example matching is:
```
/data/seatunnel/20241007/abch202410.csv
/data/seatunnel/20241002/abcg202410.csv
```
**Example 3**: *Match all file starting with abc,And the fourth character is either h or g*, the Regular Expression:
**Example 3**: *Match all files starting with abc in folder 20241007,And the fourth character is either h or g*, the Regular Expression:
```
/data/seatunnel/20241007/abc[h,g].*
```
Expand All @@ -160,7 +160,7 @@ The result of this example matching is:
```
**Example 4**: *Match third level folders starting with 202410 and files ending with .csv*, the Regular Expression:
```
/data/seatunnel/202410\d*/.*\.csv
/data/seatunnel/202410\d*/.*.csv
```
The result of this example matching is:
```
Expand Down
14 changes: 7 additions & 7 deletions docs/en/connector-v2/source/LocalFile.md
Original file line number Diff line number Diff line change
Expand Up @@ -295,12 +295,12 @@ Whether to use the header line to parse the file, only used when the file_format

### file_filter_pattern [string]

Filter pattern, which used for filtering files.
Filter pattern, which used for filtering files. If you only want to filter based on file names, simply write the regular file names; If you want to filter based on the file directory at the same time, the expression needs to start with `path`.

The pattern follows standard regular expressions. For details, please refer to https://en.wikipedia.org/wiki/Regular_expression.
There are some examples.

File Structure Example:
If the `path` is `/data/setunnel`, and the file structure example is:
```
/data/seatunnel/20241001/report.txt
/data/seatunnel/20241007/abch202410.csv
Expand All @@ -312,22 +312,22 @@ Matching Rules Example:

**Example 1**: *Match all .txt files*,Regular Expression:
```
/data/seatunnel/20241001/.*\.txt
.*.txt
```
The result of this example matching is:
```
/data/seatunnel/20241001/report.txt
```
**Example 2**: *Match all file starting with abc*,Regular Expression:
```
/data/seatunnel/20241002/abc.*
abc.*
```
The result of this example matching is:
```
/data/seatunnel/20241007/abch202410.csv
/data/seatunnel/20241002/abcg202410.csv
```
**Example 3**: *Match all file starting with abc,And the fourth character is either h or g*, the Regular Expression:
**Example 3**: *Match all files starting with abc in folder 20241007,And the fourth character is either h or g*, the Regular Expression:
```
/data/seatunnel/20241007/abc[h,g].*
```
Expand All @@ -337,7 +337,7 @@ The result of this example matching is:
```
**Example 4**: *Match third level folders starting with 202410 and files ending with .csv*, the Regular Expression:
```
/data/seatunnel/202410\d*/.*\.csv
/data/seatunnel/202410\d*/.*.csv
```
The result of this example matching is:
```
Expand Down Expand Up @@ -404,7 +404,7 @@ File modification time filter. The connector will filter some files base on the

### file_filter_modified_end [string]

File modification time filter. The connector will filter some files base on the last modification end time (not include end time). The default data format is `yyyy-MM-dd HH:mm:ss`.
File modification time filter. The connector will filter some files base on the last modification end time (not include end time). The default data format is `yyyy-MM-dd HH:mm:ss`.

### common options

Expand Down
12 changes: 6 additions & 6 deletions docs/en/connector-v2/source/OssFile.md
Original file line number Diff line number Diff line change
Expand Up @@ -265,12 +265,12 @@ Whether to read the complete file as a single chunk instead of splitting into ch

### file_filter_pattern [string]

Filter pattern, which used for filtering files.
Filter pattern, which used for filtering files. If you only want to filter based on file names, simply write the regular file names; If you want to filter based on the file directory at the same time, the expression needs to start with `path`.

The pattern follows standard regular expressions. For details, please refer to https://en.wikipedia.org/wiki/Regular_expression.
There are some examples.

File Structure Example:
If the `path` is `/data/setunnel`, and the file structure example is:
```
/data/seatunnel/20241001/report.txt
/data/seatunnel/20241007/abch202410.csv
Expand All @@ -282,22 +282,22 @@ Matching Rules Example:

**Example 1**: *Match all .txt files*,Regular Expression:
```
/data/seatunnel/20241001/.*\.txt
.*.txt
```
The result of this example matching is:
```
/data/seatunnel/20241001/report.txt
```
**Example 2**: *Match all file starting with abc*,Regular Expression:
```
/data/seatunnel/20241002/abc.*
abc.*
```
The result of this example matching is:
```
/data/seatunnel/20241007/abch202410.csv
/data/seatunnel/20241002/abcg202410.csv
```
**Example 3**: *Match all file starting with abc,And the fourth character is either h or g*, the Regular Expression:
**Example 3**: *Match all files starting with abc in folder 20241007,And the fourth character is either h or g*, the Regular Expression:
```
/data/seatunnel/20241007/abc[h,g].*
```
Expand All @@ -307,7 +307,7 @@ The result of this example matching is:
```
**Example 4**: *Match third level folders starting with 202410 and files ending with .csv*, the Regular Expression:
```
/data/seatunnel/202410\d*/.*\.csv
/data/seatunnel/202410\d*/.*.csv
```
The result of this example matching is:
```
Expand Down
10 changes: 5 additions & 5 deletions docs/en/connector-v2/source/OssJindoFile.md
Original file line number Diff line number Diff line change
Expand Up @@ -299,7 +299,7 @@ Reader the sheet of the workbook.

### file_filter_pattern [string]

Filter pattern, which used for filtering files.
Filter pattern, which used for filtering files. If you only want to filter based on file names, simply write the regular file names; If you want to filter based on the file directory at the same time, the expression needs to start with `path`.

The pattern follows standard regular expressions. For details, please refer to https://en.wikipedia.org/wiki/Regular_expression.
There are some examples.
Expand All @@ -316,22 +316,22 @@ Matching Rules Example:

**Example 1**: *Match all .txt files*,Regular Expression:
```
/data/seatunnel/20241001/.*\.txt
.*.txt
```
The result of this example matching is:
```
/data/seatunnel/20241001/report.txt
```
**Example 2**: *Match all file starting with abc*,Regular Expression:
```
/data/seatunnel/20241002/abc.*
abc.*
```
The result of this example matching is:
```
/data/seatunnel/20241007/abch202410.csv
/data/seatunnel/20241002/abcg202410.csv
```
**Example 3**: *Match all file starting with abc,And the fourth character is either h or g*, the Regular Expression:
**Example 3**: *Match all files starting with abc in folder 20241007,And the fourth character is either h or g*, the Regular Expression:
```
/data/seatunnel/20241007/abc[h,g].*
```
Expand All @@ -341,7 +341,7 @@ The result of this example matching is:
```
**Example 4**: *Match third level folders starting with 202410 and files ending with .csv*, the Regular Expression:
```
/data/seatunnel/202410\d*/.*\.csv
/data/seatunnel/202410\d*/.*.csv
```
The result of this example matching is:
```
Expand Down
12 changes: 6 additions & 6 deletions docs/en/connector-v2/source/S3File.md
Original file line number Diff line number Diff line change
Expand Up @@ -257,12 +257,12 @@ default `\n`

### file_filter_pattern [string]

Filter pattern, which used for filtering files.
Filter pattern, which used for filtering files. If you only want to filter based on file names, simply write the regular file names; If you want to filter based on the file directory at the same time, the expression needs to start with `path`.

The pattern follows standard regular expressions. For details, please refer to https://en.wikipedia.org/wiki/Regular_expression.
There are some examples.

File Structure Example:
If the `path` is `/data/setunnel`, and the file structure example is:
```
/data/seatunnel/20241001/report.txt
/data/seatunnel/20241007/abch202410.csv
Expand All @@ -274,22 +274,22 @@ Matching Rules Example:

**Example 1**: *Match all .txt files*,Regular Expression:
```
/data/seatunnel/20241001/.*\.txt
.*.txt
```
The result of this example matching is:
```
/data/seatunnel/20241001/report.txt
```
**Example 2**: *Match all file starting with abc*,Regular Expression:
```
/data/seatunnel/20241002/abc.*
abc.*
```
The result of this example matching is:
```
/data/seatunnel/20241007/abch202410.csv
/data/seatunnel/20241002/abcg202410.csv
```
**Example 3**: *Match all file starting with abc,And the fourth character is either h or g*, the Regular Expression:
**Example 3**: *Match all files starting with abc in folder 20241007,And the fourth character is either h or g*, the Regular Expression:
```
/data/seatunnel/20241007/abc[h,g].*
```
Expand All @@ -299,7 +299,7 @@ The result of this example matching is:
```
**Example 4**: *Match third level folders starting with 202410 and files ending with .csv*, the Regular Expression:
```
/data/seatunnel/202410\d*/.*\.csv
/data/seatunnel/202410\d*/.*.csv
```
The result of this example matching is:
```
Expand Down
Loading