Bug: sync_list does not suport "[" character (and I am guessing ] and others as well)

[Romadelf]

Describe the bug

I get a regex error when using the following single entry in sync_list :
/Sciences de l'ingénieur civil/Bachelier/Bloc 3/. Options/TODO (Q2) - [INFO9012-1] Parallel Programming/*

Operating System Details

Linux belalachnos 6.8.0-55-generic #57-Ubuntu SMP PREEMPT_DYNAMIC Wed Feb 12 23:42:21 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

No LSB modules are available.
Distributor ID:	Linuxmint
Description:	Linux Mint 22.1
Release:	22.1
Codename:	xia

Client Installation Method

From Source

OneDrive Account Type

Business | Office365

What is your OneDrive Application Version

onedrive v2.5.5-5-g0389483

What is your OneDrive Application Configuration

bwak@belalachnos:~$ onedrive --display-config
Reading configuration file: /home/bwak/.config/onedrive/config
Configuration file successfully loaded

WARNING: Your cURL/libcurl version (8.5.0) has known operational bugs that impact the use of this client.
         Please report this to your distribution, requesting an update to a newer cURL version, or consider upgrading it yourself for optimal stability.

Application version                          = onedrive v2.5.5-5-g0389483
Compiled with                                = LDC 2110
Curl version                                 = libcurl/8.5.0 OpenSSL/3.0.13 zlib/1.3.1 brotli/1.1.0 zstd/1.5.5 libidn2/2.3.7 libpsl/0.21.2 (+libidn2/2.3.7) libssh/0.10.6/openssl/zlib nghttp2/1.59.0 librtmp/2.3 OpenLDAP/2.6.7
User Application Config path                 = /home/bwak/.config/onedrive
System Application Config path               = /etc/onedrive
Applicable Application 'config' location     = /home/bwak/.config/onedrive/config
Configuration file found in config location  = true - using 'config' file values to override application defaults
Applicable 'sync_list' location              = /home/bwak/.config/onedrive/sync_list
Applicable 'items.sqlite3' location          = /home/bwak/.config/onedrive/items.sqlite3
Config option 'drive_id'                     = 
Config option 'sync_dir'                     = ~/OneDrive
Config option 'enable_logging'               = false
Config option 'log_dir'                      = /var/log/onedrive
Config option 'disable_notifications'        = false
Config option 'skip_dir'                     = 
Config option 'skip_dir_strict_match'        = false
Config option 'skip_file'                    = 
Config option 'skip_dotfiles'                = false
Config option 'skip_symlinks'                = false
Config option 'monitor_interval'             = 300
Config option 'monitor_log_frequency'        = 12
Config option 'monitor_fullscan_frequency'   = 12
Config option 'read_only_auth_scope'         = false
Config option 'dry_run'                      = false
Config option 'upload_only'                  = false
Config option 'download_only'                = false
Config option 'local_first'                  = false
Config option 'check_nosync'                 = false
Config option 'check_nomount'                = false
Config option 'resync'                       = false
Config option 'resync_auth'                  = false
Config option 'cleanup_local_files'          = false
Config option 'disable_permission_set'       = false
Config option 'transfer_order'               = default
Config option 'classify_as_big_delete'       = 1000
Config option 'disable_upload_validation'    = false
Config option 'disable_download_validation'  = false
Config option 'bypass_data_preservation'     = false
Config option 'no_remote_delete'             = false
Config option 'remove_source_files'          = false
Config option 'sync_dir_permissions'         = 700
Config option 'sync_file_permissions'        = 600
Config option 'space_reservation'            = 52428800
Config option 'permanent_delete'             = false
Config option 'write_xattr_data'             = false
Config option 'application_id'               = d50ca740-c83f-4d1b-b616-12c519384f0c
Config option 'azure_ad_endpoint'            = 
Config option 'azure_tenant_id'              = 
Config option 'user_agent'                   = ISV|abraunegg|OneDrive Client for Linux/v2.5.5-5-g0389483
Config option 'force_http_11'                = true
Config option 'debug_https'                  = false
Config option 'rate_limit'                   = 0
Config option 'operation_timeout'            = 3600
Config option 'dns_timeout'                  = 60
Config option 'connect_timeout'              = 10
Config option 'data_timeout'                 = 60
Config option 'ip_protocol_version'          = 1
Config option 'threads'                      = 8
Config option 'max_curl_idle'                = 120
Environment var 'XDG_RUNTIME_DIR'            = true
Environment var 'DBUS_SESSION_BUS_ADDRESS'   = true
Config option 'notify_file_actions'          = false

Selective sync 'sync_list' configured        = true
sync_list config option 'sync_root_files'    = false
sync_list contents:
------------------------------'sync_list'------------------------------
/Sciences de l'ingénieur civil/Bachelier/Bloc 3/. Options/TODO (Q2) - [INFO9012-1] Parallel Programming/*
########## Personnal reminder : see  docs in repo at docs/usage.md ##########
-----------------------------------------------------------------------

Config option 'sync_business_shared_items'   = false

Config option 'webhook_enabled'              = false
bwak@belalachnos:~$ 

What is your 'curl' version

bwak@belalachnos:~$ curl --version
curl 8.5.0 (x86_64-pc-linux-gnu) libcurl/8.5.0 OpenSSL/3.0.13 zlib/1.3 brotli/1.1.0 zstd/1.5.5 libidn2/2.3.7 libpsl/0.21.2 (+libidn2/2.3.7) libssh/0.10.6/openssl/zlib nghttp2/1.59.0 librtmp/2.3 OpenLDAP/2.6.7
Release-Date: 2023-12-06, security patched: 8.5.0-2ubuntu10.6
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM PSL SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd

Where is your 'sync_dir' located

Local

What are all your system 'mount points'

bwak@belalachnos:~$ mount
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,nosuid,relatime,size=3744968k,nr_inodes=936242,mode=755,inode64)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,noexec,relatime,size=757196k,mode=755,inode64)
efivarfs on /sys/firmware/efi/efivars type efivarfs (rw,nosuid,nodev,noexec,relatime)
/dev/sda2 on / type ext4 (rw,relatime,errors=remount-ro)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,inode64)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k,inode64)
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=32,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=323)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,nosuid,nodev,relatime,pagesize=2M)
debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime)
tracefs on /sys/kernel/tracing type tracefs (rw,nosuid,nodev,noexec,relatime)
fusectl on /sys/fs/fuse/connections type fusectl (rw,nosuid,nodev,noexec,relatime)
configfs on /sys/kernel/config type configfs (rw,nosuid,nodev,noexec,relatime)
/dev/sda1 on /boot/efi type vfat (rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro)
/dev/sdb3 on /timeshift type ext4 (rw,relatime)
/dev/sdb1 on /home type ext4 (rw,relatime)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,nosuid,nodev,noexec,relatime)
tmpfs on /run/user/1000 type tmpfs (rw,nosuid,nodev,relatime,size=757192k,nr_inodes=189298,mode=700,uid=1000,gid=1000,inode64)
gvfsd-fuse on /run/user/1000/gvfs type fuse.gvfsd-fuse (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)
portal on /run/user/1000/doc type fuse.portal (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)

What are all your local file system partition types

bwak@belalachnos:~$ lsblk -f
NAME   FSTYPE FSVER LABEL    UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
sda                                                                              
├─sda1 vfat   FAT32          1F03-23F3                             500,6M     2% /boot/efi
└─sda2 ext4   1.0            0f3fbe06-91f9-4f92-a70a-cbfd43f98e08   97,2G    11% /
sdb                                                                              
├─sdb1 ext4   1.0   Parmesan 0804bca6-f221-48f7-b010-4525b41456ad  649,4G     0% /home
└─sdb3 ext4   1.0   Backer   decc4eaa-d191-4d38-bfef-bcb8bc9dad32   69,2G    19% /timeshift

How do you use 'onedrive'

Not relevant I think. I'll just say that I just installed linux and onedrive client. I wanted to download content from university of another computer using onedrive before. As it was long (> 10GB), I decided to start with the aforementioned folder only.

Steps to reproduce the behaviour

Install my linux OS normally
Then install onedrive following install.md
Create an sync_list file with entry
/Sciences de l'ingénieur civil/Bachelier/Bloc 3/. Options/TODO (Q2) - [INFO9012-1] Parallel Programming/*

Then follow usage.md up to doing:
onedrive -s -v --resync --dry-run

Complete Verbose Log Output

bwak@belalachnos:~$ onedrive -s -v --resync --dry-run
Reading configuration file: /home/bwak/.config/onedrive/config
Configuration file successfully loaded
Using 'user' configuration path for application config and state data: /home/bwak/.config/onedrive
D-Bus message bus daemon is available; GUI notifications are now enabled

WARNING: Your cURL/libcurl version (8.5.0) has known operational bugs that impact the use of this client.
         Please report this to your distribution, requesting an update to a newer cURL version, or consider upgrading it yourself for optimal stability.

DRY-RUN Configured. Output below shows what 'would' have occurred.
DRY-RUN: No database copy created for --dry-run due to --resync also being used

The usage of --resync will delete your local 'onedrive' client state, thus no record of your current 'sync status' will exist.
This has the potential to overwrite local versions of files with perhaps older versions of documents downloaded from OneDrive, resulting in local data loss.
If in doubt, backup your local data before using --resync

Are you sure you wish to proceed with --resync? [Y/N] y

DRY RUN: Not removing hash files as --dry-run has been used
DRY RUN: Not creating backup config file as --dry-run has been used
DRY RUN: Not updating hash files as --dry-run has been used
Deleting the saved application sync status ...
DRY RUN: Not removing the saved application sync status
Forcing client to use IPv4 connections only
Attempting to contact Microsoft OneDrive Login Service
Successfully reached Microsoft OneDrive Login Service
Checking Application Version ...
Attempting to initialise the OneDrive API ...
Configuring Global Azure AD Endpoints
The OneDrive API was initialised successfully
Opening the item database ...
Downgrading all HTTP operations to HTTP/1.1 due to user configuration
Application Version:   onedrive v2.5.5-5-g0389483
Account Type:          business
Default Drive ID:      b!l6Tn2R2mdk2_2k2GdIOJzF2c3E_5Sm9Dm3X8n60Ct5q8jZou74WmTqLwFM1n1qXw
Default Root ID:       01MZ4NZHF6Y2GOVW7725BZO354PWSELRRZ
Microsoft Data Centre: West Europe
Remaining Free Space:  977.18 GB (1049240488074 bytes)
Sync Engine Initialised with new Onedrive API instance
All application operations will be performed in the configured local 'sync_dir' directory: /home/bwak/OneDrive
Fetching /delta response from the OneDrive API for Drive ID: b!l6Tn2R2mdk2_2k2GdIOJzF2c3E_5Sm9Dm3X8n60Ct5q8jZou74WmTqLwFM1n1qXw
Processing API Response Bundle: 1 - Quantity of 'changes|items' in this bundle to process: 202
Skipping path - excluded by sync_list config: ./Conseils et outils généraux
Skipping path - excluded by sync_list config: ./Conseils et outils généraux/Belnet FileSender
Skipping path - excluded by sync_list config: ./Conseils et outils généraux/Guidance Etude - Bien démarrer son année 2021
Skipping path - excluded by sync_list config: ./Conseils et outils généraux/Guidance Etude - Bien démarrer son année 2021/0.Faire le point-questionnaires
Skipping path - excluded by sync_list config: ./Conseils et outils généraux/Guidance Etude - Bien démarrer son année 2021/1.Mémoriser
Skipping path - excluded by sync_list config: ./Conseils et outils généraux/Guidance Etude - Bien démarrer son année 2021/2.Prendre des notes
Skipping path - excluded by sync_list config: ./Conseils et outils généraux/Guidance Etude - Bien démarrer son année 2021/3.Planifier son travail
Skipping path - excluded by sync_list config: ./Conseils et outils généraux/Guidance Etude - Bien démarrer son année 2021/4.Prendre en main les outils numériques
Skipping path - excluded by sync_list config: ./Conseils et outils généraux/Outils chimie
Skipping path - excluded by sync_list config: ./Conseils et outils généraux/Service d'Orientation Universitaire
Skipping path - excluded by sync_list config: ./Sciences de l'ingénieur civil
Skipping path - excluded by sync_list config: ./Sciences de l'ingénieur civil/Administratif
Skipping path - excluded by sync_list config: ./Sciences de l'ingénieur civil/Administratif/2021-2022
Skipping path - excluded by sync_list config: ./Sciences de l'ingénieur civil/Administratif/2021-2022/Bourse
Skipping path - excluded by sync_list config: ./Sciences de l'ingénieur civil/Administratif/2021-2022/Bourse/Demande initiale
Skipping path - excluded by sync_list config: ./Sciences de l'ingénieur civil/Administratif/2021-2022/Bourse/Documents demandés ultérieurement
Skipping path - excluded by sync_list config: ./Sciences de l'ingénieur civil/Administratif/2021-2022/Bourse/Réclamation
Skipping path - excluded by sync_list config: ./Sciences de l'ingénieur civil/Administratif/2022-2023
Skipping path - excluded by sync_list config: ./Sciences de l'ingénieur civil/Administratif/2022-2023/Bourse
Skipping path - excluded by sync_list config: ./Sciences de l'ingénieur civil/Administratif/2022-2023/Bourse/Demande initiale
Skipping path - excluded by sync_list config: ./Sciences de l'ingénieur civil/Administratif/2023-2024
Skipping path - excluded by sync_list config: ./Sciences de l'ingénieur civil/Administratif/2023-2024/Résultats PAE
Skipping path - excluded by sync_list config: ./Sciences de l'ingénieur civil/Bachelier
Skipping path - excluded by sync_list config: ./Sciences de l'ingénieur civil/Bachelier/Bloc 1
Skipping path - excluded by sync_list config: ./Sciences de l'ingénieur civil/Bachelier/Bloc 1/. Examen d'entrée
Skipping path - excluded by sync_list config: ./Sciences de l'ingénieur civil/Bachelier/Bloc 1/[CHIM9272-2] Chimie 1
Skipping path - excluded by sync_list config: ./Sciences de l'ingénieur civil/Bachelier/Bloc 1/[CHIM9272-2] Chimie 1/Examens
Skipping path - excluded by sync_list config: ./Sciences de l'ingénieur civil/Bachelier/Bloc 1/[CHIM9272-2] Chimie 1/Projets et labo
Attempting to perform a database vacuum to optimise database
Database vacuum is complete
std.regex.internal.ir.RegexException@/home/bwak/dlang/ldc-1.40.1/bin/../import/std/regex/internal/parser.d(1020): inverted range
Pattern with error: `/Sciences\sde\sl'ingénieur\scivil/Bachelier/Bloc\s3/\.\sOptions/TODO\s(Q2)\s-\s[` <--HERE-- `INFO9012-1]\sParallel\sProgramming/.*`
----------------
??:? [0x65021556f95e]
??:? [0x65021556f5c2]
??:? [0x65021559213e]
??:? [0x650215578f0c]
/home/bwak/dlang/ldc-1.40.1/bin/../import/std/regex/internal/parser.d:1020 [0x6502153b61a0]
/home/bwak/dlang/ldc-1.40.1/bin/../import/std/regex/internal/parser.d:559 [0x6502153b5cb1]
/home/bwak/dlang/ldc-1.40.1/bin/../import/std/regex/package.d:473 [0x6502153b5bec]
/home/bwak/dlang/ldc-1.40.1/bin/../import/std/functional.d:1386 [0x6502153b5226]
/home/bwak/dlang/ldc-1.40.1/bin/../import/std/regex/package.d:422 [0x6502153b4ffc]
/home/bwak/dlang/ldc-1.40.1/bin/../import/std/regex/package.d:429 [0x6502152e30be]
src/clientSideFiltering.d:529 [0x6502152e49e2]
src/clientSideFiltering.d:174 [0x6502152df684]
src/sync.d:5571 [0x6502153141c8]
src/sync.d:1792 [0x65021530a710]
src/sync.d:1390 [0x6502152ff1b5]
src/sync.d:843 [0x6502152fd211]
src/main.d:1374 [0x6502153ea876]
src/main.d:858 [0x6502153e7334]
??:? [0x650215578bec]
??:? [0x650215578ae7]
??:? [0x65021557893c]
/home/bwak/dlang/ldc-1.40.1/bin/../import/core/internal/entrypoint.d:42 [0x650215402001]
??:? [0x77bb33e2a1c9]
??:? __libc_start_main [0x77bb33e2a28a]
??:? [0x6502152c0004]
bwak@belalachnos:~$

Screenshots

No response

Other Log Information or Details

The logging is of great quality so it seems to be an easy find (as well as code comments in the source, great job !).
So I think everything that needs to be known is in that log entry:

Pattern with error: `/Sciences\sde\sl'ingénieur\scivil/Bachelier/Bloc\s3/\.\sOptions/TODO\s(Q2)\s-\s[` <--HERE-- `INFO9012-1]\sParallel\sProgramming/.*`

Side note : should I issue another issue for the '\s' in the log instead of expected ' ' ? Not the point of this issue but realising this minor thing on the way...

Additional context

No response

[Romadelf]

Additional test :
I tried to escape the faulty characters with using \[ and \] like this in my /home/[me]/.config/onedrive/sync_list file:
/Sciences de l'ingénieur civil/Bachelier/Bloc 3/. Options/TODO (Q2) - \[INFO9012-1\] Parallel Programming/*

Well, I don't get the regex error anymore, but then the folder remains excluded as seen in the logs:

Skipping path - excluded by sync_list config: ./Sciences de l'ingénieur civil/Bachelier/Bloc 3/. Options/TODO (Q2) - [INFO9012-1] Parallel Programming/tuto
Skipping path - excluded by sync_list config: ./Sciences de l'ingénieur civil/Bachelier/Bloc 3/. Options/TODO (Q2) - [INFO9012-1] Parallel Programming/tuto/3
Skipping path - excluded by sync_list config: ./Sciences de l'ingénieur civil/Bachelier/Bloc 3/. Options/TODO (Q2) - [INFO9012-1] Parallel Programming/Hands On 2 - OpenMP
Skipping path - excluded by sync_list config: ./Sciences de l'ingénieur civil/Bachelier/Bloc 3/. Options/TODO (Q2) - [INFO9012-1] Parallel Programming/Hands On 2 - OpenMP/06-examples
Skipping path - excluded by sync_list config: ./Sciences de l'ingénieur civil/Bachelier/Bloc 3/. Options/TODO (Q2) - [INFO9012-1] Parallel Programming/Hands On 2 - OpenMP/06-examples/.git
Skipping path - excluded by sync_list config: ./Sciences de l'ingénieur civil/Bachelier/Bloc 3/. Options/TODO (Q2) - [INFO9012-1] Parallel Programming/Hands On 2 - OpenMP/06-examples/.git/hooks
Skipping path - excluded by sync_list config: ./Sciences de l'ingénieur civil/Bachelier/Bloc 3/. Options/TODO (Q2) - [INFO9012-1] Parallel Programming/Hands On 2 - OpenMP/06-examples/.git/objects/f5
Skipping path - excluded by sync_list config: ./Sciences de l'ingénieur civil/Bachelier/Bloc 3/. Options/TODO (Q2) - [INFO9012-1] Parallel Programming/Hands On 2 - OpenMP/06-examples/.git/refs
Skipping path - excluded by sync_list config: ./Sciences de l'ingénieur civil/Bachelier/Bloc 3/. Options/TODO (Q2) - [INFO9012-1] Parallel Programming/Hands On 2 - OpenMP/06-examples/.git/refs/heads

// [[TL;DR:] [and it keeps going that way]]

[abraunegg]

@Romadelf
This needs a programmatic change to resolve. Please stop your investigations or trying to solve the issue until a code fix is done.

[abraunegg]

@Romadelf

Please can you test the following PR to potentially resolve your issue.

To test the PR, first install all the require platform dependencies to build the client on your respective platforms. Please read https://github.com/abraunegg/onedrive/blob/master/docs/install.md#building-from-source---high-level-requirements and then follow correctly for your platform.

Important

Currently there is an issue with installing the DMD or LDC compiler using the documentation instructions:
curl -fsS https://dlang.org/install.sh | bash -s dmd or curl -fsS https://dlang.org/install.sh | bash -s ldc

This is not something I can solve and I have emailed the appropriate folk and raised dlang/dmd#20830

Please augment the installation of DMD or LDC with your distribution version if it is available

Once this is done, to clone the PR to resolve your issue, you can use a script like the following:

#!/bin/bash

PR=3184

rm -rf ./onedrive-pr${PR}
git clone https://github.com/abraunegg/onedrive.git onedrive-pr${PR}
cd onedrive-pr${PR}
git fetch origin pull/${PR}/head:pr${PR}
git checkout pr${PR}

# Configure and Build
./configure --enable-debug --enable-notifications; make clean; make;
./onedrive --version

This script will create a local folder called onedrive-pr3184 with the PR version.

To run the PR, you need to run the client from the PR build directory:

./onedrive <any other options needed>

To install the PR, you will need to perform sudo make install to install the PR version to your system.

When running the PR, your version should be: onedrive v2.5.5-4-gbca5d63 or greater.

If you still encounter an issue, please generate a new verbose debug log following this process: https://github.com/abraunegg/onedrive/wiki/Generate-debug-log-for-support