Replies: 9 comments 17 replies
-
|
I am asking here before canceling because I worry I am just impatient and it needs to run from the beginning again if I cancel it now. Edit: I still have 2GiB of RAM available while this is running. My next steps would probably be to run --verbose --verbose and display_processing_time = "true". I also found these related issues but they are from before the rewrite so maybe not relevant anymore? |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
I tested with many more directories ignored with Of course this was with only ~11G of my ~400G total by ignoring a lot of directories using skip_dir intentionally, so I think the sync is just super slow still because even those 11G took ~5hrs easily. A lot of processing batches look like this: |
Beta Was this translation helpful? Give feedback.
-
|
Regardless of using any Client Side Filtering, the client must obtain and pull down every online JSON object referent initially, to ensure what is needs to be synced is processed correctly. The 'skip_dir' option is also intensive in operation - larger CPU cost. Your best approach is to:
This will traverse elements in a more friendly manner.
The processing of ~260K objects takes time
Whilst this section of prior logging is not available .. 1.4 seconds to do something - tells me you have some other issue going on with your platform. This could be disk, memory or something else .. but if your system is starting to take seconds to perform operations .. its not this client that is being slow .. but rather your actual device underlying components that is the performance impediment. To spend ~ 300+ seconds processing 500 JSON items .. is also indicating that you have an underlying operating system performance issue. Please review how to generate system performance data for performance related issues: |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
Thank you for the reply, I changed to a sync_list file containing three directories and then also added a bunch of exclusions in the config using skip_dir. Can you tell why the hell the client decided to delete 50GiB of my files???? The logs just say it deletes it, the computer was running over night with the command log file: (can provide the full log file in private via DM if you need. config file: ~/.config/onedrive/sync_list: |
Beta Was this translation helpful? Give feedback.
-
|
As per my prior comment - please format your 'sync_list' correctly.
You should be using wildcard paths in 'sync_list' and setting up your exclusions in 'sync_list' and your existing 'skip_dir' values should essentially then be blank (no entry)
Without knowing exactly what you are doing - it is hard to say - however: For something local to initiate an online delete:
Again .. without knowing your exact sequence of events - I cannot advise what is occurring or what is happening. I will say this is really odd .. and this is not something that has been detailed / articulated before .. thus - at the moment the only speculation I have is that there is something going on with how you are running the client to see this odd behaviour. |
Beta Was this translation helpful? Give feedback.
0 replies
-
Your config comments are inaccurate. The specific entries to for obsidian support are not expensive API calls. This also has zero to do with API rate limiting. What do these options do?
Please refer to https://github.com/abraunegg/onedrive/blob/master/docs/application-config-options.md for accurate descriptions of the configuration options. This option is a 'no-op' in your scenario:
When set to 'true' this option triggers the following GUI notifications to occur:
|
Beta Was this translation helpful? Give feedback.
3 replies
-
|
First off, I want to say I really appreciate your work because I can finally leave windows because of you without migrating my files :) So please don't take this the wrong way! What do you need to determine the root cause of this? From my perspective, I can just say that I ran the command specified in my message above with the config and sync_list file also specified, came back the next morning and realized that tons of files are gone. The logs just say that the client didn't find them, so it deleted online. Would you agree that the logs indicate the client deleted the dirs because they were not present locally? I ran From what you said, I should consolidate my exclusion and inclusion into sync_list instead right? So the file below is what you would use? Additionally change config to: |
Beta Was this translation helpful? Give feedback.
-
|
Btw, gemini is saying that the bug occured due to me not using absolute paths in skip_dir, i.e. missing the `/´ in front of Documents |
Beta Was this translation helpful? Give feedback.
-
|
Gemini is incorrect ... AI tools hallucinate |
Beta Was this translation helpful? Give feedback.
-
|
When this message is displayed:
This comes from this section of code: 
The client at that point exited to preserve your online data and stopped all operations. If this was not what occurred - and files were still being deleted somehow:
The client also does not just delete data. I am 100% very clear on this. The only way the client deletes data is:
Explicitly for point 3:
This entire situation is very odd .. as I have reviewed the code in depth:
I am going to spend today using this configuration you advised you were running initially and see if I can replicate in any way your experience: 
|
Beta Was this translation helpful? Give feedback.
1 reply
-
|
Just to clarify regarding point 1 and data deletion, the warning about many files being deleted only applied to the last directory that was not deleted, so your code worked correctly. The other directories likely simply had too few files for the check to be true. I probably just misconfigured the combination of skip_dir and sync_list and that caused some inconsistent state as I ran it multiple times with the clean sync_list file for include and exclude and have not been able to replicate it, works like a charm! |
Beta Was this translation helpful? Give feedback.
-
|
So looking at your original configuration, your application configuration has problems that are contributing factors to what you are experiencing. 1. 'skip_dir' excludes the same top-level directories that 'sync_list' is trying to includeSpecifically:
An application improvement can be done so that when 'skip_dir' is being used, valid 'sync_list' entries are checked for shadowing so that the user has an opportunity to resolve so that elements are not shadowed based on Client Side Filtering rule processing order. UPDATE: This issue of 'skip_dir' entries shadowing 'sync_list' inclusion rules thus rendering them non-viable is fixed in PR #3577. Client version When using client version 2. 'skip_dir' is incorrectly formattedIn your first skip_dir line, you end the line with a 
This produced an empty entry that compiled to UPDATE: This issue of 'skip_dir' and 'skip_file' being incorrectly formatted is fixed via PR #3576. Client version |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
Amazing how quickly you move it, when I have the money I will donate for sure! Thanks so much for your work :) For anyone reading this, I simply used the sync_list and config in this comment and so far no other issues. However since this initial testing was within a VM, I will have to see if anything comes up later when I use it more heavily. |
Beta Was this translation helpful? Give feedback.
-
|
Oh one last thing and feel free to ignore because I did not have time to test upstream yet, so I just wanted to ask out of curiosity if you already have heard of this issue. A few times when running |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
That is certainly interesting ... Fedora 42 uses When this next occurs, run A known working version of curl is As per https://github.com/abraunegg/onedrive/blob/master/readme.md#basic-troubleshooting-steps the recommended pathway here is:
If these 2 options resolve your exit issue, then this points to a bad curl version, and that Fedora version needs to be added to the known bad curl versions. |
Beta Was this translation helpful? Give feedback.
-
|
I see, unfortunately i already upgraded to fedora 43, which should not have
this issue right? I will run a full resync with the same command and check
if it occurs again.
…On Tue, Dec 23, 2025, 18:50 abraunegg ***@***.***> wrote:
That is certainly interesting ...
Fedora 42 uses curl 8.11.1 .. I would need to look into the curl history
for that version as to HTTP/2 bugs and see if I have missed something.
When this next occurs, run echo $? after the unexpected exit may return
an error code of 141. If this does, then this points to an internal curl
issue.
A known working version of curl is 8.14.1, but that is not available for
Fedora 42.
As per
https://github.com/abraunegg/onedrive/blob/master/readme.md#basic-troubleshooting-steps
the recommended pathway here is:
- Test first with IPv4 only
- Test second with HTTP/1.1 and IPv4
If these 2 options resolve your exit issue, then this points to a bad curl
version, and that Fedora version needs to be added to the known bad curl
versions.
—
Reply to this email directly, view it on GitHub
<#3573 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AHV7IMTI6BUSJFYTQI64FQD4DF6G3AVCNFSM6AAAAACPVNJSQWVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTKMZSHA4TSNY>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
9 replies
-
|
Same, running latest |
Beta Was this translation helpful? Give feedback.
-
|
I am curious though, are there downsides to only using ipv4? |
Beta Was this translation helpful? Give feedback.
-
|
Short answer: Longer explanation: Running the client in IPv4-only mode:
Why IPv4-only is sometimes recommended:
|
Beta Was this translation helpful? Give feedback.
-
|
Update from the experiment on fedora 43 built from source with ipv4 and the previously mentioned config. I redacted it using an LLM, if you provide an email I can also send you the full log file: |
Beta Was this translation helpful? Give feedback.
-
|
Please raise a new bug report as this appears to be a problem with the WebSockets functionality that was introduced in v2.5.8/v2.5.9 |
Beta Was this translation helpful? Give feedback.
-
|
Going back your original configuration:
A 3rd issue identified was in the application of those 'skip_dir' rules and how those are evaluated. This required an entire overhaul of the function that performs that check and is fixed via PR #3580. Client version onedrive onedrive v2.5.9-27-g91b9cc6 or greater contains this fix. Resulting Application Configuration - your original (fixed) as used in local testing:Application Result
Directories excluded by 'skip_dir' are excluded, not seen as local, and all other data is uploaded|download as per expectation. Online State Example - Account Root
Online State Example - Documents
|
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hi, I am testing the client for use on fedora 42 with release 2.5.9 and I'm wondering if I am doing something wrong because the initial sync is still running after more than 16hrs. I simply ran
onedriveto authenticate, thenonedrive --monitor --dry-runand this has been running since yesterday with the following output. As you can see I have about 260k objects but I also haveskip_dirlines in my config that should ignore more than 95% of my files so that I can test quickly. Is it normal that it takes so long? I am running on a fairly modern laptop with a ~200Mbit/s connection:Beta Was this translation helpful? Give feedback.
All reactions