V15.4 startup performance degradation (DocumentUrlService?)

[oletolshave]

Which Umbraco version are you using?

15.4.2

Bug summary

For our website Umbraco hangs over 1 minute during stratup right after "Acquired MainDom". After a pause of approx. 1 minute and 17 seconds the startup resumes and everything runs normally.

During the time there is little to no IO or sql requests, but the CPU usage is maxed out (around 20%, but could be 100% on a single core).

This started right after upgrading to 15.4.2 from 15.3.1.

I have tried troubleshooting and I believe a likely cause is in DocumentUrlService. v15.4.0 introduced a call to "ConvertToCacheModel" for the published URL's.

We have around 200.000 records in [umbracoDocumentUrl]. In ConvertToCacheModel there is a call to GetModelFromCache, which scans for a given key in an in-memory list of urls. This has complexity O(N^2) and with N=200000 this could explain why just the CPU is busy during startup.

Specifics

https://github.com/umbraco/Umbraco-CMS/blob/main/src/Umbraco.Core/Services/DocumentUrlService.cs (lines method ConvertToCacheModel.

Steps to reproduce

0. (Somehow get thousands (200K) of records into table [umbracoDocumentUrl])
1. Start up Umbraco as normal.

Expected result / actual result

The intended outcome is the Umbraco spends no more then 5-10 seconds in the "Acquired MainDom" stage.

What actually happens is that Umbraco appears to be hanging during startup for over 1 minute in "Acquired MainDom".

[github-actions]

Hi there @oletolshave!

Firstly, a big thank you for raising this issue. Every piece of feedback we receive helps us to make Umbraco better.

We really appreciate your patience while we wait for our team to have a look at this but we wanted to let you know that we see this and share with you the plan for what comes next.

• We'll assess whether this issue relates to something that has already been fixed in a later version of the release that it has been raised for.
• If it's a bug, is it related to a release that we are actively supporting or is it related to a release that's in the end-of-life or security-only phase?
• We'll replicate the issue to ensure that the problem is as described.
• We'll decide whether the behavior is an issue or if the behavior is intended.

We wish we could work with everyone directly and assess your issue immediately but we're in the fortunate position of having lots of contributions to work with and only a few humans who are able to do it. We are making progress though and in the meantime, we will keep you in the loop and let you know when we have any questions.

Thanks, from your friendly Umbraco GitHub bot 🤖 🙂

[AndyButland]

Thanks for the report and the troubleshooting here @oletolshave - I can see the issue, replicated via a benchmark unit test and believe will be able to resolve via #19498.

[AndyButland]

Fixed via #19498 for 15.4.3 and 16.0.0,