AI when enabled, makes debugging in VS 2019 2-2.5 times slower

[yahorsi]

Hi Guys,

I'm not sure this is the right place for that issue but let's at least start here.
We have regular .NET Core 2.2 SPA APP, we use SQL Server & SignalR and do not do anything extra ordinary. When we debug in the VS all requests from ther client (browser) takes huuuuge amount of time, seconds. When code runs without debugger attached everything is just flying. I started investigating and found that if I disable AI in the VS->Tools->Options->IntelliTrace->General Enable Intelli Trace things start working almost like without having debugger. We observe same behaviour on all our dev machines (about 15). So, as for me, that is usability bug. I'm happy to help to investigate, could provide any extra info needed. Feel free co contact me in skype yahor.si igf needed.

Thanks

[cijothomas]

IntelliTrace and ApplicationInsights are separate products. Though the title say AI, the descriptions and steps are about IntelliTrace.
Can you confirm if this is intellitrace? I'll tag someone with VS knowledge to send this issue to the right place based on that.

[yahorsi]

Sorry, that's intellitrace

[gardnerjr]

To identify if this has anything to do with App Insights, re-enable IntelliTrace, but then go to Tools -> extensions and disable the "Developer Analytics Tools" extension.

• If debugging is fine with IntelliTrace enabled and AppInsights disabled, this could be an issue with the AI debug extension in VS.
• if debugging is still bad with AppInsights disabled, the quickest way to get help from the right people is to submit feedback from inside VS itself via Help -> send feedback -> report a problem

[yahorsi]

Developer Analytics Tools

Today I had a bit of time, I disabled "Developer Analytics Tools" (btw it is not in Tools menu, it is in Extensions top menu). But perf is STILL really really horrible. So, perf problems are not caused by Developer Analytics Tools but by IntelliTrace itself.

But I should say, VS & IntelliTrace & still AI are on the quality state that I cannot even imagine.
It is completely unusable and it is not only from perf point of view.
I tried today to really use IntelliTrace and AI buit in VS to build request processing timeline and I should say, this thing is not used by onyone and not tested by anyone coz it just does not work.
When I enable IntelliTrace events, requests start taking seconds, it's like literally 4-10 seconds while befoe it was like 100-200ms. How that is possible at all (( And, even working so slow, the result, events and AI viewer built in VS are still unusable

And by the way, I was unable to report the bug(s) because of the bug in the bug reporting (((((
It just fails to authenticate me, while I'm successfully logged in in VS.

I'm dying ((((
Need VS quality engeneer to demo all the problems I've got

[yahorsi]

Ok, lets get back to that :)
As I wrote above, the biggest contributer to the very slow debugging performance is IntelliTrace. When we just disable IntelliTrace response time (under debugging in VS) goes from seconds (1-20) to under second 600ms - 800ms
See https://developercommunity.visualstudio.com/content/problem/620245/intellitrace-very-slow-debugging-expirience.html

But, then, if we disable ApplicationInsights in code, e.g. just comment UseApplicationInsights call WebHostBuilder, we see another ~2 to 2.5 times response time improvement (again under debugging).

I think paying 2-2.5 times is too much for having AI

PS: Please let me know if anyone is still following that issue, or I need to close this one and create new one.
PPS: I'll edit the title according to findings

// CC just to make sure someone will have a look at it once again )
СС @gardnerjr CC @Dmitry-Matveev

[Dmitry-Matveev]

Application Insights SDK does behave differently when it's used in VS Debug scenarios and indeed adds an additional overhead:

• AI SDK would write Debug.Output with all event information;
• AI SDK will push copy of this data to Event Source id Debug listeners are enabled for Event Sources (default in VS debug, I guess);
• AI SDK will write Event Source messages about each execution step if process-level listeners are enabled (I guess that's also default in VS Debug);
• Finally, DeveloperMode=true will be automatically enabled in AI SDK for Debug mode to make sure that each and every Telemetry item is not batched and is sent instead as soon as they are produced.

Overall, a series of optimizations in AI SDK is off in this case and is expected to be that way to simplify debugging (with the exception of slowing debugging down a little as you noticed).

You can control Configuration.TelemetryChannel.DeveloperMode setting for sure. I imagine there is a setting in VS Debug that may allow to disable Event Source listeners as well. I don't think Debug.Output can be disabled easily, though.

Unfortunately, I cannot speak to any possible issues with VS and Intellitrace, that portion of the issue should be moved accordingly to the places where these teams track items and/or UserVoice.

[yahorsi]

Thank you for the answer.

Is't is possible to do the things you mentioned but async?
Like writing to the Debug.Output and so on?
Even taking what you sad I still think the price of being 2.5x slower under debugger is IMHO to big.

[yahorsi]

Unfortunately, I cannot speak to any possible issues with VS and Intellitrace, that portion of the issue should be moved accordingly to the places where these teams track items and/or UserVoice.

I've already created ticked for the VS team with recorded repo
https://developercommunity.visualstudio.com/content/problem/620245/intellitrace-very-slow-debugging-expirience.html

[Dmitry-Matveev]

Developer Mode to false should make most of the operations like serialization and sending events, async.

Debug.Output, though would still be typical Debug.Output as well as any Event Source subscriptions.
I can advise on customizing ai.config and Telemetry Configuraiton options (e.g. you can try DisableTelemetry=true; in #if DEBUG scenarios).

Not sure if there is way to radically improve/change the established integration protocol with VS Debug tools, though. @gardnerjr , is there something else I'm missing that may help here or that we can easily fix without a lot of rework?

[yahorsi]

What we do right now is simple - we turn AI off (or better say do not turn AI on) for Development environment. One more reason why - is because we gave up trying to use it in real live in dev environment. E.g. we're unable to use it to analyse requests timeline in VS.

[gardnerjr]

how much console and telemetry output do you have that it is making debugging 2x slower? 2x slower to me implies that there's a LOT of console output and telemetry that has to be processed to see if it is telemetry, and then if it is telemetry to do the things the AI extension does to analyze it. The VS extension itself is full of AI telemetry so we debug the extension with AI turned on all the time and I don't know I've ever noticed debugging being 2x as slow.

if you still want telemetry but NOT any of the debug time features of the AI extension, you can turn off the console output of telemetry (documented elsewhere), which will effectively make it so that the extension doesn't see any telemetry so it doesn't do anything. telemetry would still be sent by AI though while debugging.

or you could explicitly set developermode to false to get the retail behavior instead of debug behavior?

[github-actions]

This issue is stale because it has been open 300 days with no activity. Remove stale label or comment or this will be closed in 7 days.