Replace direct telemetry deps with the electric_telemetry library #3412
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #3412 +/- ##
==========================================
+ Coverage 67.54% 75.00% +7.46%
==========================================
Files 180 50 -130
Lines 9672 2657 -7015
Branches 360 382 +22
==========================================
- Hits 6533 1993 -4540
+ Misses 3137 662 -2475
Partials 2 2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
✅ Deploy Preview for electric-next ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
Why not keep https://github.com/electric-sql/electric-telemetry as part of our mono-repo as another elixir project? Easier to access and work with. |
|
@msfstef Main reason is decoupling. Our primary use case for telemetry is in Electric Cloud. So if we're going to make any changes to the telemetry code, it'll be informed by Cloud and should be tested first in Cloud. All while Electric keeps using its own stable version of the lib. How can we achieve that?
Moving electric-telemetry into Electric's repo would work fine with the 2nd option but I'm against publishing our telemetry as a Hex package since it is only ever going to work as part of Electric. With the 1st option it becomes awkward: we cannot use electric-telemetry in Electric as a path dependency because then Electric will necessarily always use the latest version of it. And the alternative is to pull it from Github (i.e. |
alco
force-pushed
the
alco/electric-telemetry
branch
4 times, most recently
from
66f1c97 to
8fedf3d
Compare
There was a problem hiding this comment.
We have a bunch of tests still in place for these, and I suspect they are still passing because we're reusing the Electric.Telemetry namespace, whereas I think this should now be in ElectricTelemetry, and the tests should move to the new repo, no?
alco
force-pushed
the
alco/electric-telemetry
branch
3 times, most recently
from
e9756ff to
4f74858
Compare
commit: |
alco
force-pushed
the
alco/electric-telemetry
branch
6 times, most recently
from
dc3c87c to
e3b5328
Compare
These settings must be set in Electric and in Cloud, so using a single function helps avoid discrepancies.
This way, the electric_telemetry dependency can be included or excluded as a whole. And conditional compilation is only relevant to Electric anyway.
The telemetry library doesn't actually use Sentry. Cloud has its own setup for it, so electric_telemetry is not the right place for it.
Once telemetry is enabled via MIX_TARGET=application, making individual deps optional only puts the burden on the parent app to include them directly. We want it the other way around: let the telemetry library define its deps and fetch them.
The way to enable electric_telemetry is to set MIX_TARGET=application. In Cloud it is always enabled anyway.
As suggested by Stefanos, this makes it less confusing which modules belong to which package.
This is mirror what's already been done for CallHomeReporter
alco
force-pushed
the
alco/electric-telemetry
branch
from
e3b5328 to
a8fa894
Compare
This way they are easier to share between Electric and Cloud
alco
force-pushed
the
alco/electric-telemetry
branch
from
a8fa894 to
acf1224
Compare
It is now used by electric_telemetry
This version fixes an issue with unintentional overriding of OTLP headers
Chasing a bug in metric export via Otel...
alco
force-pushed
the
alco/electric-telemetry
branch
from
732c6de to
93eed43
Compare
Extract telemetry code from Electric into a separate package. On top of that, the new package accepts additional user-provided periodic measurements and metric definitions for otel/prometheus exporters; both exporters now use the same list of metrics.
One thing that's been lost in this move is the
stack_idsentry tag inside the CallHomeRepoter process. Sentry is a dependency of Electric and there's no straightforward way of propagating it into CallHomeReporter which is a module in the electric_telemetry dependency now. I don't feel like it's a big loss because we set stack_id in the logger metadata for that process, as well as mention it in the process name and label.With this merged, we can delete the electric-sql/electric-telemetry repo.