Instrumentation: port Datadog's auto-instrumentation libraries

[fbogsany]

See open-telemetry/oteps#41
Relevant code is here: https://github.com/DataDog/dd-trace-rb (instrumentation here: https://github.com/DataDog/dd-trace-rb/tree/master/lib/ddtrace/contrib).

[elskwid]

@fbogsany I'm digging in to the DataDog Integration instrumentation libraries. Not sure how you go about "assigning" these issues but I'll take this one. I'll post my thoughts here after I've had a chance to read the docs and spend a little time with the way DD has set up the integrations.

[fbogsany]

Not sure how you go about "assigning" these issues but I'll take this one.

❤️

Looks like only codeowners can assign issues to themselves, which is a little annoying for everyone else. We'll have to do it informally like this for now.

[fbogsany]

Oh, I'm wrong, I think - I assigned you.

[fbogsany]

Experimentation suggests that if you comment on an issue, then it can be assigned to you.

[elskwid]

@fbogsany @mwear here's what I have from the SIG meeting today. I can't seem to update the issue description. Let me know how this looks.

Requirements

• Create gem per integration instrumentation
• Create top-level directory for instrumentations
  • We will also do this for exporters
• Create initial integration gems without auto-installation
  • This can be added later (See Instrumentation Packaging and Installation #19)
• Unify configuration where possible

Preliminary plan

• Break out single instrumentation from dd-trace-rb
• Add enough configuration to support first instrumentation
• Use manual install method for instrumentation (versus auto-installation)
• Create simple app to wire up SDK for first instrumentation

Next up:

• Scope out auto-installation based on first instrumentation
• Add more instrumentations

Notes:

• Consider options for testing including: appraisals, ci configuration, and Docker Compose

[fbogsany]

The spec refers to these things as "instrumentation adapters":
https://github.com/open-telemetry/opentelemetry-specification/blob/57714f7547fe4dcb342ad0ad10a80d86118431c7/specification/overview.md#instrumentation-adapters

I'm not sure we need to use both words in describing them. I'd be happy with "pick one" for the top-level directory name and gem names. I.e. either:

opentelemetry-ruby/adapters/redis/opentelemetry-adapter-redis.gemspec

or

opentelemetry-ruby/instrumentation/redis/opentelemetry-instrumentation-redis.gemspec

[mwear]

That looks good to me @elskwid. I believe that instrumentation is the industry standard terminology and would prefer that we use it over adapter. I immediately know what instrumentation is, but I'd have to ask for clarification about an adapter. I could be wrong about it being a standardized term, if so, let me know.

[fbogsany]

I believe the rationale for "adapter" is that we eventually intend for components (e.g. a Redis driver, Rails, Faraday) to provide built-in instrumentation using the OpenTelemetry API, whereas the instrumentation provided by OpenTelemetry would "adapt" components (e.g. through monkey-patching or well-defined hooks).

[elskwid]

@mwear @fbogsany makes me wonder if this is why other libs, the js lib for example, chose the term plugin. It feels a bit generic to me but perhaps it's shorter and leads to less confusion. Maybe? 😄

In thinking about plugins, would having a plugins directory at the top-level make sense and give us a place to put all the other packages? We could put all instrumentation adapters and exporters here. We would still need to pick adapter or instrumenter but at least it would be on the package level.

- api
- plugins
 | - framework_adapter
 | - platform_exporter
- sdk

Is the layout above better than this (still need to pick adapter v instrumenter/instrumentation):

- adapters
 | - framework
- api
- exporters
 | - platform
- sdk

As I look at it this way, I kind of like plugins. I'll defer to your judgment and opinions.

[bogdandrutu]

Please start a new SIG (probably same members) that will have a different repo/gitter room similar with https://github.com/open-telemetry/community#java-auto-instrumentation

[fbogsany]

I'm less fond of the plugins name. It leads to a lot of noise IMO as you replace the nesting with suffixes. I prefer the second, although I'm not clear on the duplication between adapters and instrumenters -- I believe you only need one.

@bogdandrutu why does this need another SIG? The group of people interested and working on OT Ruby is already pretty small. Splitting into 2 SIGs seems excessive.

[bogdandrutu]

You can initially start with the same set of people, but in the future there may be people specialized in one or the other.

Also the main argument is that we want to have it in a separate repo. We can simply start in a separate repo with the same teams.