OpenTelemetry .NET Core User API

A comprehensive .NET Core REST API example demonstrating user CRUD operations with OpenTelemetry integration and Alloy for observability.

📊 Project Status

✅ Build Status:
✅ Code Coverage: 80%+ (target achieved)
✅ Code Quality: SonarQube analysis enabled
✅ Security: Vulnerability scanning enabled
✅ Linting: EditorConfig + .NET analyzers
✅ Formatting: dotnet format enforcement

Features

User Management: Complete CRUD operations for users
In-Memory Database: Entity Framework Core with In-Memory database
OpenTelemetry Integration: Automatic instrumentation for traces, metrics, and logs
Alloy Integration: Sends telemetry data to Grafana Alloy
Docker Support: Easy deployment with Docker and Docker Compose
API Documentation: Swagger/OpenAPI documentation
Health Checks: Built-in health check endpoint

API Endpoints

GET /api/user - Get all users
GET /api/user/{id} - Get user by ID
POST /api/user - Create new user
PUT /api/user/{id} - Update user
DELETE /api/user/{id} - Delete user
GET /health - Health check

Quick Start

Full Observability Stack

Start the complete observability stack with one command:

# Start the complete stack
docker-compose up -d

This provides:

API: http://localhost:8080 - Your .NET Core API
Grafana: http://localhost:3000 - Dashboards and visualization (admin/admin)
Alloy UI: http://localhost:12345 - Telemetry collection status
MinIO Console: http://localhost:9001 - Object storage (admin/password123)
Tempo: http://localhost:3200 - Distributed tracing backend
Mimir: http://localhost:9009 - Metrics storage backend
Loki: http://localhost:3100 - Log aggregation backend

Generate Test Data

# Create test data
curl -X POST http://localhost:8080/api/user \
  -H "Content-Type: application/json" \
  -d '{"firstName":"Test","lastName":"User","email":"[email protected]","phoneNumber":"+1234567890"}'

# Make some requests to generate traces
curl http://localhost:8080/api/user
curl http://localhost:8080/api/user/1

Option 3: Pre-built Docker Image

Using the published Docker image:

# Pull and run the latest image
docker run -p 8080:8080 thiagosg/otel-crud-api-net-core:latest

# Or with basic Alloy for observability
docker run -d --name alloy -p 4320:4320 -p 4321:4321 -p 12345:12345 grafana/alloy:latest
docker run -p 8080:8080 --link alloy -e OpenTelemetry__OtlpEndpoint=http://alloy:4320 thiagosg/otel-crud-api-net-core:latest

Manual Setup

Prerequisites:
- .NET 9 SDK
- Docker (for Alloy)
Start Alloy:

docker run -d --name alloy -p 4320:4320 -p 4321:4321 -p 12345:12345 \
  -v $(pwd)/alloy.config:/.config/alloy/config.alloy \
  grafana/alloy:latest run --server.http.listen-addr=0.0.0.0:12345 \
  --storage.path=/var/lib/alloy/data /.config/alloy/config.alloy

Run the API:

dotnet restore
dotnet run

The API will be available at:

API: http://localhost:5000
Swagger UI: http://localhost:5000

🔍 Observability Features

This project demonstrates enterprise-grade observability using the OpenTelemetry standard with Grafana's LGTM stack (Loki, Grafana, Tempo, Mimir).

📊 What's Included:

Distributed Tracing (Tempo)

✅ Automatic HTTP request tracing
✅ Database operation tracing with EF Core
✅ Custom activity sources for business operations
✅ Exception tracking and error correlation
✅ Cross-service trace correlation

Metrics Collection (Mimir)

✅ HTTP request metrics (duration, status codes, throughput)
✅ Database operation metrics
✅ .NET runtime metrics (GC, memory, CPU, threads)
✅ Process metrics (uptime, resource usage)
✅ Custom business metrics

Structured Logging (Loki)

✅ Structured logging with Serilog
✅ Log correlation with traces (trace/span IDs)
✅ Automatic log forwarding to Alloy
✅ JSON structured output for better parsing
✅ Log levels and filtering

Health Monitoring

✅ /health - Detailed health check with database status
✅ /info - Service information (version, uptime, environment)
✅ /metrics - Prometheus-compatible metrics endpoint

🏗️ Architecture Overview

This project implements the LGTM Stack (Loki, Grafana, Tempo, Mimir) for comprehensive observability:

┌─────────────┐    ┌─────────────┐
│  ASP.NET    │    │   Grafana   │
│    Core     │───▶│   Alloy     │
│    App      │    │ Collector   │
└─────────────┘    └─────────────┘
                          │
            ┌─────────────┼─────────────┐
            ▼             ▼             ▼
    ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
    │   Tempo     │ │   Mimir     │ │    Loki     │
    │  (Traces)   │ │ (Metrics)   │ │   (Logs)    │
    └─────────────┘ └─────────────┘ └─────────────┘
            │             │             │
            └─────────────┼─────────────┘
                          ▼
                  ┌─────────────┐
                  │   MinIO     │
                  │ (Storage)   │
                  └─────────────┘

Components

Component	Purpose	Port	UI
ASP.NET Core App	Main application	8080	http://localhost:8080
Grafana Alloy	Telemetry collector	4320 (gRPC), 4321 (HTTP)	http://localhost:12345
Tempo	Distributed tracing	3200, 4317, 4318	-
Mimir	Metrics storage	9009	-
Loki	Log aggregation	3100	-
MinIO	Object storage	9000	http://localhost:9001
Grafana	Visualization	3000	http://localhost:3000

🚀 Key Features:

Feature	Implementation	Status
Framework	.NET 9.0	✅
Language	C# 13	✅
OpenTelemetry	1.12.0	✅
Tracing	Auto + Manual	✅
Metrics	OpenTelemetry	✅
Logging	Serilog + OTEL	✅
Health Checks	ASP.NET Health Checks	✅
Database	EF Core In-Memory	✅
Container	Docker	✅
Concurrency	async/await	✅

🚀 Performance Characteristics:

Container Size: ~120MB
Startup Time: ~1-2s
Memory Usage: ~80MB
Throughput: High performance with async/await
Resource Efficiency: Optimized for cloud deployments

Configuration

Environment Variables

OpenTelemetry__OtlpEndpoint: Alloy endpoint (default: http://localhost:4320)
ASPNETCORE_ENVIRONMENT: Environment (Development/Production)

Alloy Configuration

The alloy.config file configures Alloy to:

Receive OTLP data on ports 4320 (gRPC) and 4321 (HTTP)
Process and batch telemetry data
Export to various backends (configurable)

Sample Data

The application includes sample users for testing:

John Doe ([email protected])
Jane Smith ([email protected])

🧪 Testing

Running Tests

# Run all tests
dotnet test

# Run tests with coverage
dotnet test --collect:"XPlat Code Coverage"

# Run tests with coverage and generate report
./scripts/test.sh

# Run tests with specific coverage threshold
./scripts/test.sh Release 85

Test Coverage

The project maintains 80%+ code coverage with comprehensive test suites:

Unit Tests: Service layer testing with mocked dependencies
Integration Tests: Full API endpoint testing with test database
Test Utilities: Helper classes for test data generation
AutoFixture: Automated test data generation
FluentAssertions: Readable test assertions

Test Structure

UserApi.Tests/
├── Controllers/
│   └── UserControllerIntegrationTests.cs
├── Services/
│   └── UserServiceTests.cs
├── TestConfiguration.cs
├── TestUtilities.cs
└── GlobalUsings.cs

Testing the API

Using curl

# Get all users
curl http://localhost:8080/api/user

# Create a new user
curl -X POST http://localhost:8080/api/user \
  -H "Content-Type: application/json" \
  -d '{"firstName":"Alice","lastName":"Johnson","email":"[email protected]","phoneNumber":"+1234567890"}'

# Get user by ID
curl http://localhost:8080/api/user/1

# Update user
curl -X PUT http://localhost:8080/api/user/1 \
  -H "Content-Type: application/json" \
  -d '{"firstName":"John","lastName":"Doe Updated","email":"[email protected]"}'

# Delete user
curl -X DELETE http://localhost:8080/api/user/1

Using Swagger UI

Visit http://localhost:8080 to access the interactive API documentation.

Observability

Alloy UI

Access the Alloy UI at http://localhost:12345 to view:

Configuration status
Component health
Telemetry flow

Logs

All logs are structured and include:

Trace correlation IDs
Service information
Request/response details
Error context

Metrics

Key metrics include:

HTTP request duration and count
Database operation metrics
Application performance metrics
System resource usage

🛠️ Development

Code Quality

This project enforces high code quality standards:

EditorConfig: Consistent code formatting across editors
.NET Analyzers: Built-in code analysis and best practices
SonarQube: Comprehensive code quality analysis
dotnet format: Automated code formatting
TreatWarningsAsErrors: All warnings treated as errors
Nullable Reference Types: Enabled for better null safety

Running Code Quality Checks

# Format code
dotnet format

# Verify formatting (CI check)
dotnet format --verify-no-changes

# Run linting
dotnet build --verbosity normal --configuration Release /p:TreatWarningsAsErrors=true

# Run SonarCloud analysis locally (example)
dotnet sonarscanner begin \
  /k:"devops-thiago_otel-core-example" \
  /o:"devops-thiago" \
  /d:sonar.host.url="https://sonarcloud.io" \
  /d:sonar.token="your-token"
dotnet build
dotnet test --collect:"XPlat Code Coverage"
dotnet sonarscanner end /d:sonar.token="your-token"

Project Structure

├── Controllers/
│   └── UserController.cs           # REST API controller
├── Data/
│   └── UserDbContext.cs            # Entity Framework context
├── DTOs/
│   └── UserDto.cs                  # Data transfer objects
├── Models/
│   └── User.cs                     # User entity model
├── Services/
│   └── UserService.cs              # Business logic layer
├── UserApi.Tests/                  # Test project
│   ├── Controllers/
│   │   └── UserControllerIntegrationTests.cs
│   ├── Services/
│   │   └── UserServiceTests.cs
│   ├── TestConfiguration.cs
│   ├── TestUtilities.cs
│   └── GlobalUsings.cs
├── config/                         # Observability configurations
│   ├── alloy.alloy                # Alloy configuration
│   ├── tempo.yaml                 # Tempo configuration
│   ├── mimir.yaml                 # Mimir configuration
│   ├── loki.yaml                  # Loki configuration
│   └── grafana/                   # Grafana configurations
├── scripts/                       # Build and utility scripts
├── docker-compose.yml             # Full stack deployment
├── Dockerfile                     # Application container
├── Program.cs                     # Application entry point
├── UserApi.csproj                 # Project file
└── README.md                      # This file

CI/CD Pipeline

The project includes a comprehensive GitHub Actions workflow:

Build & Test: Compile, run tests, generate coverage
Code Quality: Formatting, linting, security scanning
SonarQube Analysis: Code quality and security analysis
Docker Build: Container image creation and publishing
Deployment: Automated deployment to staging

Dependency Management

This project uses Dependabot for automated dependency updates:

Automated Updates:

🔄 NuGet Packages: Weekly updates every Monday at 9 AM EST
🔄 GitHub Actions: Weekly updates every Monday at 10 AM EST
🔄 Docker Images: Weekly updates every Tuesday at 9 AM EST

Intelligent Grouping:

OpenTelemetry: All OpenTelemetry packages updated together
Microsoft: Microsoft packages grouped for consistency
Testing: Test-related packages updated as a group

Features:

✅ Automatic PR creation with detailed changelogs
✅ Automatic reviewer assignment (@devops-thiago)
✅ Proper labeling and commit message formatting
✅ Limited concurrent PRs to avoid overwhelming
✅ Security and compatibility checks via CI pipeline

Manual Dependency Checks:

# Check for outdated packages
dotnet list package --outdated

# Check for vulnerable packages
dotnet list package --vulnerable --include-transitive

# Update specific package
dotnet add package PackageName --version x.x.x

Adding Custom Metrics

using System.Diagnostics.Metrics;

// In your service
private static readonly Meter Meter = new("UserApi");
private static readonly Counter<int> UserCreatedCounter = Meter.CreateCounter<int>("users_created_total");

// Increment counter
UserCreatedCounter.Add(1, new KeyValuePair<string, object?>("user.email", email));

Contributing

Fork the repository
Create a feature branch (git checkout -b feature/amazing-feature)
Make your changes
Run tests and quality checks (./scripts/test.sh)
Commit your changes (git commit -m 'Add amazing feature')
Push to the branch (git push origin feature/amazing-feature)
Open a Pull Request

Pull Request Requirements

✅ All tests must pass
✅ Code coverage must be 80%+
✅ No linting errors or warnings
✅ Code must be properly formatted
✅ SonarQube quality gate must pass
✅ Security scan must pass
✅ Documentation updated if needed

Troubleshooting

Alloy not receiving data: Check if Alloy is running and accessible on the configured ports
Database issues: The in-memory database resets on restart
Port conflicts: Ensure ports 8080, 4320, 4321, and 12345 are available

License

This project is licensed under the MIT License.

Name		Name	Last commit message	Last commit date
Latest commit History 99 Commits
.github		.github
Controllers		Controllers
DTOs		DTOs
Data		Data
Infrastructure		Infrastructure
Models		Models
Services		Services
UserApi.Tests		UserApi.Tests
config		config
scripts		scripts
.editorconfig		.editorconfig
.gitignore		.gitignore
Directory.Build.props		Directory.Build.props
Dockerfile		Dockerfile
LICENSE		LICENSE
Program.cs		Program.cs
README.md		README.md
UserApi.csproj		UserApi.csproj
UserApi.sln		UserApi.sln
appsettings.Development.json		appsettings.Development.json
appsettings.Docker.json		appsettings.Docker.json
appsettings.json		appsettings.json
docker-compose.yml		docker-compose.yml
global.json		global.json

License

devops-thiago/otel-core-example

Folders and files

Latest commit

History

Repository files navigation

OpenTelemetry .NET Core User API

📊 Project Status

Features

API Endpoints

Quick Start

Full Observability Stack

Generate Test Data

Option 3: Pre-built Docker Image

Manual Setup

🔍 Observability Features

📊 What's Included:

Distributed Tracing (Tempo)

Metrics Collection (Mimir)

Structured Logging (Loki)

Health Monitoring

🏗️ Architecture Overview

Components

🚀 Key Features:

🚀 Performance Characteristics:

Configuration

Environment Variables

Alloy Configuration

Sample Data

🧪 Testing

Running Tests

Test Coverage

Test Structure

Testing the API

Using curl

Using Swagger UI

Observability

Alloy UI

Logs

Metrics

🛠️ Development

Code Quality

Running Code Quality Checks

Project Structure

CI/CD Pipeline

Dependency Management

Automated Updates:

Intelligent Grouping:

Features:

Manual Dependency Checks:

Adding Custom Metrics

Contributing

Pull Request Requirements

Troubleshooting

License

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Contributors 3

Uh oh!

Languages

Packages