AB testing for ads services

.env.template

@@ -23,9 +23,10 @@ NEXT_PUBLIC_DD_VERSION_FRONTEND=1.0.0 # use NEXT_PUBLIC_* to expose var to store
 DD_VERSION_BACKEND=1.0.0 # Use for worker service as well, since they use the same Docker image
 DD_VERSION_DBM=1.0.0
 DD_VERSION_ADS=1.0.0
+DD_VERSION_ADS_PYTHON=1.0.0
 DD_VERSION_DISCOUNTS=1.0.0
-DD_VERSION_POSTGRES=1.0.0
-DD_VERSION_NGINX=1.0.0
+DD_VERSION_POSTGRES=15.0
+DD_VERSION_NGINX=1.28.0
 DD_VERSION_REDIS=6.2
 
 # =============================================
@@ -35,8 +36,15 @@ POSTGRES_USER=postgres
 POSTGRES_PASSWORD=postgres
 DB_HOST=postgres
 DB_PORT=5432
-DB_POOL=25                    # Database connection pool size set in backend service (default: 25)
-MAX_THREADS=5                 # Maximum number of concurrent threads set in backend service (default: 5)
+DB_POOL=25 # Database connection pool size set in backend service (default: 25)
+MAX_THREADS=5 # Maximum number of concurrent threads set in backend service (default: 5)
+
+# =============================================
+# Nginx/Service Proxy Configuration
+# =============================================
+ADS_A_UPSTREAM=ads:3030
+ADS_B_UPSTREAM=ads-python:3030
+ADS_B_PERCENT=0 # Set < 0 > 100 to split traffic between ads services
 
 # =============================================
 # Frontend Service Configuration
@@ -65,4 +73,4 @@ DISABLE_SPRING=1
 STOREDOG_URL=http://service-proxy:80 # base url for storedog service (default: 'http://service-proxy:80')
 PUPPETEER_TIMEOUT=30000 # timeout for puppeteer (default: 30000)
 
-SKIP_SESSION_CLOSE= # skip session close for puppeteer (default: ''). note that the current puppeteer script doesn't make use of this environment variable but can easily be updated to do so
+SKIP_SESSION_CLOSE= # skip session close for puppeteer (default: ''). Note that the current puppeteer script doesn't make use of this environment variable but can easily be updated to do so

README.md

@@ -154,15 +154,15 @@ Common Datadog variables that can be set in application services:
 - `DD_LOGS_INJECTION`: Enable log injection into traces. Some languages' trace libraries turn this on by default, but we turn it on explicitly to prevent confusion.
 - `DD_PROFILING_ENABLED`: Enable the Continuous Profiler 
 - `DD_RUNTIME_METRICS_ENABLED`: Enable runtime metrics
-- 
 
 Service-specific versions:
 - `DD_VERSION_FRONTEND`: Frontend version (default: `1.0.0`)
 - `DD_VERSION_BACKEND`: Backend version (default: `1.0.0`)
 - `DD_VERSION_DISCOUNTS`: Discounts service version (default: `1.0.0`)
 - `DD_VERSION_ADS`: Ads service version (default: `1.0.0`)
-- `DD_VERSION_NGINX`: nginx service version (default: `1.0.0`)
-- `DD_VERSION_POSTGRES`: PostgreSQL service version (default: `1.0.0`)
+- `DD_VERSION_ADS_PYTHON`: Ads Python service version (default: `1.0.0`)
+- `DD_VERSION_NGINX`: nginx service version (default: `1.28.0`)
+- `DD_VERSION_POSTGRES`: PostgreSQL service version (default: `15.0`)
 - `DD_VERSION_REDIS`: Redis version (default: `6.2`)
 
 > [!NOTE]
@@ -186,7 +186,23 @@ Puppeteer service configuration:
 - `PUPPETEER_TIMEOUT`: Sets max timeout for Puppeteer, in case the session is unresponsive
 - `SKIP_SESSION_CLOSE`: Skip closing browser sessions
 
-## Feature flags
+### Nginx/Service Proxy Configuration Variables
+
+These variables control the upstream configuration for the ads services in the Nginx (service-proxy) container, enabling A/B testing and traffic splitting between the Java and Python ads services.
+
+> [!IMPORTANT]
+> When `ADS_B_PERCENT` is greater than zero, the `ADS_B_UPSTREAM` endpoint must be reachable. Otherwise, the service-proxy will crash and restart. Uncomment the `ads-python` section in `docker-compose.yml` to enable the service.
+
+- `ADS_A_UPSTREAM`: Host and port for the primary (A) ads service (default: `ads:3030`)
+- `ADS_B_UPSTREAM`: Host and port for the secondary (B) ads service (default: `ads-python:3030`)
+- `ADS_B_PERCENT`: Percentage of traffic to route to the B (Python) ads service (default: `0`). The remainder goes to the A (Java) ads service.
+  - Set to a value between `0` and `100` to control the split.
+
+## Optional features
+
+There are several features that can be enabled by setting environment variables and feature flags.
+
+### Feature flags
 Some capabilities are hidden behind feature flags, which can be controlled via `services/frontend/site/featureFlags.config.json`. 
 
 > [!NOTE]
@@ -197,7 +213,7 @@ Some capabilities are hidden behind feature flags, which can be controlled via `
 >   - ./services/frontend/site/featureFlags.config.json:/app/featureFlags.config.json
 > ```
 
-### dbm 
+#### dbm 
 Enables a product ticker on the homepage with a long-running query to demonstrate DBM. 
 
 **How to use**:
@@ -209,7 +225,7 @@ Enables a product ticker on the homepage with a long-running query to demonstrat
 
 You can modify the ticker functionality in `services/frontend/components/common/NavBar.tsx`.
 
-### error-tracking 
+#### error-tracking 
 Introduces an exception in the Ads services to demonstrate Error Tracking by setting a header in to a value that is not expected by the Ads service.
 
 **How to use**:
@@ -220,7 +236,7 @@ Introduces an exception in the Ads services to demonstrate Error Tracking by set
 
 Modify this functionality in `services/frontend/components/common/Ad/Ad.tsx` and respective Ads service being used.
 
-### api-errors
+#### api-errors
 This introduces random errors that occur in the frontend service's `/api` routes.
 
 **How to use**:
@@ -230,7 +246,7 @@ This introduces random errors that occur in the frontend service's `/api` routes
 
 Modify this functionality in `services/frontend/pages/api/*`.
 
-### product-card-frustration
+#### product-card-frustration
 This will swap out the product card component with a version that doesn't have the thumbnails linked to the product page. When paired with the Puppeteer service, this can be used to demonstrate Frustration Signals in RUM.
 
 **How to use**:
@@ -240,6 +256,19 @@ This will swap out the product card component with a version that doesn't have t
 
 Modify this functionality in `services/frontend/components/Product/ProductCard.tsx` and `services/frontend/components/Product/ProductCard-v2.tsx`.
 
+### A/B Testing Ads services
+
+Run two Ads services and split traffic between them. The amount of traffic sent to each service is set with a percent value.
+
+**How to use**
+1. Add a second Ads service to the `docker-compose.yml`
+1. Add and set these environment variables to the `service-proxy` service:
+  - `ADS_A_UPSTREAM`: Host and port for the primary (A) ads service (default: `ads:3030`)
+  - `ADS_B_UPSTREAM`: Host and port for the secondary (B) ads service (default: `ads-python:3030`)
+  - `ADS_B_PERCENT`: Percentage of traffic to route to the B (Python) ads service (default: `0`). The remainder goes to the A (Java) ads service.
+    - Set to a value between `0` and `100` to control the split.
+1. Start the app via `docker compose up`
+
 ## Image publication
 Images are stored in GHCR. On PR merges, only the affected services will be pushed to GHCR, using the `latest` tag. For example, if you only made changes to the `backend` service, then only the `backend` Github workflow will trigger and publish `ghcr.io/datadog/storedog/backend:latest`. 
 
@@ -252,7 +281,7 @@ All of the services in the Storedog application are Dockerized and run in contai
 Below is a breakdown of services and some instructions on how to use them.
 
 ### Ads
-There are two advertisement services, the default service is built in Java and there is another option available in Python. These services do the same thing, have the same endpoints, run on the same port (`3030`), and have the same failure modes. The biggest difference is the Python service uses a Postgres database to store the ads, while the Java service uses an in-memory list. These ads are served through the `Ads.tsx` component in the frontend service.
+There are two advertisement services, the default service is built in Java and there is another option available in Python. These services do the same thing, have the same endpoints, run on the same port (`3030`), and have the same failure modes. These ads are served through the `Ads.tsx` component in the frontend service.
 
 To switch between the Java and Python services, see the instructions in the [Ads service README](./services/ads/README.md).
 

docker-compose.dev.yml

@@ -165,8 +165,12 @@ services:
     build:
       context: ./services/ads/java
     depends_on:
+      - postgres
       - dd-agent
     environment:
+      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-postgres}
+      - POSTGRES_USER=${POSTGRES_USER:-postgres}
+      - POSTGRES_HOST=postgres
       - DD_AGENT_HOST=dd-agent
       - DD_SERVICE=store-ads
       - DD_VERSION=${DD_VERSION_ADS:-1.0.0}
@@ -179,6 +183,25 @@ services:
     labels:
       com.datadoghq.ad.logs: '[{"source": "java"}]'
 
+  # OPTIONAL: Advertisement service (Python)
+  # ads-python:
+  #   build:
+  #     context: ./services/ads/python
+  #   depends_on:
+  #     - postgres
+  #     - dd-agent
+  #   environment:
+  #     - POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-postgres}
+  #     - POSTGRES_USER=${POSTGRES_USER:-postgres}
+  #     - POSTGRES_HOST=postgres
+  #     - DD_AGENT_HOST=dd-agent
+  #     - DD_SERVICE=store-ads-python
+  #     - DD_VERSION=${DD_VERSION_ADS_PYTHON:-1.0.0}
+  #   networks:
+  #     - storedog-network
+  #   labels:
+  #     com.datadoghq.ad.logs: '[{"source": "python"}]'
+
   # Web server and reverse proxy
   service-proxy:
     build:
@@ -192,9 +215,12 @@ services:
       - frontend
       - dd-agent
     environment:
+      - ADS_A_UPSTREAM=${ADS_A_UPSTREAM:-ads:3030} # Only needed if using AB testing
+      - ADS_B_UPSTREAM=${ADS_B_UPSTREAM:-ads-python:3030} # Only needed if using AB testing
+      - ADS_B_PERCENT=${ADS_B_PERCENT:-0} # Percent of traffic to ads-python; remainder goes to ads-java. Only needed if using AB testing
       - DD_AGENT_HOST=dd-agent
       - DD_SERVICE=service-proxy
-      - DD_VERSION=${DD_VERSION_NGINX:-1.0.0}
+      - DD_VERSION=${DD_VERSION_NGINX:-1.28.0}
     labels:
       com.datadoghq.ad.logs: '[{"source": "nginx"}]'
       com.datadoghq.ad.check_names: '["nginx"]'
@@ -217,7 +243,7 @@ services:
       - POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-postgres}
     labels:
       com.datadoghq.tags.service: 'store-db'
-      com.datadoghq.tags.version: '${DD_VERSION_POSTGRES:-1.0.0}'
+      com.datadoghq.tags.version: '${DD_VERSION_POSTGRES:-15.0}'
       com.datadoghq.ad.check_names: '["postgres"]'
       com.datadoghq.ad.init_configs: '[{}]'
       com.datadoghq.ad.instances: '[{"host":"%%host%%", "port":5432, "username":"datadog", "password":"datadog"}]'
@@ -234,6 +260,7 @@ services:
       - storedog-network
     labels:
       com.datadoghq.tags.service: 'redis'
+      com.datadoghq.tags.env: '${DD_ENV:-development}'
       com.datadoghq.tags.version: '${DD_VERSION_REDIS:-6.2}'
       com.datadoghq.ad.check_names: '["redisdb"]'
       com.datadoghq.ad.init_configs: '[{}]'

docker-compose.yml

@@ -110,11 +110,14 @@ services:
   ads:
     image: ghcr.io/datadog/storedog/ads-java:${STOREDOG_IMAGE_VERSION:-latest}
     depends_on:
+      - postgres
       - dd-agent
     environment:
+      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-postgres}
+      - POSTGRES_USER=${POSTGRES_USER:-postgres}
+      - POSTGRES_HOST=postgres
       - DD_AGENT_HOST=dd-agent
       - DD_SERVICE=store-ads
-      - DD_ENV=${DD_ENV:-production}
       - DD_VERSION=${DD_VERSION_ADS:-1.0.0}
       - DD_LOGS_INJECTION=true
       - DD_PROFILING_ENABLED=true
@@ -125,8 +128,23 @@ services:
     labels:
       com.datadoghq.ad.logs: '[{"source": "java"}]'
       com.datadoghq.tags.service: 'store-ads'
-      com.datadoghq.tags.env: '${DD_ENV:-production}'
       com.datadoghq.tags.version: '${DD_VERSION_ADS:-1.0.0}'
+  # ads-python:
+  #   image: ghcr.io/datadog/storedog/ads:${STOREDOG_IMAGE_VERSION:-latest}
+  #   depends_on:
+  #     - postgres
+  #     - dd-agent
+  #   environment:
+  #     - POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-postgres}
+  #     - POSTGRES_USER=${POSTGRES_USER:-postgres}
+  #     - POSTGRES_HOST=postgres
+  #     - DD_AGENT_HOST=dd-agent
+  #     - DD_SERVICE=store-ads-python
+  #     - DD_VERSION=${DD_VERSION_ADS_PYTHON:-1.0.0}
+  #   networks:
+  #     - storedog-network
+  #   labels:
+  #     com.datadoghq.ad.logs: '[{"source": "python"}]'
   service-proxy:
     image: ghcr.io/datadog/storedog/nginx:${STOREDOG_IMAGE_VERSION:-latest}
     restart: always
@@ -138,9 +156,12 @@ services:
       - frontend
       - dd-agent
     environment:
+      - ADS_A_UPSTREAM=${ADS_A_UPSTREAM:-ads:3030} # Only needed if using AB testing
+      - ADS_B_UPSTREAM=${ADS_B_UPSTREAM:-ads-python:3030} # Only needed if using AB testing
+      - ADS_B_PERCENT=${ADS_B_PERCENT:-0} # Percent of traffic to ads-python; remainder goes to ads-java. Only needed if using AB testing
       - DD_AGENT_HOST=dd-agent
       - DD_SERVICE=service-proxy
-      - DD_VERSION=${DD_VERSION_NGINX:-1.0.0}
+      - DD_VERSION=${DD_VERSION_NGINX:-1.28.0}
     labels:
       com.datadoghq.ad.logs: '[{"source": "nginx"}]'
       com.datadoghq.ad.check_names: '["nginx"]'
@@ -160,7 +181,7 @@ services:
       - POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-postgres}
     labels:
       com.datadoghq.tags.service: 'store-db'
-      com.datadoghq.tags.version: '${DD_VERSION_POSTGRES:-1.0.0}'
+      com.datadoghq.tags.version: '${DD_VERSION_POSTGRES:-15.0}'
       com.datadoghq.ad.check_names: '["postgres"]'
       com.datadoghq.ad.init_configs: '[{}]'
       com.datadoghq.ad.instances: '[{"host":"%%host%%", "port":5432, "username":"datadog", "password":"datadog"}]'

services/ads/README.md

@@ -4,6 +4,10 @@
 
 This service is responsible for managing the banner advertisements served to the frontend service of the application. There are two variations of this service, one uses Python and the other uses Java.
 
+## Java service
+
+The Java service is the default used with Storedog. It uses the Spring framework. It uses the PostgreSQL JDBC driver to connect to a PostgreSQL database. It uses an H2 in-memory database when running build tests.
+
 ## Python service
 
 The Python service is a Flask application that uses SQLAlchemy to connect to a PostgreSQL database. The service is packaged as a Docker image and typically used in a Docker Compose file (see the root of this repo).
@@ -163,7 +167,4 @@ ads:
     volumes:
       - ./services/ads/python:/app
     labels:
-      com.datadoghq.ad.logs: '[{"source": "python", "service": "store-ads"}]'
-      com.datadoghq.tags.env: '${DD_ENV-dev}'
-      com.datadoghq.tags.service: 'store-ads'
-      com.datadoghq.tags.version: ${DD_VERSION_ADS-1.0.0}
+      com.datadoghq.ad.logs: '[{"source": "python"}]'

services/ads/java/Dockerfile

@@ -20,13 +20,16 @@ ARG DD_GIT_REPOSITORY_URL
 ARG DD_GIT_COMMIT_SHA
 
 ENV APP_HOME=/usr/app/ \
-  ADS_SERVER_PORT=3030 \
-  DD_GIT_REPOSITORY_URL=${DD_GIT_REPOSITORY_URL} \
-  DD_GIT_COMMIT_SHA=${DD_GIT_COMMIT_SHA} 
+    ADS_SERVER_PORT=3030 \
+    POSTGRES_PASSWORD=postgres \
+    POSTGRES_USER=postgres \
+    POSTGRES_HOST=postgres \
+    DD_GIT_REPOSITORY_URL=${DD_GIT_REPOSITORY_URL} \
+    DD_GIT_COMMIT_SHA=${DD_GIT_COMMIT_SHA}
 
 WORKDIR $APP_HOME
 COPY --from=temp_build_image $APP_HOME/build/libs/*.jar store-ads.jar
 
 RUN wget -O dd-java-agent.jar 'https://dtdg.co/latest-java-tracer'
 
-ENTRYPOINT ["java", "-javaagent:/usr/app/dd-java-agent.jar", "-jar", "store-ads.jar"]
+ENTRYPOINT ["java", "-javaagent:/usr/app/dd-java-agent.jar", "-jar", "store-ads.jar"]

services/ads/java/build.gradle

@@ -15,11 +15,14 @@ repositories {
 dependencies {
 	implementation ( 
     'commons-io:commons-io:2.4',
-    'org.springframework.boot:spring-boot-starter-web'
+    'org.springframework.boot:spring-boot-starter-web',
+    'org.springframework.boot:spring-boot-starter-data-jpa'
   )
+	runtimeOnly 'org.postgresql:postgresql'
 	implementation 'com.fasterxml.jackson.core:jackson-databind:2.15.2'
-  implementation 'net.logstash.logback:logstash-logback-encoder:7.0'
+  	implementation 'net.logstash.logback:logstash-logback-encoder:7.0'
 	testImplementation 'org.springframework.boot:spring-boot-starter-test'
+	testImplementation 'com.h2database:h2'
 }
 
 bootJar {