Keep separate NTT P3A events for Constellation metrics

[DJAndries]

In order to deprecate JSON reporting completely for iOS, we need to maintain separate NTT event counts for Constellation reporting. The rotated method in NewTabPageP3AHelper must acknowledge Constellation rotations.

[GeetaSarvadnya]

Verification PASSED on iPhone 8 version 16.7.10 running on Brave Nightly 1.75 (118)

• Verified the test plan from Add Constellation support for express & slow P3A metrics brave-core#20264

Step 1_PASSED

• Confirmed that JSON metrics are sent as usual to the endpoint p3a-json.brave.com
• Confirmed that each metric is sent to the server and marked as "sent" = true in the local state file

randomness metrics at http://p3a-json.brave.com/

Example	Example	Example	Example	Example	Example	Example	Example	Example	Example	Example	Example

• Confirmed that if the metric is in the logs object (in local state), then the cadence is typical and the same is displayed in p3a-json.brave.com endpoint in the server

Example	Example	Example	Example	Example	Example

• Confirmed that, If the metric is in logs_slow then the cadence is slow and same is displayed in the p3a-json.brave.com endpoint in the server
  

• Confirmed that, If the metric is in logs_express then the cadence is express and the same is displayed in p3a-json.brave.com endpoint in the server

• Only 2 metrics are shown under the logs_express and in p3a-json endpoint, I could see two cadence express metrics sent to the endpoint p3a-json.brave.com

Example	p3a-json.brave.com	p3a-json.brave.com

• Only 1 metrics is shown under logs_slow same metrics is shown under https://p3a-json.brave.com as expected

Example	Example

• Confirmed that the cadence field in the payload contains the correct value
• The metrics Brave.Shields.AdBlockSettings response value is same on both local state file and p3a-json... endpoint

local-state	p3a-json	local-state	p3a-json	local-state	p3a-json	local-state	p3a-json	ocal-state	p3a-json

Step 2_PASSED

1. Confirmed that requests are being made to star-randsrv.bsg.brave.com to prepare Constellation metrics
2. Confirmed that the URL path is /instances/<cadence type>/randomness
3. Confirmed that when one request is sent for a particular cadence, check to see that metrics are being marked as "sent" as they are being sent to the server, by checking the correct "logs" object in local state.

Step 1,2	Step 1, 2(1)	Step 1, 2(2)

Logs metrics

Step 3	Step 3(1)

4. There are 14 metrics listed under Logs as shown in the images above Step 3, Step 3(1) and 14 metrics are listed under logs_constellation_prep shown in the images below Step 4(1), Step 4(2)
   All the metrics under Logs and logs_constellation_prep are marked as sent:true, when we look at the https://star-randsrv.bsg.brave.com endpoint at the server, all the 14 typical metrics are sent to the server in the form of randomness instances as shown in the images above Step 1,2(1) and Step 1,2(2)

logs_constellation_prep metrics

Step 4(1)	Step 4(2)

5. Confirmed that no. of metrics (3) shown under logs_constellation_prep_express are same as the no. of metrics (2) shown under logs_express but there are (3) express instances at the star-randsrv.bsg.brave.com server endpoint - This is expected as per the comments below Keep separate NTT P3A events for Constellation metrics #42108 (comment)

logs_constellation_prep_express	logs_express	/instances/express/randomness

6. Confirmed that no. of metrics (2) shown under logs_constellation_prep_slow and no. of metrics (1) shown under logs_slow and no of metrics (2) slow instances at the star-randsrv.bsg.brave.com server endpoint ? - This is expected as per the response below Keep separate NTT P3A events for Constellation metrics #42108 (comment)

logs_constellation_prep_slow	logs_slow	slow

Step 3_PASSED

Ensure final measurements are being sent to collector.bsg.brave.com. The URL path should be /<cadence type>. Before metrics are sent to this server, they should appear under constellation_logs, constellation_logs_express and constellation_logs_slow after being prepared (from the requests being made in step 2). As soon as the metrics are sent to the collector, the metric should disappear in the relevant local state object.

1. Confirmed that, before the metrics are sent to server they are prepared to sent and hence appear under logs_constellation_prep_express, logs_constellation_prep_slow and logs_constellation_prep after that the metrics are appear under constellation_logs, constellation_logs_express and constellation_logs_slow .
   But the metrics listed under constellation_logs, constellation_logs_express and constellation_logs_slow are removed once the metrics are sent to the server

metrics prepared

Example	Example	Example	Example	Example

constellation logs

2. Confirmed that the final measurements are sent to the endpoint collector.bsg.brave.com
   • Two slow metrics is sent to the endpoint collector.bsg.brave.com as expected
   • Two express metrics are sent to the endpoint collector.bsg.brave.com (looks like this is expected as per the response below)
   • There is one creative cadence type metrics are sent to the endpoint collector.bsg.brave.com - (this is expected as per the response below)
   • There are 14 typical metrics are sent to the endpoint collector.bsg.brave.com as there are 14 typical metrics are listed under Logs in brave://local-state file

collector.bsg.brave.com

slow	slow	express	express	typical

[GeetaSarvadnya]

@kylehickinson: In the Step 2 above (#42108 (comment)),
For point 5, I am seeing some discrepancy with no. of metrics count shown at star-randsrv.bsg.brave.com server endpoint. The no.of metrics count is (3) express shown at star-randsrv.bsg.brave.com, this is not matching with no. of metrics (2) shown under logs_constellation_prep_express and no. of metrics (2) shown under logs_express via brave://local-state via browser

logs_constellation_prep_express	logs_express	express

• For point 6, I am seeing some discrepancy with no. of metrics (1) shown under logs_slow via brave://local-state and this is not matching with the no. of metrics (2) shown under logs_constellation_prep_slow and no of metrics (2) slow instances at the star-randsrv.bsg.brave.com server endpoint

logs_constellation_prep_slow	logs_slow	slow

[DJAndries]

In the Step 2 above (#42108 (comment)), For point 5, I am seeing some discrepancy with no. of metrics count shown at star-randsrv.bsg.brave.com server endpoint. The no.of metrics count is (3) express shown at star-randsrv.bsg.brave.com, this is not matching with no. of metrics (2) shown under logs_constellation_prep_express and no. of metrics (2) shown under logs_express via brave://local-state via browser

Are you certain that all three randomness requests were sent to /instances/express/randomness. Requests are usually also sent to /instances/slow/randomness and /instances/typical/randomness. If you are not certain, I encourage you to test this once more and inspect the request paths.

For point 6, I am seeing some discrepancy with no. of metrics (1) shown under logs_slow via brave://local-state and this is not matching with the no. of metrics (2) shown under logs_constellation_prep_slow and no of metrics (2) slow instances at the star-randsrv.bsg.brave.com server endpoint

This is fine. The "primary language" metric is only sent via Constellation, so this is expected.

[GeetaSarvadnya]

@DJAndries: Thanks for the response!.

I have re-verified the STEP2 again on iOS today. I am seeing the correct path for each metrics type typical/express/slow.

I can see 3 metrics reported at the /instances/express/randomness via star-randsrv.bsg.brave.com endpoint, there are 2 metrics shown under the logs_express block and 3 metrics shown under logs_constellation_prep_express block via brave://local-state file. The 3rd metrics creative_instanceId was not reported earlier when I tested the issue last week, now I can see 3rd metrics creative_instanceId reported at logs_constellation_prep_express block. Looks like this is expected. Can you review the same? Thanks!

logs_constellation_prep_express	logs_express	/instances/express/randomness