Comparing Profile Aggregates
Profile Aggregates can be compared across versions, events, and paths. It’s often helpful to see which functions have regressed between two aggregates. Comparing aggregates helps answer questions such as:
- Which functions regressed the p95 of click responsiveness between versions v1 and v2 of my app?
- Which functions regressed keypress reponsiveness on mobile?
- Which functions regressed my “dashboard_render_time” custom metric on the
/dashboardpage?
- Flamegraph
- Call Tree
Left and Right Aggregates
Palette takes two filters and compares their resulting profiling data. The right aggregate is compared against the left aggregate.
Deltas in total time between left and right aggregates are colored by their change in total time:
- Red: Increase in total time
- Green: Reduction in total time
- Gray: No increase in total time
Examples of Compare Filters
Which functions regressed the p95 of Keydown to Paint between versions v1 and v2 of my app?
Left Aggregate Filters:
| Field | Value |
|---|---|
| Version | v1 |
| Event | Keydown to Paint |
| Path | / |
| Duration | 287ms (p95 of v1) |
Right Aggregate Filters:
| Field | Value |
|---|---|
| Version | v2 |
| Event | Keydown to Paint |
| Path | / |
| Duration | 314ms (p95 of v2) |
Which functions regressed my dashboard_render_time custom metric on the /dashboard page?
Left Aggregate Filters:
| Field | Value |
|---|---|
| Version | v1 |
| Event | dashboard_render_time |
| Path | /dashboard |
Right Aggregate Filters:
| Field | Value |
|---|---|
| Version | v2 |
| Event | dashboard_render_time |
| Path | /dashboard |
FAQs
What if left and right aggregates differ in total sample count?
Samples are normalized to percentage amongst all the samples for that aggregate. If the left and right aggregates have 100 and 1000 samples in total respectively and function x has 1 and 10 samples respectively, then no regression will be shown for function x and it will be colored gray.
What happens if a function is renamed?
Renamed functions are treated as new functions and are not compared against their previous name. The previous name will not show up in the compared profile aggregate.
What happens if a function is in left aggregate and not right aggregate?
A function that appears in the left profile aggregate (P1), but not in the right profile aggregate (P2), will not be visible in the flame graph because the flame graph is drawn in relation to P2. However, it will appear in the call tree because the call tree is the result of merging P1 and P2.
Do custom metrics (aka markers) work with Profile Aggregates?
Yes, filtering samples by custom metrics is supported. To filter samples for a specific custom metric, select the custom metric in the "Metric" dropdown.