Pay area revamp

This project reorganized the Pay/Pagar Area entry points and included a list of the next bills to be paid. It represented a solid partnership between Design and Data, and it required me stakeholder management between different teams.

Evidence

We ran an experiment that presented a similar experience on Barcode Payment Flow, it reflected an uplift of ~30% on barcodes paid through DDA and 1.11% on barcodes paid in general.

This reinforced the opportunity of anticipate upcoming bills in a contextual approach as a lever of commitments. With that in mind, we wanted to test displaying some of the upcoming bills from Payments Assistant within the Pagar area to raise the number of payments in this area and the overall amount of scheduled bills on PA.

Hypothesis to be validated

By anticipating upcoming bills outside of AP dash, in a contextual approach, we would smooth the experience, and it would positively impact the number of payments/commitments and the perception of value of Payment Assistant.

Variant A

This variant tested the impacts of adding upcoming bills from PA's dashboard to the Pagar Area, with no changes in the design of the original entry points to Payments Assistant, Buscador de Boletos, Automatic Debit, and Detran.

Variant B

This variant tested the impacts of adding upcoming bills from PA's dashboard to Pagar Area, and the format of entry points (pill widget instead of list row, without subtitles).

Variant C

This variant tested the impacts of adding upcoming bills from PA's dashboard to Pagar Area and adding the "Programar Contas" button to the top, as one of the main actions.

Variant D

This variant tested both the impacts of adding upcoming bills from PA's dashboard to Pagar Area, the addition of the "Programar Contas" button to the top, and the change for the Credit Card Bill entry-point (list row instead of the main action). We are also changing the order of the buttons shown on the top.

The experiment

The test design

As household bills usually have long cycles (30 days), to capture the effects of these changes, we needed our test running for at least 1 month. This was the main reason for going with a multivariate approach rather than doing serial AB tests.

There was no statistically significant impact on the test main metrics, total number of payments, and customers with payments (considering products available in the Pay area: PIX QR Code, PIX copy and paste, barcode, credit card, and automotive debit).

Overall, Variant C performed the best in the test. Compared to Variant E, it stood out by prioritizing barcodes in the upcoming bills list, which generated more barcode payments and consequently more DDA commitments. Additionally, it did not affect the interaction with other entry points in the Pay Area. We recommended rolling out this variant and conducting new tests in the future to include the CC bill in the area without affecting DDA payments.

Challenges

This project required a roadshow presentation to align cross-functional stakeholders on the implementation timeline. Furthermore, it involved a strategic partnership with Business Analysts to define success metrics and establish a robust tracking plan for variable testing.