Fintech company is running a test to evaluate whether or not they should launch a spend tracking feature to their banking app. to evaluate the launch, they are considering the following 3 areas:
- Acquisition Cost
- Profitability
- Value to members
The company ran an A/B test to a population of 40M people randomly distributed into two groups of ~20M in each advertising its product without spend tracker (control) vs with spend tracker (test).
In addition to overall CAC (cost per conversion), the company wants to see if the spend tracker feature skews respondent distribution. In particular, the company will analyze the impact on two segments: Segment A and Segment B (higher income segment.)
- Average monthly spend for segment B is higher than Segment A
- Members in Segment B are also more likely to have additional bank acocunts on top of companies banking software
The spend tracker feature will be free to members and it will cost the company $5 per month for every member that uses this feature. The spend tracker will generate no revenue.
The company makes money every time a members makes a purchase using their the company debit card. A small portion of every dollar spent on the app is revenue to the company. (This is called interchange.) Even though spend tracker costs money, we believe adding the feature could help profitability since the more cumulative spend on the app per member, the more revenue to the company. The tracker could increase profitability through higher retention and/or higher average monthly spend. To evaluate profitability, we look at LTV, CAC to LTV Ratio, ROI.
To evaluate value to members, we look at what percentage of our members test the feature and continue to use it through time.
This project is licensed under the MIT License.