Analyze Product Feature Usage

PetBox is a fictional eCommerce company that sells monthly subscription boxes containing animal care products. They have recently expanded their business by offering a line of branded webcams on their website.

The “PetCam” allows users to monitor their cat’s behavior throughout the day. PetBox had an existing app for box subscribers, and are expanding it to support PetCam functionality for both subscribers and non-subscribers. In the app, users are able to livestream their PetCam feed. Users may also create clips from their feed to save or share on social media.

PetBox wants to analyze the onboarding flow for users connecting their PetCams, specifically the step in their flow that displays an orientation screen. The user can choose to skip this step, and PetBox wants to know how this affects onboarding progression. Additionally, they are conducting an A/B test on copy for one of the buttons in their onboarding process. They want to see which group of users was more likely to complete the full onboarding flow.

Questions

• How does interaction with the orientation screen affect onboarding flow conversion?
• What is the impact of the A/B test on onboarding flow conversion?

Create Funnel Visualization

To create our funnel, click the ”+” icon in the left-hand navigation bar, and select Conversion and Drop Off. Now we can begin adding the events in our onboarding flow. Funnel measures user progression from event to event, so let’s add our onboarding flow events in the following order:

1. Create Profile: Within the app, users must first create a profile.
2. Start Support Chat: Users have the option to skip the support chat if they wish.
3. Connect PetCam: Users connection to webcams is successful.
4. Open PetCam: Users open the PetCam, beginning a streaming session.

Executing this query gives us our onboarding flow visualization. In the donuts, we can see the number of users who progressed to that event. In the webbings between them, we can see the percentage of users who proceeded to the next step.

Make a Step Optional

The event Start Support Chat is an optional action by the user. A user may choose to skip the orientation screen or go through the orientation, before progressing through the flow. However, our current funnel requires users to have completed the event Start Support Chat.

To also see what our numbers look like for users who chose not to Start Support Chat, we can utilize Analytics’ optional step functionality.

• Let’s click the pushpin to the left of Start Support Chat in the query builder.
• Next, let’s execute the query to update it.

Above, we can see a Funnel visualization with multiple paths. It is currently showing the onboarding flow with the Start Support Chat event included. As we can see, the numbers are the same as in our initial funnel.

• Let’s click on the path above Start Support Chat to look at the flow that excludes this event.

Observation

• 24.83% of users who did Start Support Chat completed the onboarding flow.
• 36.25% of users who did not do Start Support Chat completed the onboarding flow.

Users who did Start Support Chat were more likely to complete the onboarding flow. In light of these results, PetBox should consider enhancing or removing their Support Chat feature for PetCam onboarding.

Based on the results of our first analysis, PetBox chose to delete the invitation to use Support Chat. They now want to run an A/B test of new copy on the Connect PetCam page to measure the effect on flow completion.

• Group A sees the previous instructions to connect the PetCam.
• Group B sees a revamped walkthrough.

Create Funnel Visualization

To create our funnel, let’s drag in the events in our new onboarding flow.

1. Connect PetCam: Within the app, users click this button to begin the flow.
2. Open PetCam: Once connected, users open the PetCam and start streaming.
3. Subscribe: Users who appreciate the service will subscribe for future access.

Create a Shared Breakout

The Shared Breakout feature in Funnel allows us to segment a specific step using a property and follow those users throughout the flow.

• Underneath Connect PetCam in the query builder, select +group by. Choose AB Group under Event Properties.
• By default, the query results will be grouped by the first events’ AB test. This is because we want to measure what happens to users after seeing the different instructions for connecting the PetCam.

This segments the users who did Connect PetCam by their group and follows them through the rest of the funnel. We can then measure and compare conversion rates and drop off.

Compare AB Groups

By clicking on a wedge representing either Group A or Group B, we can highlight this group. After a group has been highlighted, the percentages in the webbings will only reflect step-to-step conversion for that group.

Above we can see that 94.63% of Group A users progressed from Connect PetCam to Open PetCam. For Group B, this number was 94.91%.

Observation

• 47.76% of Group A users completed the onboarding flow
• 51.72% of Group B users completed the onboarding flow
• Users from Group A showed a slightly higher likelihood of completing the onboarding flow. This shows that from the copy options tested, Group A’s copy would be the best to implement.