7.1 KiB
Bolt
10/03/2022
- Engineering on Bolt side will have more time to in app experience
- Goal for in app by 31 October
- Vera to talk about another issue:
- Retraining & no access to courses
- Google Sheets collecting information
- How many cases and tickets are coming in per week? - 300
- Able to complete it, but "blocked"
- Open the course but redirected to Bolt home page
- Or can't open the course
- South Africa & Tunisia - 7 markets total
- Could be network connectivity
- These markets tend to have older phones with much older versions of Android
- iOS defaults to Safari, Android uses "web view" which is old and could not work well with react framework that Northpass uses
- It is curious that this is happening with retraining.
- Charlie - the only thing I can think of doing is reverting the look to an older look
- Single example of Azerbaijani with OS 11, but not clear what the actual issue is
- Shows as complete, but he claims he could not complete. This sounds like a different issue.
- Nigerian driver running 8.1, this is the common issue and it seems to be an OS versioning issue
- Possibility of the UID/Email not getting added in correctly?
11/14/2022
Bolt In-App Training Release
- Launching in four markets: Azerbaijan (two languages, RU, AZ), Nigeria, Kenya, UK
- No training exists in France
- France and UK you can't mention "training" they are all suggestions
- "All is under control"
- Vera is back on Wednesday to talk about Arabic course.
- Everyone is waiting anxiously for in-app training
- Contract ends in January, she wanted to release earlier to have more analytics before renewal
- If it goes well, more markets will want this training.
- When everything is in-app, she's expecting higher adoption
- Do African markets adopt differently during the Winter Holidays? They may not celebrate Christmas.
11/28/2022
Post-Android Release Meeting
Release of the new courses keeps being delayed. Bolt engineers sent video of what the in-app training looks like, but the drivers see the category of the course. However, the course category is just for Doris. The drivers should not see the category on the course overview page. DONE: Hide category from course overview page. Doris also getting asked about the in-app learning launch. She asked about Crowdin - a translation service. Do we have an integration? DONE: Research a Crowdin integration. Ask Vera what her workflow is like.
12/8/2022
Prep for QBR
The issue from Joonas, all in African Market. Mostly in Nigeria. Azerbaijan, SA, Tunisia, etc. The recording is from someone who actually came to the hub and recorded their screen. What Bolt can see is the attempts which will show/not show if people are completing or having trouble. From launch, 5 people have not completed it. They were validating training vs no training in each markets. Acceptance rates and reviews are lower for those who haven't completed training. This pilot is the first time we are able to test training within the app. Non-Trainees have to figure out the system on their own. Trainees are told to accept everything, and so their acceptance rates are higher. Non-Trainees ask other users, i.e veterans, and have more street smarts and therefore earn more than people who take the training.
To better grasp the content:
- They took this content themselves, and have A/B/C/D course tests.
- Testing Internal Platform (called Stories) and how it relates to NP and courses.
- Stories is a set of slides, and they are beginning to leverage it for learning.
- Group D is Edume. They don't have a contract, but wanted to see the test.
- The three platforms look very similar. Designer created a similar experience so people all have the same experience.
- Stories is marked as completed by going to the last slide.
- Average Training Attempts, Northpass is the worst.
- Started attempts... something is causing the users to restart.
- Northpass, median minutes of completion is the worst.
- If a driver doesn't organically reach the last step in registration, they don't get enrolled in the courses and are locked out.
- Usually they don't reach if they don't offer some documents.
- Training is sitting on the last step. So without the steps.
- In-app training should solve this last issue due to them already being registered and in the app.
- Median minutes is very preliminary since its only been 8 days. This first is from enrollment to completion.
- Median minutes for spent on training is the entire course. Not activity.
- Out of 10 cases of average start count is > 5... more than 5 times they have tried to start training. 10 cases total, 8 are Northpass.
TODO: Look in DataDog after Doris sends time or people that are being stuck. Something else that Joonas has mentioned, enrollment process between Northpass & Edume is quite different. Differences:
- Northpass needs first, last, and email
- Edume doesn't need any information. This is helpful with GDPR.
By next Friday, Doris is hoping to have more data and details to see how the previous data changes. What Doris is happy to see, is that the activation rate of non training users is much lower. So training has specific and real effects. Between Markets, Training vs No Training is quite different in Azerbaijan and UK
12/15/2022
Bolt Prep
We have made 3 changes since their renewal:
- Updates
- Better CS
- Improve Platform Capabilities
We have made updates to the course player. Redesign Course. Help build Rich Text content. Setup in-app learning. We've accomplished everything that's been asked.
DONE: Build Slide deck for Sten
Data Recap, Reports to Pull:
- Data before Course Player Upgrades
- Improvements after Course Player Upgrades
- Improvements after end of course screen
12/16/2022
QBR
Sten: Technical Reliability is an issue. What are the specifics?
- Re-training is causing issues
- Blank Screen in web view is specific to Android Web view Issue
- Blank Screen for "many months" might be an internet connectivity issue.
- Time in Training - is this good or bad? Northpass loads times are longer than all other trainings. Why?
Baseline? They have discussed - UX researcher thinks the training should take under 10 minutes, but in reality it is closer to 5 min.
Kenya and Nigeria have the highest training attempts
- Northpass time is 10, others are 4 and 6.
Sten's biggest worry is reliability. re-training really impacts the business. AB Testing results show abnormality with Northpass platform.
01/11/2023
Mobile & App Credentials
- Just maybe ask them to thoroughly check how their app is working on mobile browsers.
- Check browser logs from mobile device if possible.
- We clearly see the difference between mobile and desktop browser behaviour, and the link we’re using is not a reason as we checked with you.
Did Joonas fix according to that SO article that Charlie sent? Supposedly, yes. It is difficult to see the impact right away. For them to see an error with a driver is two ways:
- Start rate > 5
- Driver contacts customer support with error.
If a driver has registered, then they have completed the training and all is working successfully. Joonas will be back on Monday to give us the credentials.