Beta-testing is a critical stage of app development that a lot of indie developers don’t spend enough time on. I know, because I was one of them.
Many indies build their app and they test it within their personal network and perhaps get their friends to test the app, but don’t test the app with people who are the target users of their app. Some developers find it enough to just test an app themselves before releasing it, but a personal test run-through can never catch bugs which might only happen in the real use case.
Wait, why is this a big deal?
Mostly, this leads to some usability challenges with the app; i.e. function x and y are not immediately obvious to the user, which is the difference between a 4 or 5 star review. That’s not good, but it’s not crippling either.
However, not testing the app thoroughly can lead to a bug on a settings screen that only happens under x circumstances, but causes the app to delete all of the users data from their session. These are the kinds of bugs that lead people to angrily leave 1 star reviews on your iTunes page.
Reviews don’t go away once you update your app; reviews continue to affect your app reviews’ average for the rest of it’s lifetime. And like your grades in college, if you get all C’s your first semester, you’re going to spend the next three years trying to drag that grade point average up. It’s even worse with apps - with apps, there’s no graduation finish line.
This is the mistake I made, and it led to a series of one and two star reviews which continue to drag down the rating of Pufferfish Software apps, even two and three brand new versions later. Not making the apps execute well the first time was one of the biggest mistakes I made.
How to (Properly) Beta-Test an App
In order to avoid this poor outcome, make sure the people beta-testing the app are real potential users. Better yet, beta-test the app in the real circumstances that they’d be using the app.
Is it an app to help Minecraft gamers have information when they need? Have them play minecraft, and use the app when they need information. Is it an app to help people develop characters while writing a fiction novel? Have them use the app to develop their characters while actually writing their novel.
This may all seem obvious, but it is meant to illustrate that what you should not do is just hand the app to them and say “use it,” which is what people end up doing. We all know we should really beta-test with our actual users, but fail to do so.
There is a problem with the deliberate testing style of just saying “use it,” which is that they’re aware they’re testing. People treat an app differently if they know they’re testing it, treat an app differently if you’re watching. People need to test the app the way they’d use it; in the spare 2 seconds before they can turn left on the road, sitting on the subway from work to home, lounging on the couch watching tv.
You could just give them the app to use this way, but you can’t watch your users use the app. You can only receive commentary back, and unless your beta team is a team of designers, chances are slim you’ll receive great design & use feedback. The feedback will be all bug-oriented. Make sure to watch some people use the app in real-time, to draw conclusions by yourself.
When letting people beta-test your app, do not help them use the app. If they can’t understand a gesture in the app, only help them out when otherwise they’ll have stopped using the app - this tells you where the pain point is for your app, and you can fix these by providing new controls, gestures, or repairing bugs.
A fine thing to look at is what gestures they try to use, regardless of whether or not they actually exist yet. The user isn’t wrong when they use an unsupported gesture, you just need to add it if enough users try and use it.
This is good for seeing those little things with use that you’re missing, not for identifying a bug. You need to see people interacting with your app and the world, and since usually you can’t go creeping around convenience stores and watching people, this will have to do.
How Do I Get My App on Their Phone?
It’s really difficult to get the app on many potential users’ devices, because you have to 1) find these users, and 2) put the beta app on their device. Luckily, there exist free and paid services to get the job done for you. You can install your beta app over the internet with:
This is great because it will be on their device whenever they want to use it, and they’ll experience it like they would any other app. You can get the app on users devices, even if they’re across the globe, and use it to collect feedback about the app.
A lot of these services have paid options, that tell you which views of the app people go to, how long they use them, what features they use, and variables like that. You can identify extraneous parts of your app, and you can catch members of your team saying they beta-tested something when they didn’t.
This means you release a bug-free app the first time, and you can release something people will really love the next. However, the free versions are easily adequate for allowing you to appropriately beta-test your app.
Where do I find beta-testers?
You can offer the beta to people who are interested in beta software - there are websites which are directories of beta software, where people interested in beta-testing can sign up to test. People like using these services because they get to be part of something from the ground floor up, and participate in the community.
However, this route tends to have a limited audience of high-tech power-users, and if your target market isn’t young and technical people, you may have difficulty with this direction.
There are also other places on the internet where you can find people interested in beta-testing software - you can post on relevant subreddits asking people if they’re interested in beta-testing software, or find other forums with a relevant userbase.
You can also reach out to people who signed up for a wait list / newsletter or followed your app page on social media to solicit beta testers. An email list of interested people is one of the greatest assets you have.
You can offer some of your future service or app for free in exchange for being a beta-tester, which gets a lot of people interested because they hear “free stuff!” This is especially great if you make it exceedingly easy to download the beta and don’t require any active feedback, and use automatic data collectors from services outlined above.
Takeaways:
- Beta-test with real users in real situations
- Test every function thoroughly, and don’t skip any elements.
- Don’t just do a 10-minute function runthrough when you test
Nice article. You can also find various Beta Testers at “BetaPage” a platform where Founders/Innovators can submit their startup, for getting feedbacks, beta subscription, early adopters, traffic and users.
For more information visit https://betapage.co/
Thank you.