Quality Assurance Testing
Testing WalkMe items prior to publishing is essential to ensure an optimal user experience. Checking that everything works as intended will save the user time and create a more smooth and efficient experience. WalkMe provides a number of resources for help with testing Walk-Thrus, including the Flow Tracker and a QA Template.
The Short Version
Before publishing to your Production environment, it is critical to run a quality assurance (QA) test to make sure that everything is working properly. We recommend having an individual other than the content builder perform the QA test.
When testing, the tester should try to get into the frame of mind of a new user who is unfamiliar with the site. The tester should aim to replicate common user behavior and ensure that WalkMe provides clarity and guides the user through necessary tasks or queries with ease and efficiency. In addition to finding flaws in WalkMe content, the tester should feel free to suggest any ideas that might add value.
Use Cases
QA testing use cases include the following:
- Ensure that Walk-Thrus play properly from beginning to end without stopping in the middle.
- Ensure that WalkMe items are consistent in appearance and content.
- Ensure that WalkMe works consistently across all desired browsers.
- Ensure that relevant WalkMe Items are visible to their target audiences.
How It Works
The best way to test Walk-Thrus is in a “neutral” environment, without the Editor (using a test snippet or the extension). This is very important because it simulates an authentic user experience.
The QA tester should use the Flow Tracker (see link in Related Resources section below) to see real-time information about Walk-Thrus and Goals as they are previewed. The Flow Tracker displays information including step number, step status, and any Rules applied to the step. In a separate tab, the Flow Tracker also displays whether Goals are being met and displays alerts for any settings configured incorrectly on the back end.
QA testing typically takes place in two rounds: the “Full Round” and the “Sanity Check.”
Order of Operations
1) Enable the Flow Tracker
- From the Editor, open the Settings menu and go to the General tab.
- Toggle Flow Tracker on for both Preview Mode and Play Mode.
- Turn on the published environment as well if you want to be able to open the Flow Tracker there; don't worry, the end user will not see it.
- Click Save and close the Settings window.
- Click Preview to view your implementation from the Editor. The Flow Tracker will appear in its minimized view.
To view the Flow Tracker in the published environment, access your Production or Testing environment, open the WalkMe Menu, and type "walkmetracker" in the search bar.
The Flow Tracker will immediately appear in the background. This will only work if you have turned on Flow Tracker in the published environment before you publish your items.
2) Full Round QA
Perform a Full Round QA. Test from all the pages within the process. The QA Template details all things to test, including the following:
- Layout Issues: Make sure that Walk-Thrus, Launchers and all WalkMe elements appear cohesively and intuitively on the page, that text is correct, important elements are visible on the page, balloons don't cover important parts of the site, and colors and design match the site environment.
- Alerts: Alerts will display on a separate tab of the Flow Tracker. Usually there's one alert saying that the extension is active on this page (this one can be ignored). Take a screenshot of any other alert messages (click the icon to display the alerts).
- Walk-Thru Goals: The best time to check Goals is in the initial Sanity Test. Make sure the Goals are reached in the Flow Tracker and that the timing makes sense. Each Walk-Thru should include Goals.
- Onboarding Goals: These should be set at the end of the process to ensure the task won't be scratched off the Task List before the process has been completed.
- Launchers: Make sure that all Launchers look good and are working properly. Return to these pages after the sanity check to see if they work properly. Make sure they play the Walk-Thru they are supposed to play. Make sure they are located and segmented correctly.
3) Final Check (AKA "Sanity Check")
Finally, run a Sanity QA check. enter the testing environment and follow the Walk-Thru exactly as instructed in the balloons. Run the Walk-Thru A-to-Z once or twice on different browsers. The best practice is to run the sanity check on a smaller screen to check that everything still appears correctly and on the lowest supported browser versions.
Using the QA Template
The QA Template (see link in Related Resources section below) can be used to keep track of the testing process so the builder can return to address issues after the full test.
If the Walk-Thru worked well, log it and mark it Green. If there is an issue with the Walk-Thru, log it and mark it Red.
QA Questions
The chart below summarizes the items to check and questions to ask when testing a build:
| FEATURE | WHAT TO CHECK | QUESTIONS |
|---|---|---|
| Walk-Thrus | Best Practices | Read about Walk-Thru Best Practices |
| Normal Flow A to B |
Does the Walk-Thru play properly without breaking? Do all trigger options work correctly? |
|
| Layout Issues |
Is the balloon layout appropriate in relation to screen size? (Note: balloons may look fine when building on a large monitor but change on smaller screen sizes) Check that all clickable elements are clearly defined (if not, consider using highlighter around the element) Do balloons block important information on the screen? |
|
| Consistency, Language and Spelling |
Are instructions clear and concise? Is terminology consistent? Does capitalization match the text in buttons on the page? |
|
| Challenge Testing |
What happens is there is user error? (i.e., leaving mandatory fields blank, typing incorrect email/password, clicking the wrong button, etc.) What happens if the user encounters site error notifications? Does the Walk-Thru start from the appropriate step if the user is already in the middle of the process? Multiple triggers (i.e., two Save buttons that do the same action) |
|
| Goal Completion | Are Goals and Milestones met as expected? | |
| Safe Start | Do Walk-Thrus redirect to a page where it can start or show an error? | |
| Launchers and SmartTips | Position | Do Launchers appear correctly for different screen sizes? |
| Visibility | Does the Launcher stay in place when scrolling the page? | |
| Associated Actions | Are all items that are triggered by the Launcher also published? | |
| Resources and Shuttles | Check Functionality | Did the Resource play / open as expected? |
| Goals | Are Goals reached? | |
| Keywords | Does the Resource appear in search results? | |
| Security | If opening a Resource article or video in a lightbox, check for security protocol conflicts. Note that HTTP Resources cannot open in a lightbox on an HTTPS domain. | |
| Make sure the video is either in YouTube or Vimeo format and that there are no passwords required to view the video / article | ||
| Onboarding | Name & Description | Are there typos or spelling errors? |
| Related Items | Are all related items published (or marked to be published)? | |
| Task Completion |
Do tasks get crossed off when completed? In case tasks are already crossed off: |
|
| Activation Rules | Does the task become active at the right time? | |
| Task Arrangement | Are tasks aligned in logical order in the menu? | |
| Is WalkMe tracking unique users? | ||
| Surveys | Language and Spelling | Do questions make sense? Are there spelling mistakes? |
| Segmentation | Does the survey appear only at the relevant place and time? | |
| ShoutOut | Language and Spelling | Is the announcement clear and concise? |
| Action Button | Does the Action button trigger the right WalkMe item? | |
| Auto Play |
Does the ShoutOut Auto Play when it's supposed to? Do other items Auto Play at the same time? |
|
| Goals | Are Goals reached? (Use the Flow Tracker to check Goals) | |
| General | Custom CSS | Is design consistent throughout all WalkMe items and in line with the site design? |
| Segmentation |
How many segments are there? Are Rules set for segments valid both to Sandbox and Production environments? |
|
| Auto Play |
Which Walk-Thrus are set to play automatically? Are the Rules set for these Walk-Thrus valid both to Sandbox and Production environments? |
|
| WalkMe Menu |
Is the WalkMe Menu in the best position? Is it working properly? |
|
| Search |
What search providers are defined in the account? Does the Walk-Thru's search work? |
|
| Help Desk |
Where does it point to? Is it working properly? |
|
| Publish | Is all content published? |