Quality Assurance Testing Best Practices By App Type

Download as PDF

Brief Overview

Quality Assurance (QA for short) is the testing phase of the WalkMe implementation. It is an opportunity to catch and resolve any errors before WalkMe content goes live to your users. This article is a set of guidelines to follow while testing.

There are a lot of similar patterns when looking for issues, but every environment is different. Being familiar with all of your website's interactive elements before testing will be largely beneficial.

Always keep the user's perspective in mind, and ask yourself these questions while testing:

• What would a user do in this scenario?
• Would a user click on this element?
• Are the instructions clear?
• Would a user need assistance here?

Before You Get Started — Pre-Requisite Knowledge

Digital Adoption Institute

Before you begin testing, we recommend that you complete the “Test” portion of the Digital Adoption Institute.

Understanding Available Testing Environments

Play Mode gives you the ability to view one piece of content at a time, but you won't see any other content that may be on the page while in Play Mode. (See Image 1)

Image 1: Enable Play Mode by hovering over content and clicking Play.

Preview Mode allows you to view WalkMe and all of your content within the browser you are working in. (See Image 2)

Publish Mode allows you to select, organize, test, and publish items that you create in the WalkMe Editor to your Test or Production environments. This will allow you and others to view the content on your site. Read our Publishing article for more information about publishing. (See Image 2)

Image 2: Preview and Publish can be accessed through the menu bar at the bottom of the Editor.

Screen Resolution Tips

When testing content, we recommend that you test on the smallest resolution or screen size you have available.  This will allow you to uncover any layout issues with the WalkMe content, as they tend to present themselves on small resolutions.  The most common issues are as follows:

• Launchers covering up website elements on smaller screens.
• Balloons interfering or covering too much on a small screen.
• Balloons being cut off the screen on a smaller resolution (See Images 3 and 4)

  

  

• Custom designed CSS might be affected and might not be consistent.
• If the site is responsive, balloons may behave oddly during the transition to a different resolution or display.

App/App-Specific Config Best Practices

Smart Walk-Thru

Smart Walk-Thrus are the most common WalkMe content to QA. Because of the many features involved in a Smart Walk-Thru, multiple run-throughs are recommended.

Walk-Thru Check

• Play the Smart Walk-Thru completely and confirm you can start from the first step and reach the last step.
  • If a step is not playing, the Smart Walk-Thru can't find the element on the page.
  • If a step is playing but not visible, this could be a layout issue.
• Check any Auto Play Rules. Starting from the homepage, navigate to the page with Auto Play Rules and confirm that it plays correctly.

Step-by-Step Check

• Although the Editor has built in spell-check, it's still a good idea to double check text for clarity, spelling, grammar, and make sure instructions are clear.
• Confirm that the balloons are pointing to the appropriate elements.

Smart Walk-Thru Start Points

Start Points: Start Points should be used on the first step every time there is a new page. (See Images 5 and 6)

Image 5:  Starting the Smart Walk-Thru on the Promotions Page with no Start Point will ask the user to click Promotions (redundant step).

Image 6: With a Start Point, the Smart Walk-Thru plays the correct step.

How to QA Start Points

• Whenever reaching a new page, start the Smart Walk-Thru from the WalkMe menu and verify that the Walk-Thru starts on the correct step.
• To properly test Start Points, start the Smart Walk-Thru from the WalkMe menu and not the Player.

Solutions for Start Points

• Adjust the Start Point rules. To verify the rule(s) are reading as true, check the Rule Engine in the Editor to see if there is a green check mark to the far right of the rule. (See Image 7)
  

  

   

Smart Walk-Thru Error Handling

How to QA Error Handling

• Test if it is possible for a user to proceed by purposely creating an error. On forms, leave out the mandatory fields and click the Submit button. (See Image 8)
  
• Image 8: Required fields in a form.
• Always test your Error Handling Group by purposely producing an error to see if it triggers correctly. Once the Error Handling detects errors in the form, try submitting with the correct information and confirm that Error Handling does not trigger incorrectly. (See Image 9)
  Image 9: The user has made an error in their form, but they received the success message because Error Handling is not used.

Solutions for Error Handling

• Adjust the Error Handling Rules. To quickly verify the Rule, check to see if there is a check mark to the far right of the Rule. Users will only be put into the Error Handling Group if the Rules evaluate to true.

Smart Walk-Thru Triggers

How to QA Triggers

• Check that the correct trigger(s) is/are being used for the action needed. Triggers must agree with the instructions listed on the balloons. (See Image 10)
  Image 10: The balloon is directing the user to click 'Add to cart,' but the  trigger for this step is the Next button.
• Evaluate if the step can be triggered with another action and needs an additional trigger. In the example in Image 11, the user can click either Register button to create their account.
  Image 11: There are two possible ways for the user to click Register, so the step must have two triggers.          
• For Click triggers, try clicking around the element and ensure it's not triggering prematurely.  (See Image 12)
  Image 12: The step triggers when the user selects a shipping type, but does it also trigger if they click the white space around the element?

Solutions for Triggers

• Create a trigger for every possible option to continue to the next step.

For an instance where multiple buttons can be clicked, a Custom trigger should be used.

Smart Walk-Thru Looking for Element Error

Locating Looking for Element

• This error is denoted by the magnifying glass icon in the Flow Tracker. (See Image 13)
  Image 13: WalkMe is unable to find the element for step # 29.

Solutions for Looking for Element

• If you are getting this error message, you may need to add a Start Point on that step.  (See Image 14)
  

  

• Otherwise, try selecting a new element with a high element grade. (See Image 15)
  Image 15: Element Grade.

If that doesn't work, try adjusting the precision settings.

Launchers

How To QA Launchers

• Do other elements overlap with the Launcher? (See Image 16)
  

  

• Verify the functionality of the Launcher and test if it is working as intended. Note: this needs to be done in either Preview Mode or a published environment. (See Image 17)
  Image 17: This Launcher action is set to play a Walk-Thru.
• If Segmentation is on the Launcher, make sure the Launcher appears on the correct pages and to the correct users.
• Verify that the Launcher does not block any important elements or text.
• Scroll up and down the web page. Does the Launcher lag behind the element?
  

Solutions for Launchers

• If the Launcher overlaps with elements, change the z-index.
• If the Launcher doesn't function correctly, check to see if an action is assigned.
• If the Launcher does not appear, or appears incorrectly, check the Segmentation Rules or Element Behavior.
• If the Launcher is blocking an element, reposition it.

If the Launcher lags behind the element it's attached to, toggle Lock to Element.

ShoutOuts

How to QA ShoutOuts

• Verify that Auto Play is working correctly. If Segmentation is on the ShoutOut, make sure the ShoutOut appears on the correct page(s) and to the correct user(s).
• Verify the functionality of the ShoutOut and test if it is working as intended. (See Image 18)
  Image 18: Clicking the hyperlink will bring the user to the Recent Purchases page

Solutions for ShoutOuts

• If the ShoutOut appears more times than intended, check the Auto Play rules.
• If the ShoutOut appears to the wrong user, adjust the Segmentation Rules.
• If the ShoutOut doesn't function correctly, check to see that an action is assigned. (See Image 19)
  

  

SmartTips

How to QA SmartTips

• Do other elements overlap with the SmartTip? (See Images 20 and 21)
  Image 20: SmartTip displayed before user clicks the Country drop-down.

  

• Verify that the SmartTip does not block any important elements or text.
• For Validation SmartTips, confirm that they give the appropriate messages when there is an error and when conditions are met.  (See Images 22 and 23)
  

  

  

• If Segmentation is on the SmartTip, make sure the SmartTip appears on the correct page(s) and to the correct user(s).
• Scroll up and down; does the SmartTip lag behind the element?

Solutions for SmartTips

• If the SmartTip overlaps with elements, change the z-Index.
• If the SmartTip does not appear or appears incorrectly, check the Segmentation rules or Element Behavior.
• If the SmartTip is blocking an element or text, reposition it.
• If validation is not functioning, check validation rules.

If the SmartTip lags behind the element it's attached to, toggle Lock to Element.

General Config Best Practices

Z-Index

Z-index is a property that specifies the vertical layering of different elements that appear on the same webpage. An element with a higher z-index value will appear in front of an element with a lower z-index value. Z-index issues occur when a WalkMe element overlays or blocks a site element.  (See Images 24 and 25)

How to QA Z-index

• Look for elements (dropdowns, pop-ups, etc.) surrounding the balloon and confirm that WalkMe appears below all site elements.

Solutions for Z-index

• Toggle Z-Index to “ON” and adjust the value as needed. Foreground elements have a higher z-index than background elements. (See Image 26)
  

  

   

Lock to Elements

How to QA Lock to Element

• Try scrolling up and down the web page and observe the behavior of the balloon. If the balloon is jumping around the page, this setting may need to be enabled. (See Images 27 and 28)
  Image 27: User scrolling with Lock to Element OFF (balloon lags behind).Image 28: User scrolling with Lock to Element ON (balloon is connected).
• Expand or collapse any interactive elements (e.g., navigation menus, drop-downs, search boxes) around the step.

Solutions for Lock to Element

• Toggle Lock to Element to “ON.”
• If Lock to Element does not work, the solution may be to select a different element on the page. (See Image 29)
  Image 29: Lock to Element enabled.

Segmentation

How to QA Segmentation

• Log in as different kinds of users and open the WalkMe Widget. Verify the segmented Smart Walk-Thrus only appear to the correct users. (See Images 30 and 31)
  

  

  

Solutions for Segmentation

• Add or modify Segmentation rules to the Smart Walk-Thrus that appear incorrectly. Make sure the rules are displayed as “true” to determine if your rule is working correctly. (See Image 32)
  

  

Goals

How to QA Goals

• Track Goals through the Flow Tracker using the Flow tab or Goals tab. (See Image 33)
• Check to see if the Goal was reached at the expected step and not triggering prematurely.
• Check that the Goal isn't triggering twice or triggering for the wrong Walk-Thru (if multiple are added).

Solutions for Goals

• If your Goal isn't triggering, add or modify your Goal rules. (See Image 34)