User Tests – Build Better Shiny Apps with Effective User Testing

1. Why User Tests
2. useR! 2021 Presentation: How to Conduct Effective User Tests
3. Good Practices
4. Q&A From Maria’s Presentation at UseR! 2021
5. useR! 2021 Follow Up

Why User Tests

User tests are paramount for creating a smoother development process and ultimately, a user-friendly tool. It’s easy to lose track of the end-game, which is adoption by users. Remember, the success of the application depends on the user experience. This is where user tests are key – they help determine if the application is intuitive, fun to use and if users will continue to use it. 

User tests at Appsilon typically involve one-on-one meetings consisting of three parts:

1. Asking the user about their daily work
2. Asking the user to perform actions in the application (If the meeting is remote, ask the user to share their screen)
3. Gathering general feedback 

See how Marcin Dubel, Shiny Contest Grand Prize Winner, applied user testing to improve his award winning game, Shark Attack!

useR! 2021 Presentation: How to Conduct Effective User Tests

by Maria Grycuk

Abstract:
User tests are a crucial part of development, yet we frequently skip over them or conduct them too late in the process. Involving users early on allows us to verify if the tool we want to build will be used or will be forgotten in the next few months. 

Another risk that rises significantly when we don’t show the product to end-users before going live is that we will build something unintuitive and difficult to use. When you are working with a product for a few months and you know every button and feature by heart, it is hard to take a step back and think about usability. 

In this talk, I would like to share a few tips on how to perform a user interview, based on my experience working with Fortune 500 clients on Shiny dashboards. I will show why conducting effective user tests is so critical, and explain how to ask the right questions to gain the most from the interview.

Good Practices

• Do not wait until the product is “good enough”
  • Let users explore the product as soon as they have something to interact with.
  • Start with a small POC with mocked features, validate ideas, and then progress to a final product.

• Limit the explanation to the minimum, do not describe anything without being asked first
  • The goal here is to see how a typical user would interact without guidance from the developer.
  • Take note if any part of the interaction is confusing.

• Ask users to perform business actions
  • Monitor flow efficiency when the user is given an indirect task (e.g., Display sales revenue from X Region).
  • Pay attention to any errors and if the user was able to recover.

• Ask users to speak aloud about their thinking process and expectations
  • This helps provide a clear understanding of any disconnect between the developer and the user’s thought process.

• Focus on how they try to achieve the goal
  • Again, monitor flow efficiency.
  • Are there any steps to add/remove that will improve the user flow?

• When asking about the general feedback, ask open, not leading questions
  • The goal here is to obtain honest, constructive feedback.

Discover the possibilities of R Shiny with Appsilon – let’s build something beautiful together!

Q&A From Maria’s Presentation at useR! 2021

Q: How long do these one-on-ones usually take, per session?
A: It depends on the app, but is usually 1.5-2h. And if the application is more complex we are usually going only through a part of it. I always try to save time for an open discussion at the end.

Q: You mentioned sharing POCs with mocked functionality – are there situations where users spend too much time stumbling over these mocks or spend too much time discussing desired functionality that is scheduled for “too far” in the future?
A: Oh yes!  This is quite common. I am open to all feedback but if I see the conversation is not going anywhere I ask the user to go to a different part of the app. Recently I had a problem with deprecated data in the application. Users were mostly focused on this for the first few minutes. But I took some time to explain to them that this is just temporary and I would like to focus on usability.

Q: What motivated your switch from Shiny to Python+React between your POC and Final App? Or in other words, what were the limitations of Shiny or R?
A: It was a complex decision. The application has some elements written in R, but the core application is written in React and Python. The main reason was the tech stack supported by the client and the number of users (it is in the thousands and will grow). Note: It was decided at the beginning so we knew we were going to develop the final application in React and Python.

useR! 2021 Follow Up

Thank you to the 1800+ members who joined the Appsilon channel and to those who participated in Maria’s presentation channel. This year’s conference was filled with informative talks and impressive keynote speakers. Appsilon is proud of our team members that gave non-sponsored talks and we are honored to help support the global R community as a platinum sponsor of useR! 2021. We are excited to see what innovations grow from the R community’s experience at useR! 2021. 

We’re Hiring!

Interested in working with the leading experts in Shiny? Appsilon is looking for creative thinkers around the globe. We’re a remote-first company, with team members in 7+ countries. Our team members are leaders in the R dev community and we take our core purpose seriously.

To preserve and improve human life through exploration and technology #purpose

We promote an inclusive work environment and strive to create a friendly team with a diverse set of skills and a commitment to excellence. Contact us and see what it’s like to work on groundbreaking projects with Fortune 500 companies, NGOs, and non-profit organizations.

Appsilon is hiring for remote roles! See our Careers page for all open positions, including a React Developer and R Shiny Developers. Join Appsilon and work on groundbreaking projects with the world’s most influential Fortune 500 companies.

Article User Tests – Build Better Shiny Apps with Effective User Testing comes from Appsilon | End­ to­ End Data Science Solutions.