Accelerating Application Security Testing

Project Brief

I was a UX designer charged with studying the profile of the company’s users, and through this project, we sought to develop a comprehensive knowledge base that could be leveraged for strategic decision-making in a new Application Security Testing product that was already in development. This aimed to fill a gap in our existing product design process, where our understanding of our users was based on assumptions and occasionally unjustified extrapolations based on the user base of a related, but altogether different product.

As advocates of a shift left approach in application security testing, that is, earlier security testing in the software development lifecycle, we were interested in studying the current landscape of how companies could be encouraged to build more robust processes and prevent the accumulation of expensive technical debt. We wanted to know what the existing behaviours of development teams were and what challenges they faced with managing security vulnerabilities.

ROLE

User Researcher

Tools

Miro
Whimsical
Figma

Research Process

Problem

We started with a stocktake of what we knew. As a team, we were familiar with the needs of our clients within an existing product that we had, where the majority of them tended to make up large, multinational corporations (MNCs) that would invest significant amounts in application security. However, the new product that we sought to develop was intended to target small to medium-sized enterprises that would not be able to afford the same price tags as their larger counterparts. The gap was immediately obvious - the large MNCs tended to have sizeable dedicated application security teams and therefore had different needs and expectations due to their internal levels of expert knowledge. In comparison, smaller organisations that did not have such budgets most likely functioned differently, and in some cases may not have application security experts at all. In that scenario, who was responsible? What did their process look like? What kind of pain-points did they encounter, and what kind of specialised needs did they have?

Approach

In order to adequately answer these questions, we sought to take a hybrid, mixed-methods approach so that a more holistic picture can be painted. We began with a series of qualitative user interviews, designed to be exploratory in nature. We conducted a total of 15 interviews spanning 45 minutes each, all of them recruited through a third-party recruitment tool. To make sense of the data, affinity diagramming was used to identify themes and develop our findings. After that, based on the results of the study, we embarked on a limited-size quantitative study to triangulate our findings in the prior study through a 10-minute survey, also conducted through a third-party recruitment firm.

There were two important considerations at play here that needed to be carefully balanced. First, we needed a fresh sample of people made up of people who were not existing users of our product. Apart from wanting to avoid overburdening the well that was already frequently dug from, we were conscious of the fact that this product was meant to seek a new market - as such, we needed to probe beyond the bubble that we were existing in. At the same time, we had to be mindful of our costs as we were working on a limited budget, with resources reallocated elsewhere. Hence, we had to ensure a maximum return on this project.

Conclusions

Through this process, we were able to uncover three things - the ways in which application security interacted with and was involved in the software development lifecycle across different companies; four personas, and a framework for classifying companies based on their application security processes.

As a result of this research effort, we created a foundation that would enable the development of a product that could target users with better precision. It also exposed opportunities for design improvement and areas that require more detailed, in-depth examination, such as a content analysis to study whether we use language effectively, and in turn, improve the overall user experience of our product.

Notes

As this product is still ongoing development, I have opted to obscure the specific findings of this research project.