It’s tough to evaluate UX designers. User experience encompasses a lot of different skills, and as a relatively new field, there isn’t a shared understanding.
Many interviewers just go with their gut instinct, and as with all subjective decision-making, our innate biases can unintentionally influence our choices. To counter this, researchers recommend developing structured criteria across all interviews to make more accurate evaluations.
At Opower, we strive to hire the best UX designers for our line of work, while providing a fair and unbiased interview process. To create our criteria, we started with a set of skills based on UX industry standards, then added ones unique to our design problems.
We use Irene Au’s framework for 4 UX skills: user research, UX design/interaction design, visual design, and prototyping. And added a fifth one due to Opower’s unique products–content strategy.
Irene Au’s UX skills map adapted from Paul Adams’s “How to Hire Designers” chart
I won’t go into what these UX skills are since she’s written eloquently on them– but rather share how to evaluate the specific skill of interaction/UX design.
11 qualities to look for in a UX designer
Design skills
What we want to know: Can this designer do the job?
Evaluated through: portfolios, design exercises.
1. Problem setting
- Do they just solve the problem they are given? Or do they question if it is the right problem?
- Can they prioritize the top problems they are solving?
- Bonus: Can they persuade you that this is an important problem to solve?
2. User-centered process
- Do they understand where research, prototyping, and testing fits into the design process?
- Do they understand when to do generative vs. evaluative research?
- Do their user insights carry into their designs?
3. Idea generation
- Do they explore both a high quantity and high quality of ideas?
- Are they curious for new information?
- Are they able to think on their feet? Build upon the ideas of others?
- Bonus: What part of the design did they spend extra time exploring? Why?
4. Systems thinking
- Do they understand how their solution fits in the user’s life?
- Are they able to consider the whole picture while working through details?
- Bonus: Do they show where this product fits into a product ecosystem?
5. Craft
- Do they pay attention to the details? Do they know where to focus their time?
- Do they use common patterns and interaction models appropriately?
- Is their final design visually appropriate for the audience?
6. Innovation
- Do they have at least one example of an innovative solution to a problem?
- Does their work feel new or fresh, or does it look derivative?
- Bonus: Do any parts of their design methods push forward the design practice?
People skills
What we want to know: Do I want to work with this person?
Evaluated through: 1-on-1 interviews, team presentations, in-person work sessions, reference checks, cover letter and back-and-forth emails
7. Communication
- Are they an engaging presenter?
- Can they clearly articulate their design rationale?
- Are they a good listener?
- Do they answer questions directly?
8. Collaboration
- What’s their primary contribution as part of a team?
- Do they have experience collaborating with product managers? engineers? or researchers?
- Are they flexible and open to feedback?
- How do they deal with conflict or navigate disagreements?
9. Cultural contribution
- Do they represent your company’s values? At Opower, we have defined both people and company values.
- Do they contribute a unique skill or personality trait to the team?
- Do they inspire people around them?
- Is your team excited to work with them?
10. Leadership
- Do they drive groups to consensus?
- Do they take ownership of their work and decisions?
- Can they make tough decisions or take smart risks?
11. Mission
- Do they show a passion for your company and mission?
- Did they do any research on your company prior to interviewing?
- Have they done any prior work in the field?
How we use these criteria for interviewing
We use these same criteria across all designer levels, but weighted differently. For example, we don’t expect an entry-level designer to have the same level of leadership skills as a senior designer.
This is our typical process:
- Before: Each interviewer is assigned an interview type and handful of criteria to focus on — though they’re welcome to note other qualities too.
- During: In the portfolio review, the interviewer may focus on the candidate’s problem setting, process, and craft, then do a short design exercise to get a sense of their collaboration and idea generation.
- After: We use Greenhouse to individually fill out scorecards. This allows us to formulate our own opinions without group think biases. It also saves us time– we only debrief in person if our ratings don’t match.
Greenhouse scorecard for a UX designer
Hiring is an iterative process
We’ve been using these hiring criteria for 6+ months now at Opower, and like with design, we treat hiring as an iterative process.
We’ll continue to train interviewers, source/interview/evaluate candidates, and hopefully give out offers. As we do so, we’ll keep improving our interview questions and design exercises to best evaluate each of these qualities. But that will have to be for a future post…
Recommended further reading
Irene Au’s trilogy– Understanding UX Skills, Writing a Job Description for UX People, Where to find great UX candidates.
Julie Zhou– An Inside Look at Facebook’s Method for Hiring Designers