Creating a Self-Service Reviewer Onboarding Experience for NSF

Project Overview

I designed and launched a new onboarding flow for Research.gov that allows users to self-register as volunteer peer reviewers for proposal evaluation within the National Science Foundation (NSF). Prior to this effort, no public-facing "Volunteer to Review" path existed, and reviewers had to be manually recruited by NSF program staff. This initiative demanded a balance between user experience, strict federal data rules, and backend identity-matching constraints.
‍
Within two weeks after the release, this new self-service experience has onboarded 431 reviewers through this flow alone (out of ~1000 total in this time period with our invited-to-review flow), helping NSF dramatically expand its reviewer pool and lower administrative burden on NSF staff members.

Link to live solution: www.research.gov

Methods and Tools Used

My Contributions

As the primary UX resource on an agile product team, I was responsible for the following: 

• Designing end-to-end flows and wireframes in Axure
• Coordinating with developers, business analysts and product owners to ensure design feasibility within system constraints
• Developing wireframes into interactive prototypes with conditional logic
• End-to-end usability test planning, execution, and reporting
• Providing reviews and design quality control during development
• Planning and launching an ongoing post-production monitoring effort using Crazy Egg

Background

Context

The National Science Foundation (NSF) relies on expert peer reviewers to evaluate federal research grant proposals. Traditionally, reviewer participation has been driven by program staff outreach, where individuals are selected and invited to join review panels based on their expertise. While this approach ensures quality and relevance, it also places a continual outreach responsibility on NSF staff and limits opportunities for interested researchers to proactively express interest.
The Volunteer to Review initiative aimed to solve a structural policy gap: there was no way for qualified researchers to proactively express interest in participating. By designing a self-service onboarding flow for Research.gov, I helped introduce a scalable mechanism for self-nomination, enabling researchers to “raise their hand” to serve as reviewers.
This project was a UX problem just as much as it was a service design challenge with organizational impact. The onboarding flow supported NSF’s broader goals to:

1. Reduce administrative burden on internal staff by streamlining reviewer recruitment
2. Broaden and diversify the reviewer pool through open access and improved discoverability
   
3. Align a new user pathway with strict federal privacy regulations and legacy backend constraints

The Challenge

This project required designing not just a usable interface, but a trustworthy and policy-compliant service entry point into one of the federal government’s most critical scientific review pipelines. A core challenge was enabling self-nomination without compromising data security, identity-matching accuracy, or user trust. Many past reviewers had incomplete or mismatched records across legacy systems, and some lacked updated contact information.

Design decisions had to navigate strict privacy requirements (particularly around social security numbers and historical emails), conform to legacy backend matching logic (based on phone, name, and social security numbers to prevent duplicate user IDs being created), and work within a rigid Research.gov information architecture that could not be altered without agency approval.

Methodology: Design

Mapping the Current State and Business Rules

I collaborated with the development team to create a decision tree that captured all possible reviewer identities (e.g., new, past, unsure, changed name). This logic formed the basis of the conditional screens that guided users to the correct onboarding path.

Building the Initial Wireframes

After validating the verification flow, I then drafted wireframes of the 'Volunteer to Review' pre-check form, The design includes different cases for when the system could or couldn't verify a user's information, email one-time-password (OTP) verification, and confirmation email templates.

Axure Wireframe Folder Structure: Outlining versions of the form for each verification state.

Animated demo: One user path through the Volunteer to Review onboarding flow. This flow adapts dynamically based on user input, verification status, and data-matching outcomes.

Wireframe: Signed-out  Research.gov homepage state

Wireframe: Signed-out  Research.gov homepage state with captured feedback from stakeholders

Wireframe: Verification state for users who need to validate with phone and social security numbers (note - all scenarios were fictional, and we gave the users fake data, no real phone or social security numbers were used in this study).

Wireframe: One Time Password (OTP) modal

Wireframe: Email verification template

Methodology: Moderated Usability Test

Research Questions

Once I had finalized a set of wireframes, the next step was to plan for usability testing to validate them and identify any areas of improvement and potential long-term enhancements for the product backlog. The goal of my usability study was to address the following questions:

1. How well will users respond to all of the dynamic content on the Volunteer to Review page? 
2. How easily can users get to the Volunteer Workflow from the signed in/signed out page? 
3. Can users easily understand the instructions for providing their past reviewer names and emails? Why or why not? 

Usability Test Structure

Seven participants were recruited for this usability test. I moderated each test session remotely using Zoom, and participants were required to share their screens as they clicked through the wireframes. In order to ensure my sample accurately reflected the intended audience, I worked with my UX team and the NSF to recruit principal investigators and researchers from the academic community who had never conducted review or panel activities.

My test comprised of four scenarios:

1. Creating a Research.gov account and volunteering to review, end-to-end.
2. Accessing our Volunteer to Review informational splash page (designed by another UX team member) from our Research.gov home page.
   
3. Volunteering to review with previous contact information on file
4. Volunteering to review with no previous contact information on file.

Surveying Customer Experience

While peer reviewers aren’t traditional customers, the dynamics of trust, communication, and service expectations in their relationship with the NSF closely mirror a customer-business interaction. As part of a lightweight Voice of the Customer (VoC) effort, I asked participants open-ended questions before and after usability sessions to understand their broader experiences with NSF and Research.gov.
Questions included:

• If you wanted to find more information about reviewing for NSF, where would you go?
• How often do you visit Research.gov?
  
• When was the last time you updated your account?
• How satisfied are you with managing your profile?
• What are your expectations or past experiences with Research.gov or reviewing for NSF?

These responses uncovered service-level pain points around navigation, trust, and onboarding, which were insights that wouldn’t have emerged from task flows alone. I incorporated this VoC feedback into final leadership presentations to advocate for clearer reviewer guidance and expanded support for first-time users beyond the immediate Volunteer to Review form experience.

Surveying Screen Satisfaction

After each task, I asked the users to rate the screens they clicked through on a 5-point Likert scale from "Very Dissatisfied" through "Very Satisfied", and also asked what changes, if any, would they make to the screens tested.

At the end of each session, I also asked the users to then give an overall rating of the entire set of wireframes that they had tested, in addition to some targeted follow-up questions about their preferred methods of adding review meeting attendees.

Usability Test Results

Qualitative Analysis: What users were thinking and saying?

As a part of my analysis, I categorized key user behaviors and feedback using the Six Minds of UX framework, organizing observations into cognitive dimensions such as Vision, Wayfinding, Memory, Language, Decision Making, and Emotion.

Major Finding: Users found the Volunteer to Review form itself to be easy to navigate, but the identity verification section caused some hesitation - even in the context of fictional scenarios with fake data.

Major Finding: Getting to the Volunteer to Review form however,  from any of the NSF entry points was a major pain point for users.

Quantitative Results: What were the completion rates and satisfaction rates? 

When asked to given a satisfaction rating for all  wireframes, everyone responded with 'Satisfied' or higher. Task completion and satisfaction rates are listed below:

Major Finding: Higher completion and satisfaction rates for Tasks 3 and 4, compared to Tasks 1 and 2, aligned with qualitative observations: the primary usability barrier was locating the Volunteer to Review flow. Once users entered the form, they were generally able to complete their tasks successfully and with minimal friction.

Takeaways from Usability Testing

Key Findings - What Worked Well

• Users overall were  satisfied with the Volunteer to Review screen, and responded well to the dynamic content and identity verification steps from a task completion standpoint. There were concerns with some question wording and trust issues with providing their social security numbers.
  ‍
  Design Change: I recommended adding additional reassurance through clearer contextual messaging or inline legal guidance near the SSN field. In response, the legal team provided an expanded Privacy Act Statement, which we implemented within the modal accessible from the SSN input.

Wireframe: Expanded Privacy Act Statement

• Users were able to understand the instructions for providing previous names and email addresses, but found the wording overly long and unclear.
  
  ‍Design Change: Based on user and stakeholder feedback, I revised the formatting and streamlined the language of these questions to improve clarity and reduce cognitive load.

Wireframe: Tested question wording

Wireframe: New question wording

Key Findings - What Needs Improvement

• Wayfinding on the Research.gov homepage and sign-in screen emerged as the most significant pain point during usability testing. These screens had the lowest completion and satisfaction rates, indicating consistent user confusion in locating the Volunteer to Review workflow.
  ‍This is arguably the most important finding of the study: the volunteer onboarding process represents a new business channel that invites an entirely different user base to engage with NSF's digital platforms. If users struggle to find the entry point, they may be discouraged from engaging with the NSF altogether.
  
  Recommendation and Monitoring Strategy: At the time of testing, changes to the homepage and overall information architecture were not feasible due to broader platform constraints and the need for cross-agency sponsorship. To continue advocating for the end user, I proposed a post-launch monitoring effort using Crazy Egg to track real-world behavior.

Post-Launch Monitoring with Crazy Egg

Monitoring Plan

This ongoing monitoring effort is designed to validate usability test findings at scale by using behavioral analytics to assess whether the homepage remains a barrier for a broader user base. The goal is to supplement qualitative insights with real usage data and inform future design or content improvements with evidence.
Key areas of focus include:

• Clickthrough rates on the Volunteer to Review call-to-action links from homepage and splash pages
• Scroll depth and interaction patterns on the onboarding form itself

In addition to tracking scroll and click behavior, I collaborated with our development team to monitor account activity and conversion rates. Specifically, how many users who visited Research.gov proceeded to sign up and volunteer. My goal was to ensure that usability metrics were tied directly to business and customer success outcomes, so we could continue building solutions that deliver real value for end users and the NSF.
‍
Some snapshots of our current heatmaps are included below: 

Crazy Egg Heatmap Snapshot: Signed-in view of Research.gov home page

Crazy Egg Heatmap Snapshot: Volunteer to Review splash page

Crazy Egg Heatmap Snapshot: Volunteer to Review Form

Final Reflections

Takeaways

This was a large-scale, high-impact government onboarding project that blended customer experience strategy, legal compliance, and user empathy. I owned the experience end-to-end—leading flow design, wireframing, usability test planning and moderation, and post-launch monitoring. I consider this my flagship project because it demonstrates my ability to navigate complex, multi-stakeholder design challenges with clarity, pragmatism, and rigor.

What I'd Do Differently

If I had more time or resources for this project, I would have:

• Tested alternate question phrasing for the Volunteer to Review form to further reduce user hesitation
• Conducted card sorting or tree testing to evaluate the Research.gov homepage labeling and information architecture, particularly with the new workflow in place
• Explored mobile-first design for Volunteer to Review, including responsive prototypes and mobile usability testing. Research.gov is currently desktop-first, but mobile support may expand access and increase reviewer engagement over time. While mobile usage is currently limited, enabling mobile access may help expand reviewer participation and accessibility across demographics.