GOAPPLY REVIEWER ADMIN GUIDE

 

INTRODUCTION 

 

GOapply applications can be reviewed by both internal and external reviewers using the GOapply review features. These features allow the user to create and manage specific review groups for particular phases of their submitted applications. In addition, GOapply Users can be made reviewers and can be added to as many review groups, as necessary.  

 

The GOapply review features are divided into several components, which can be found on the left-hand navigation pane in the GOapply area.

 

 

Tip: We offer a Suggested Learning Course for GOapply Reviewer, which includes several instructional videos. GOapply Reviewer Course

OVERVIEW OF GOAPPLY REVIEWER COMPONENTS 

 

GOapply Reviewer references many of the same tables that you may be familiar with from using GOapply. Users, Applications, and Status Tracking records are all employed in Reviewer, in addition to several Reviewer-specific tables. These include Reviewers and Review Groups. Below is a high-level overview of Reviewer tables; we will dive deeper into these topics later in this document. 

 

Reviewers are GOapply Users with a role of Reviewer or Applicant and Reviewer.

To create GOapply Users for Reviewers, there are two options:

1. The Reviewer can register for a GOapply account independently through the registration process in the GOapply UI. If reviewers will register on their own, the process is the same as applicant registration, which we will walk through step-by-step later in this document.  

2. You have the option of creating their GOapply user records for them using the Add GOapply Users process. Steps for manually creating GOapply Users are also linked later in this document.

 

On a Review Group, you will list an Opportunity and Phase. You can see which Review Groups an Opportunity or Phase is related to by choosing Related > Review Groups

 

Status Tracking records are what informs Reviewer which phases/applications are ready for review.  

 

GOapply Admins can upload an alternative file to be displayed for the reviewer. In the Status Tracking Record, there is an upload field called Reviewer File. Here is where you would upload what you want displayed to the reviewer if it is different from the submitted PDF. This file will override the submitted PDF.

 

 

Reviewers are all GOapply Users that have the Reviewer or Applicant and Reviewer role assigned. 

 

Review Groups will be set up for each Review cycle. This is where you link the people who will Review (Reviewers) with the submissions they will review. Please note Reviewers will not have access to any submissions to review until they have been added to Review Group.

 

Review Group Applications are a hidden table. Review Group Applications use the GOapply Status Tracking record to display the application for the reviewer. If the GOapply Status Tracking record does not exist or does not contain a PDF application, the application will not display correctly in the reviewer dashboard.  

 

In all areas of akoyaGO CRM, you have a system dashboard called GOapply Reviewer Dashboard that tracks applications that need to be assigned to a review group, applications with a conflict of interest, applications with scores from review groups, and active reviewer request scores. 

 

GETTING STARTED 

 

Now that you know how the GOapply Reviewer tables work together, let’s walk through setting up a Review Group and adding Reviewers. 

 

Each review group consists of general info (including the application and phase), settings to determine what reviewers can see, the reviewers themselves, and submissions they will review. All of these components are necessary for reviewers to have everything they need.  

 

The following steps detail the process to create a completed review group.  

 

1. From the Review Group table, click “New Review Group.”  

 

 

The New Review Group form will load.  

 

2. Name your review group.  

 

3. Select the opportunity and phase that the review group will be assigned to review. It is our best practice that each review group will review one opportunity and phase. However, if your review group will be reviewing multiple phases, there are steps for this at the end of this Knowledge Article in the Manually Adding a Status Tracking section.

 

4. If you would like to set a due date, set a review group deadline in the Review By field. Once the Review By date has passed, the review group will no longer appear in the reviewer’s list in GOapply. This field is optional.

 

5. Lead Reviewer Recommends: This settings allows your lead reviewer to recommend which application gets awarded and if awarded, the amount awarded. This feature will ONLY apply to the reviewer where Lead Reviewer = Yes. Choose from the drop down

• No (or null): This feature is turned off 

• Recommend to Request: If chosen, this setting will map the lead reviewer's choices to the associated Request record. When this option is chosen, an additional question will be added to your reviewer form called Recommended Award Status.

The lead reviewer can choose Award or Deny. If Award, an additional Recommended Amount field will be added where they will input the amount they recommend to award. The amount field will be required.

 

Once submitted, their choices will be mapped to the Request. A field called Review Recommender will populate with the GOapply User (Process tab > Decision section).

 

 

The Recommended Grant amount will also fill in with the amount they recommended. Their choices will also map to the Review Group Application. 

 

• Recommend to Review Group: If selected, this option will also add the Recommended Award Status (and Recommended Amount field if Award) questions to the review form. This option will also map the lead reviewers choices to the Review Group Application. This option will NOT fill in any field on the request. 

6. Click Save to create the Review Group. Click the Settings tab of the newly created Review Group to continue choosing additional settings for this Review Group. 

 

 

The Settings tab contains various customization options for each review group. These settings involve the information you wish to share with the reviewers. All but Anonymous Review will default to Yes. Anonymous Review will default to No. Scoring Type will default to Sum (total score).

 

 

Show Other Reviewers Comments: If Yes, the Reviewer Comments button will be visible on the application review form. If set to No, the Reviewer Comments button will not be visible on the application review form. Please note that if Show Other Reviewers Comments is set to No, the Show Reviewer Weighted Score setting is rendered null, and the Public setting in Form Builder (page 10 of this guide) will also be null.  

 

If Show Other Reviewers Comments is Yes:  

 

 

If No:  

 

 

Show Mark for Conflict of Interest: If set to Yes, the Conflict of Interest checkbox will appear on the Reviewer’s review form. If set to No, the Conflict of Interest checkbox will not be visible on that form. 

 

 

 

Anonymous Review: Select Yes if you want applicants to appear anonymous. If Yes is selected, the applicant’s name will be removed from the PDF that the reviewer sees. The reviewer PDF will be stored in a field called Reviewer File in the GOapply Status Tracking record. 

 

Show Reviewer Weighted Score: If Yes, reviewers will be able to see how other reviewers scored the application. It renders as the far right column in the Reviewer Comments. If No, the weighted score column is not visible.  

 

 

Scoring Type: Choose Sum, Average or Sum by Reviewer then Average by Application.  

 

 

 

GOapply Review Groups have command bar buttons which will duplicate the review group or the review group plus the reviewers.

 

In the case that you need to duplicate a Review Group but you are planning to add new Reviewers, use the Duplicate Review Group button.

 

From the Review Group form, choose the Duplicate Review Group command bar button.

 

 

You will be prompted to name the new review group.

 

 

This function duplicates application, phase, review by date and time, review form definition, review form and all properties, all settings, show staff comments and attachments setting, and staff notes. 

 

In the case that you will be re-using a Review Group and plan to have the same reviewers as members, use the Duplicate Review Group + Reviewers command bar button.

 

 

Give the new review group a name.

 

 

This process will duplicate everything the above process does, except it will also add all existing reviewers as members.

 

 

 

Reviewers are the GOapply users that will be evaluating the applications in their review groups. Reviewers must be added to each review group they are reviewing applications for. In order to be added to a Review Group, the user(s) must have the Reviewer or Applicant and Reviewer role.

 

Add in Bulk:

 

From a GOapply Users view, select the users you wish to add to the Review Group. Then click the Add to Review Group command bar button. 

 

 

Next, you will see a popup box called Add to Review Group. Select a review group from the dropdown list provided and click Submit. Please note, only active review groups will appear in the list.

 

 

Once the users are added to the Review Group, you will see a success popup.

 

 

*Please note that if you run this process on a user whose Role does NOT include Reviewer, they will be switched to the Applicant and Reviewer role.

 

Add Individually:

 

From the Reviewers tab of the review group, click New Reviewer in the All Reviewers subgrid. 

 

The New Reviewer form will populate with the review group information. Review Complete and Lead Reviewer will default to No. 

 

 

Select a GOapply User in the Reviewer field. If the user is the lead reviewer, make that designation in their reviewer record. Lead Reviewers will be able to see all reviewers' comments regardless of the Show Reviewers Comments setting in the review group settings. Please note that the intended functionality is to have one lead reviewer per review group. If you try to set another reviewer as lead reviewer, it will override the previous lead reviewer. 

 

Review Complete: this field will automatically flip to Yes when the Reviewer has completed reviewing all the applications in all the review groups that they are assigned to. 

 

The Reviewers tab of the Review Group record are three subgrids: All Reviewers, In Progress, and Complete.

 

 

 

 

 

When Review Complete flips to Yes, the Reviewer will receive an email thanking them for finishing their review. This email is determined by an Email Template called 'GOapply Reviewer Completed Email'. 

 

 

 

Bulk Import

Rather than adding reviewers one by one, GOapply Administrators also have the option to bulk import reviewers by using the GOapply Add Users table. Steps for this can be found in this related Knowledge Article: Add GOapply Users

 

 

Add GOapply Users Manually

You can also enter users into the Add GOapply Users table manually or on a one-off basis. If you have less than approximately 20 GOapply users, you may find creating them this way is less work than completing a bulk import.

 

Click the +New command bar button.

 

 

Fill in the form. 

 

 

 

• First Name: The GOapply User's First Name
• Last Name: The GOapply User's Last Name
• Applicant Type: Individual or Organization. If you are creating Reviewers, the Applicant Type should be set to Individual. 
• Parent Constituent: This field will only appear if the Applicant Type is Organization. Select the constituent in akoyaGO that this user will be applying on behalf of.
• Email: The GOapply User's email address. An email will be sent when you run the flow to create this person as a GOapply user (more information on this in the Bulk Import Users section below)
• Role: Applicant, Reviewer, or Applicant and Reviewer. This field determines how GOapply will appear to this user. If set to Applicant, the user will see items related to applying in the GOapply UI, such as Opportunities, In Progress drafts, and past Submissions. If set to Reviewer, they will see Review Groups. If set to Applicant and Reviewer, they will see both.
• User Permissions: Submitter or Contributor. Make sure to set this to Submitter if you are creating Reviewers, so they can submit their Reviews. This field should only be set to Contributor for Organization users who should only be able to contribute to an application (they cannot submit.)
• Review Group: optional field; if this user should belong to a specific Review Group, list it here

 

 

The Staff Attachments and Comments tab in the Review Group record allows for staff notes that apply to ALL applications for that review group. Attachments can also be added here as public documents for the review group.  

 

These notes and attachments are displayed above the review group applications. 

 

 

REVIEW FORMS 

 

Review forms are displayed for the reviewer side by side with the application. Question types include a text field, a radio button, a drop down, a long text comment, and a rich text editor. For example, below you’ll see the submitted application (or whatever is in the Reviewer file tab of the Status Tracking record) on the left, and the reviewer’s questionnaire on the right. At the bottom of the page they can save a draft or submit.

 

 

 

From the review group, navigate to the Form Builder tab. Select Launch Form Editor. Please note you must be a member of the GOmanager Security Group to access the GOmanager Form Editor.

 

 

The Form Editor will load in a separate tab.  

 

 

Unlike the form editor in GOapply, the form editor in Reviewer is only able to support one page.

 

Please ensure you design your form to account for this. You will be familiar with this Form Editor; it is the same one you use to create your GOapply Applications. On the right-hand side, you will see the 6 question types. Drag and drop the question types from the toolbox into the form to add.  

 

Keep in mind that if the Lead Reviewer Recommends setting in the Review Group settings = Recommend to Request or Recommend to Review Group, potentially two additional questions will appear on your form.

 

 

There are five types of question: a text field, a radio button, a drop down, a long text comment, and a rich text editor. Please note that only Radiogroup and Drop-Down field types can have score values assigned. 

 

Single Input: a text or number response field. These fields cannot be used to calculate scores.  

 

Radiogroup: a single selection radio button style multiple choice. These responses can have score values assigned.  

 

Drop Down: a drop-down list of responses. These responses can have score values assigned.  

 

Comment: a long text or number response field. This field cannot be used to calculate scores.  

 

Editor: A field that is usually used to convey information to the applicant. This field cannot be used to calculate scores. 

 

File: This is a file upload field. Add this field if you want reviewers to be able to upload files. Supported file types include 'txt', 'rtf', 'doc', 'docx', 'xls', 'xlsx', 'png', 'jpg', 'jpeg', 'gif', 'csv', 'ppt', 'pptx', 'pdf', 'mp4', and 'heic'. Max size for all uploads total is 20MB. When a reviewer uploads a file with their review form, it will be stored in the Review Response record for that question. For example:

 

If the question type is a type used to calculate reviewer scores, the scoring must be included in the choices for that question.  

 

 

Each question field has a properties menu. Open properties by clicking on the question and selecting Properties from the far right.

 

 

Common settings for each criteria include:  

 

Required: Required setting can be found in the General dropdown in the form of a toggle called “Is Required?” If on, these criteria must be completed for the reviewer to mark the review as “complete.” Required criteria will be denoted with an asterisk in the review form. Required will default to no.  

 

Weight: Weight can be found under akoyaGO. This functionality is only available if the Scoring Type in the Review Group’s setting is set to Average. If the criteria are averaged, enter the weight of the score. Note: if averaging is used, the weights of all criteria must total 100.00.  

 

Public: This setting can be found in the akoyaGO dropdown. If checked, the criteria responses will be shared with the review group in the Reviewer Comments tab. Note: the review group setting Show Other Reviewers Comments must also be turned on for this to appear. Lead Reviewer will be able to see all reviewer responses. 

 

 

Scoring can be set on the dropdown or radiogroup question types only.

 

1. In the Properties for the scoring field, drop down Choices 

 

 

2. For Value, enter the option, then two colons, then the numerical value of that choice. The proper format for this formula is no spaces. For example, Agree::80. In the Text field, simply enter the option that you want the reviewer to see. In this example, you’d just put Agree. Repeat this process for each option in the drop down.

 

If your Review Group settings designate the scoring type as Average, you will see a property called Weight under the akoyaGO section. Here is where you will set the weight of each scored question. Weight is only available for drop down and radiogroup question types.

 

 

 

3. Once you have added all questions, click Save at the bottom of the Form Editor or wait for auto-save to take place.

Back on the Review Group in akoyaGO, you will notice the Review Form Definition field has now populated with its unique string of code and has locked to prevent accidental deletion. This Form Definition JSON File will be created & locked upon entering the Form Editor and saving a copy of the Review Form. To download the Review Form Definition, click the link in the field. 

 

 

Now that you have created your review group, designated reviewers, and created the review form, it is time to add the submissions for the reviewers to review.  

 

There are four options to add submissions to the Review Group (see section below.)

 

Please note the System View that determines what Reviewers will see in the GOapply UI is accessed through the Review Group Application table and is called “GOapply Review”. Through this view, you can manipulate the applications that a reviewer sees. For example, deactivating a record in that view would remove it from the list of applications that the reviewer sees. If you customize the columns in the System View, it will change the columns the Reviewers see in their Dashboard in the GOapply UI.

 

 

Next, you’ll assign the submitted applications to the Review Group. There are four supported ways to do this, as well as one manual way detailed at the end of this section.

 

1. Using the GOapply Add All Applications to Review Group flow (access directly from Review Group)

This options will allow you to add ALL submissions matching the Phase and Opportunity set on the Review Group in bulk. You can access the flow from the Review Group by choosing "Flow" from the top navigation bar.

For example, for this Review Group, the flow will add all submitted Status Trackings for Opportunity 001007, Phase 001085:

 

2. Add submissions directly from a Request record or Requests view:

If you have more than one Review Group reviewing the set of applications, you will not want to use the flow mentioned above. Instead, you will want to pick and choose which Requests each Review Group will review. You can do this for multiple Requests from a view, or from an individual Request record by clicking the Add to Review Group command bar button. You may want to create a Personal View, filtering by columns such as GOapply Opportunity and GOapply Phase.

 

Here is the button from a View on Requests:

The button is also available on an individual Request record: 

 

When you click this button,  you will be asked to select the Review Group you would like to add the Request(s) to:

 

*Note: this function validates based on the GOapply Opportunity and Phase set on the Review Group. For example, if you have an LOI Phase and an Application Phase that were submitted for this Request, the correct Phase will be chosen based on what is set on the Review Group.

This also means if the phase is different than what is set on  the Review Group, you will not be allowed to add Requests this way. If you need to add a Request to a Review Group that does not match the Phase and Opportunity set on the Review Group, you will need to do it manually. There are instructions for manually adding a Status Tracking record to a Review Group at the end of this section. 

 

3. Add submissions directly from a Status Tracking record or Status Tracking view:

You also have the option to add submissions from a Status Tracking record. You can do this for multiple Status Tracking records from a view, or from an individual Status Tracking record by clicking the Add to Review Group command bar button. You may want to create a Personal View, filtering by columns such as GOapply Opportunity and GOapply Phase.

 

Here is the button from a View on Status Tracking:

The button is also available on an individual Status Tracking record: 

 

When you click this button,  you will be asked to select the Review Group you would like to add the Status Tracking record(s) to:

 

*Note: this function validates based on the GOapply Opportunity and Phase set on the Review Group.  This also means if the phase is different than what is set in the Review Group, you will not be allowed to add Status Tracking records this way. If you need to add a Status Tracking to a Review Group that does not match the Opportunity and Phase set on the Review Group, you will need to do it manually. There are instructions for manually adding a Status Tracking record to a Review Group at the end of this section. 

 

 

4. Add submissions directly from a Requested Scholarship record or Requested Scholarships view:

Your last option is to add submissions from a Requested Scholarship record. You can do this for multiple Requested Scholarship records from a view, or from an individual Requested Scholarship record by clicking the Add to Review Group command bar button. You may want to create a Personal View, filtering by columns such as Scholaship and the GOapply Opportunity and GOapply Phase from the related Request. This option is particularly useful if you have different Review Groups set up for each Scholarship.

 

Here is the button from an individual Requested Scholarship record:

 

 

Here is the button from a View of Requested Scholarships: 

When you click this button, you will be asked to select the Review Group you would like to add the Requested Scholarship(s) to.

 

You can also access Requested Scholarship records from a Scholarship record by choosing Related > Requested Scholarships. 

 

*Note: this function validates based on the GOapply Opportunity and Phase set on the Review Group.  This also means if the phase is different than what is set in the Review Group, you will not be allowed to add Requested Scholarship records this way. If you need to add a Status Tracking to a Review Group that does not match the Opportunity and Phase set on the Review Group, you will need to do it manually. There are instructions for manually adding a Status Tracking record to a Review Group at the end of this section. 

 

MANUALLY ADDING A STATUS TRACKING

If you would like to add a submission that does NOT match the GOapply Opportunity and Phase set on the Review Group, you need to do so manually. 

 

1. Starting from the submitted Status Tracking record, click on the Related tab, and choose Review Group Applications. 

 

 

2. Click +New Review Group Applications 

 

 

3. The New Review Group Application form will open and autofill the Request and Status Tracking field. All you need to do is assign the Review Group. The Recommended Amount and Recommended Award Status fields will only populate if the Lead Reviewer Recommends setting in the Review Group = Recommend to Request or Recommend to Review Group. If this setting is not being utilized for the review group, those fields will not auto populate. Save.

 

 

 

Now, the submission(s) you just added will appear in the review group’s dashboard in GOapply. Repeat this process for each submission that the review group needs to review.  

 

REVIEWER RESPONSES 

 

Reviewer responses are recorded in the Review Responses table, Reviewer Request Score table, in the Reviewer Scores tab of the Review Group Application table, and in the Responses tab of the Reviewer record.  

 

 

Review Responses is a table (not included in site navigation) that stores every response from each reviewer. For example, if your reviewer form has five questions, every time a reviewer submits the form, five reviewer response records will be created.

 

Review Responses can be found in the Responses tab of a reviewer record, or in the Related tab of the Review Group record.

Inside a Review Response record, you will see:

 

• Question Type: the type of question from the Form Builder toolbox, i.e., radiogroup, dropdown, text, file.

 

REVIEWER REQUEST SCORE 

 

Reviewer Request Score records contain: 

 

 

REVIEWER RECORD         

 

Reviewer responses and scores can also be seen from within the Reviewer record. 

 

 

REVIEWER REPORTS 

 

From a Review Group, you can run a report which will show you all responses from all Reviewers for all Requests they have scored. To access this report, choose Reports from the top navigation bar (you may need to click the three dots to expand) and choose GOapply Review Responses:

 

 

You can also run a report from an individual Request by choosing Run Report in the top navigation bar and choosing GOapply Review Responses:

 

 

 

 

Lastly, there are scholarship specific reports that you can run from a Review Response record or group of Review Responses. To access Review Responses, click on the Responses tab of a Reviewer record, or choose Related > Review Responses from a Request or Review Group. 

 

 

 

 

 

These report include:

Scholarship: this report contains the following data points -- Request #, Primary Contact, Review Group Application Score, High School, College, Major, GPA, Requested Scholarships, Reviewer, Question, Conflict of Interest, Response, Response Score, Weight.

 

Scholarship (Totals Only Report): this report contains the following data points -- Request #, Primary Contact, Review Group Application Score, High School, College, Major, GPA, Requested Scholarships, reviewer, Question, Conflict of Interest, Response, Weighted Response Score (= score * weight).

 

Scholarship with Weighted Score: this report contains the following data points -- Request #, Primary Contact, Review Group Application Score, High School, College, Major, GPA, Requested Scholarships.

 

Upon running a report, the report viewer will open in a new browser tab. 

 

You can run this report on multiple review response records by selecting some or all of the review responses in a Reviewer record then clicking the ellipses, then Run Report.