GOAPPLY REVIEWER ADMIN GUIDE

 

INTRODUCTION 

 

GOapply applications can be reviewed by both internal and external reviewers using the GOapply review features. These features allow the user to create and manage specific review groups for particular phases of their submitted applications. In addition, GOapply Users can be made reviewers and can be added to as many review groups, as necessary.  

 

The GOapply review features are divided into several components, which can be found on the left-hand navigation pane in the GOapply workspace.

 

 

 

Tip: We offer a Suggested Learning Course for GOapply Reviewer, which includes several instructional videos. GOapply Reviewer Course

OVERVIEW OF GOAPPLY REVIEWER COMPONENTS 

 

GOapply Reviewer is made up of many of the same entities as GOapply. Users, Applications, and Status Tracking are all employed in Reviewer in addition to several Reviewer-specific entities. These include Reviewers, Review Groups, Review Group Applications, and Reviewer Request Score. Below is a high-level overview of Reviewer entities; we will dive deeper into these topics later in this document. 

 

Your reviewers will register for a GOapply account independently. This process is the same as applicant registration, which we will walk through step-by-step later in this document.  

 

From the Application, you can see which Review Groups it is attached to.  

 

Status Tracking records are what informs Reviewer which phases/applications are ready for review.  

 

Reviewers are all GOapply Users that have the Reviewer or Applicant and Reviewer role assigned. 

 

Reviewers will not have access to any applications to review until they have been added to a review group.  

 

Review Group Applications use the GOapply Status Tracking record to display the application for the reviewer. If the GOapply Status Tracking record does not exist or does not contain a PDF application, the application will not display correctly in the reviewer dashboard.  

 

GOapply Admins can upload an alternative file to be displayed for the reviewer. In the Status Tracking Record, there is an upload field called Reviewer File. Here is where you would upload what you want displayed to the reviewer if it is different from the submitted PDF. This file will override the submitted PDF.

 

 

Reviewer Request Score records allow you to see the score that a reviewer gave. These records also contain the Reviewer Form Response json file. This json file is a string of code; it is similar to the Form Response in GOapply Status Tracking records.  

 

In all areas of akoyaGO CRM, you have a system dashboard called GOapply Application Review that tracks applications that need to be assigned to a review group, applications with a conflict of interest, applications with scores from review groups, and active reviewer request scores. 

 

GETTING STARTED 

 

Now that you know how the GOapply Reviewer entities work together, let’s get started creating Review Groups and adding Reviewers. 

 

Each review group consists of general info (including the application and phase), settings to determine what reviewers can see, the reviewers themselves, and review group applications. All of these components are necessary for reviewers to have everything they need.  

 

The following steps detail the process to create a completed review group.  

 

1. From the Review Group entity, click “New Review Group.”  

 

The New Review Group form will load.  

 

2. Name your review group.  

 

3. Select the application and phase that the review group will be assigned to.  

 

4. Set a review group deadline in the Review By field. Once the Review By date has passed, the review group will no longer appear in the reviewer’s list in GOapply. 

 

5. Lead Reviewer Recommends: This settings allows your lead reviewer to recommend which application gets awarded and if awarded, the amount awarded. This feature will ONLY apply to the reviewer where Lead Reviewer = Yes. Choose from the drop down

• No (or null): This feature is turned off 

• Recommend to Request: If chosen, this setting will make it so the lead reviewer's choices will map to the associated Request record. When this option is chosen, an additional question will be added to your reviewer form called Recommended Award Status.

The lead reviewer can choose Award or Deny. If Award, an additional Recommended Amount field will be added where they will input the amount they recommend to award. The amount field will be required.

 

Once submitted, their choices will be mapped to the Request. A field called Review Recommender will populate with the GOapply User (Process tab > Decision section).

 

 

The Recommended Grant amount will also fill in with the amount they recommended. Their choices will also map to the Review Group Application. 

 

• Recommend to Review Group: If selected, this option will also add the Recommended Award Status (and Recommended Amount field if Award) questions to the review form. This option will also map the lead reviewers choices to the Review Group Application. This option will NOT fill in any field on the request. 

 

6. Click Save. Upon save, the Review Form Definition field will populate with its unique string of code and lock to prevent accidental deletion. To download the Review Form Definition, click the link in the field. 

 

Click the Settings tab of the newly created Review Group. 

 

 

The Settings tab contains various customization options for each review group. These settings involve the information you wish to share with the reviewers. All but Anonymous Review will default to Yes. Anonymous Review will default to No. Scoring Type will default to Sum (total score).

 

 

Show Other Reviewers Comments: If Yes, the Reviewer Comments button will be visible on the application review form. If set to No, the Reviewer Comments button will not be visible on the application review form. Please note that if Show Other Reviewers Comments is set to No, the Show Reviewer Weighted Score setting is rendered null, and the Public setting in Form Builder (page 10 of this guide) will also be null.  

 

If Show Other Reviewers Comments is Yes:  

 

 

If No:  

 

 

Show Mark for Conflict of Interest: If set to Yes, the Conflict of Interest checkbox will appear on the Reviewer’s review form. If set to No, the Conflict of Interest checkbox will not be visible on that form. 

 

 

 

Anonymous Review: Select Yes if you want applicants to appear anonymous. If Yes is selected, the applicant’s name will be removed from the PDF that the reviewer sees. The reviewer PDF will be stored in a field called Reviewer File in the GOapply Status Tracking record. 

 

Show Reviewer Weighted Score: If Yes, reviewers will be able to see how other reviewers scored the application. It renders as the far right column in the Reviewer Comments. If No, the weighted score column is not visible.  

 

 

Scoring Type: Choose Sum, Average or Sum by Reviewer then Average by Application.  

 

 

 

Two flows exist to duplicate a review group. To run the flow, select the review group, click Flows, and choose either Duplicate Review Group or Duplicate Review Group and Reviewers.

 

 

Duplicate Review Group: Duplicates application, phase, review by date and time, review form definition, review form and all properties, all settings, show staff comments and attachments setting, and staff notes 

 

Duplicate Review Group and Reviewers: Duplicates application, phase, review by date and time, settings, show staff comments and attachments setting, staff notes, reviewers 

 

Please note the staff notes and staff attachments are NOT duplicated with these flows. 

 

 

Reviewers are the GOapply users that will be evaluating the applications in their review groups. Reviewers must be added to each review group they are reviewing applications for. In order to be added to a Review Group, the user(s) must have the Reviewer or Applicant and Reviewer role.

 

Add in Bulk:

 

From a GOapply Users view, select the users you wish to add to the Review Group. Then click the Add to Review Group command bar button. 

 

 

Next, you will see a popup box called Add to Review Group. Select a review group from the dropdown list provided and click Submit. Please note, only active review groups will appear in the list.

 

 

Once the users are added to the Review Group, you will see a success popup.

 

 

*Please note that if you run this process on a user whose Role does NOT include Reviewer, they will be switched to the Applicant and Reviewer role.

 

Add Individually:

 

From the Reviewers tab of the review group, click New Reviewer in the All Reviewers subgrid. 

 

The New Reviewer form will populate with the review group information. Review Complete and Lead Reviewer will default to No. 

 

 

Select a GOapply User in the Reviewer field. If the user is the lead reviewer, make that designation in their reviewer record. Lead Reviewers will be able to see all reviewers' comments regardless of the Show Reviewers Comments setting in the review group settings. Please note that the intended functionality is to have one lead reviewer per review group. If you try to set another reviewer as lead reviewer, it will override the previous lead reviewer. 

 

Review Complete: this field will automatically flip to Yes when the Reviewer has completed reviewing all the applications in all the review groups that they are assigned to. 

 

The Reviewers tab of the Review Group record are three subgrids: All Reviewers, In Progress, and Complete.

 

 

 

 

 

When Review Complete flips to Yes, the foundation staff person will receive an email alerting them to the Reviewer who completed their reviews and the Review Group to which they belong. The Reviewer will also receive an email thanking them for finishing their review. This email is determined by an Email Template called 'GOapply Reviewer Completed Email'. 

 

 

 

Rather than adding reviewers one by one, GOapply Administrators also have the option to bulk import reviewers.

 

Step 1: Create an Excel document and save it as a .csv. 

 

CRM import relies on an Excel spreadsheet where columns = fields and rows = individual records. For bulk importing reviewers, your spreadsheet should contain, at minimum, columns for the First Name, Last Name, and Email. Best practice is to include as many columns as you can define. Potentially, each field on the GOapply User form can be a column. Below is an example of a reviewer import file.

 

 

Step 2: In the GOapply workspace, navigate to the Reviewer Bulk Import entity.

 

 

 

Step 3: Select Import from Excel > Import CSV

 

From the toolbar at the top of the screen, select Import from Excel (you may need to click the elipses depending on your screen width), then click the carrot to select Import from CSV.

 

 

Step 4: Choose File and follow the wizard. 

 

An Import wizard will pop out from the right side of the screen. Choose your .csv file and click Next.

 

 

Make sure ‘First row contains column headings’ is checked and click Review Mapping. Allow Duplicates should be set to No.

 

 

If your mapping looks correct, select Finish Import. You will see a message that says, “Your data has been submitted for import.” You can close the wizard.

 

Step 5: Run the GOapply – bulk import users workflow

 

Depending on the size of your import, it may take a few minutes, but after you refresh the page, you should see all the records you imported in the Bulk Reviewer Import view. Once you see all your records, select all using the check mark at the top of the screen and click Flow in the toolbar at the top of the screen and choose GOapply – bulk import users. This workflow will create the records as GOapply User records.

 

 

 

When the workflow is finished running, all records should disappear from the Reviewer Bulk Import table and move into the GOapply Users table.

 

Step 6: Reviewers get an email and set up their account.

 

When the workflow runs, it includes a step to send an email to each reviewer created. Depending on whether the user is brand new or already had a GOapply account, they will see an email alerting them to the fact that they have been added as a reviewer. If they are brand new to GOapply, they will also be asked to create a password.

 

The emails are using the email templates GOapply – Invite New User or GOapply – User Already Exists. The workflow will check and apply the proper template. If you wish to edit the templates, you can do so from the Tools workspace.  

 

 

Step 7: Add reviewers to review groups.

 

Use the steps described above to add your new reviewers to their review group(s).

 

Enter Users Manually

 

You can also enter users into this table manually or on a one-off basis. Click the +New command bar button.

 

 

Fill in the form. 

 

 

First Name, Last Name: User name

Email: User email. This is the email to which the invitation to the portal will go. 

Applicant Type: Choose Individual or Organization. If Organization, a new required field called Parent Constituent will appear. Choose the constituent with which the user belongs. 

User Permissions: Choose from the list. Contributor can fill in phases but not submit, Submitter can fill in and submit phases.

Role: Choose Administrator (typically foundation staff), Applicant, Applicant and Reviewer, or Reviewer. 

Review Group: Optional. If this user should belong to a Review Group, select it here.

 

 

 

The Staff Attachments and Comments tab in the Review Group record allows for staff notes that apply to ALL applications for that review group. Attachments can also be added here as public documents for the review group.  

 

These notes and attachments are displayed above the review group applications. 

 

 

REVIEW FORMS 

 

Review forms are displayed for the reviewer side by side with the application. Question types include a text field, a radio button, a drop down, a long text comment, and a rich text editor. For example, below you’ll see the submitted application (or whatever is in the Reviewer file tab of the Status Tracking record) on the left, and the reviewer’s questionnaire on the right. At the bottom of the page they can save a draft or submit.

 

 

 

From the review group, navigate to the Form Builder tab. Select Launch Form Editor.

 

 

The Form Editor will load in a separate tab.  

 

 

Unlike the form editor in GOapply, the form editor in Reviewer is only able to support one page.

 

Please ensure you design your form to account for this. You will be familiar with this Form Editor; it is the same one you use to create your GOapply Applications. On the right-hand side, you will see the 6 question types. Drag and drop the question types from the toolbox into the form to add.  

 

Keep in mind that if the Lead Reviewer Recommends setting in the Review Group settings = Recommend to Request or Recommend to Review Group, potentially two additional questions will appear on your form.

 

 

There are five types of question: a text field, a radio button, a drop down, a long text comment, and a rich text editor. Please note that only Radiogroup and Drop-Down field types can have score values assigned. 

 

Single Input: a text or number response field. These fields cannot be used to calculate scores.  

 

Radiogroup: a single selection radio button style multiple choice. These responses can have score values assigned.  

 

Drop Down: a drop-down list of responses. These responses can have score values assigned.  

 

Comment: a long text or number response field. These fields cannot be used to calculate scores.  

 

Editor: a long text or number response field, with rich text formatting. These fields cannot be used to calculate scores. 

 

File: This is a file upload field. Add this field if you want reviewers to be able to upload files. Supported file types include 'txt', 'rtf', 'doc', 'docx', 'xls', 'xlsx', 'png', 'jpg', 'jpeg', 'gif', 'csv', 'ppt', 'pptx', 'pdf', 'mp4', and 'heic'. Max size for all uploads total is 20MB. When a reviewer uploads a file with their review form, it will be stored in the Review Response record for that question. For example:

 

 

If the question type is a type used to calculate reviewer scores, the scoring must be included in the choices for that question.  

 

 

Each question field has a properties menu. Open properties by clicking on the question and selecting Properties from the far right.

 

 

Common settings for each criteria include:  

 

Required: Required setting can be found in the General dropdown in the form of a toggle called “Is Required?” If on, these criteria must be completed for the reviewer to mark the review as “complete.” Required criteria will be denoted with an asterisk in the review form. Required will default to no.  

 

Weight: Weight can be found under akoyaGO. This functionality is only available if the Scoring Type in the Review Group’s setting is set to Average. If the criteria are averaged, enter the weight of the score. Note: if averaging is used, the weights of all criteria must total 100.00.  

 

Public: This setting can be found in the akoyaGO dropdown. If checked, the criteria responses will be shared with the review group in the Reviewer Comments tab. Note: the review group setting Show Other Reviewers Comments must also be turned on for this to appear. Lead Reviewer will be able to see all reviewer responses. 

 

 

Scoring can be set on the dropdown or radiogroup question types only.

 

1. In the Properties for the scoring field, drop down Choices 

 

 

2. For Value, enter the option, then two colons, then the numerical value of that choice. The proper format for this formula is no spaces. For example, Agree::80. In the Text field, simply enter the option that you want the reviewer to see. In this example, you’d just put Agree. Repeat this process for each option in the drop down.

 

If your Review Group settings designate the scoring type as Average, you will see a property called Weight under the akoyaGO section. Here is where you will set the weight of each scored question. Weight is only available for drop down and radiogroup question types.

 

 

 

Now that you have created your review group, designated reviewers, and created the review form, it is time to add the submissions for the reviewers to review.  

 

This is done through the entity called Review Group Applications. In that entity is a view called “GOapply Reviewer”. Through this view, you can manipulate the applications that a reviewer sees. For example, deactivating a record in that view would remove it from the list of applications that the reviewer sees. 

 

 

Next, you’ll assign the submitted applications to the review group.

 

Review Group Applications is an entity in akoyaGO CRM and uses Status Tracking records to determine which submissions get shown to the reviewers.

 

Bulk add submissions from Requests entity:

 

You will only want to add applications this way if you have more than 1 review group reviewing the set of applications. This will allow you to select a set of Request from a view and add them to a review group.

 

Step 1: Copy the name of the Review Group because you will need to enter it correctly in a workflow dialog.

 

Step 2: You can go to an individual Request record or a view to choose multiple to add

 

Step 3: Go to Flow > GOapply Add Request to Review Group

 

 

Step 4: Click Continue in the first box that pops up

 

Step 5: In the next window paste in the name of the Review Group and click Continue.

 

 

Manual add:

 

1. Starting from the submitted Status Tracking record, click on the Related tab, and choose Review Group Applications. 

 

 

2. Click +New Review Group Applications 

 

 

3. The New Review Group Application form will open and autofill the Request and Status Tracking field. All you need to do is assign the Review Group. The Recommended Amount and Recommended Award Status fields will only populate if the Lead Reviewer Recommends setting in the Review Group = Recommend to Request or Recommend to Review Group. If this setting is not being utilized for the review group, those fields will not auto populate. Save.

 

 

 

Now, the submission(s) you just added will appear in the review group’s dashboard in GOapply. Repeat this process for each submission that the review group needs to review.  

 

REVIEWER RESPONSES 

 

Reviewer responses are recorded in the Review Responses entity, Reviewer Request Score entity, in the Reviewer Scores tab of the Review Group Application entity, and in the Responses tab of the Reviewer record.  

 

 

Review Responses is an entity that stores every response from each reviewer. For example, if your reviewer form has five questions, every time a reviewer submits the form, five reviewer response records will be created.

 

Inside a Review Response record, you will see:

 

 

• Question Type: the type of question from the Form Builder toolbox, i.e., radiogroup, dropdown, text, file.

 

 

 

 

 

 

 

 

 

REVIEWER REQUEST SCORE 

 

Reviewer Request Score records contain: 

 

 

 

 

 

REVIEWER RECORD         

 

Reviewer responses and scores can also be seen from within the Reviewer record. 

 

 

REVIEWER REPORT 

 

In the Review Groups entity, you have a report called GOapply Review Responses that you can run to see each reviewer’s responses for each application assigned to their group.