The mission of the University of Illinois is to enhance the lives of the diverse people of Illinois, the nation, and the world through our leadership in discovery, learning, and economic development. Faculty public engagement is critical to the success of the University’s mission. When such engagement takes the form of research conducted with public partners (e.g., local, state, national, or international communities or organizations), evaluation of impact may require alternative approaches. To address this issue in the promotion and tenure process, faculty members at the University of Illinois Urbana- Champaign (UIUC) can select the Public Engagement Research Option (PERO; see II.C.8 of Provost Communication 9).
It is expected that publicly engaged research conducted by faculty members will generate scholarly and/or creative products (e.g., peer reviewed journal articles, books, and/or grants) whose impact can be captured by traditional metrics. Publicly engaged research may also generate products (e.g., policy reports, museum exhibits, and/or websites) whose impact may not be adequately captured by traditional metrics. PERO is suitable when a substantial proportion of a faculty’s record is comprised of such products. PERO allows traditional metrics to be supplemented by alternative metrics that provide insight into societal impact.
Alternative metrics are likely to evolve over time and may need to be identified by candidates and their Executive Officers (EOs). Thus, the conversation about PERO between faculty and their EO should begin as early as possible and continue over time so they can collaboratively develop and execute a plan for evaluation that will capture the intended impact of the research. The decision of whether to take PERO is up to each faculty member but should be made in collaboration with their EO and the PERO Advisory Committee (Please send an email to Office of the Provost email@example.com for additional information and/or to request a meeting with members of the PERO Advisory Committee). If additional input or guidance is needed (e.g., the EO is unsure and/or the candidate and EO disagree), the Dean’s office should be consulted.
Regardless of rank, the decision to select PERO must be made at least two calendar years prior to the candidate’s submission of their materials for promotion. Candidates selecting PERO may later decide, in collaboration with their EO, that the option is not appropriate for them (e.g., because most of their research can be evaluated with traditional metrics). Such a decision must be made prior to the deadline for the candidate’s submission of their list of external evaluators to their committee (see Section II.C.10 for guidance on External Evaluation of Research and Other Accomplishments in Provost Communication 9).
As highlighted in Provost Communication 9 (see section II.A.1), publicly engaged research draws on a faculty member’s expertise to define and address societal problems, concerns, issues, or interests to contribute to the public good. Such research occurs in collaboration with communities and/or organizations at the local, state, national, or international level. In its model form, publicly engaged research exists through a mutually beneficial exchange of knowledge and resources in a context of partnership and reciprocity. Central in the evaluation of publicly engaged research is gauging its societal impact.
External Evaluation Letters
For faculty members opting into PERO, two of the five required letters from external evaluators must be from experts outside academia who can objectively evaluate the societal impact of the candidate’s publicly engaged research (see section II.C.10 of Provost Communication 9). The other three letters must be from external evaluators at academic institutions; it is advised that at least one of these evaluations be from a scholar with expertise in publicly engaged research in the candidate’s field at an academic institution.
The wording in the PERO: Template Letter for External Evaluators in Academia and PERO: Template Letters for External Evaluators Outside of Academia (see attachments to Provost Communication 9) should be used in requesting the evaluations.
Candidates and their EOs should discuss what types of experts outside of academia have the relevant knowledge to evaluate the societal impact of the candidate’s research. Specifically, based on their dossier, experts outside of academia must be able to discern whether the candidate made an impact on society. Candidates may also find it useful to consult with others with more expertise, or the PERO Advisory Committee. Types of experts who may serve as external evaluators must be listed on the MOU. External evaluators outside of academia may include, but are not limited to, an expert in industry or government, a leading public figure, and someone in another community who holds a parallel position to a community partner. Regardless of the letter writer’s position, letters from external evaluators outside academia must be appropriate along two key dimensions:
- Be from highly qualified individuals with the expertise and/or perspective (e.g., as a consequence of their experience working in a relevant organization and/or as demonstrated by awards) to evaluate the societal impact of the candidate’s contributions; and
- Be from objective evaluators. Evaluators cannot have a conflict of interest in that they may directly benefit from the candidate’s promotion. For example, letters should not be solicited from a candidate’s collaborator in the community. Indirect benefits (e.g., if a faculty member’s innovative school discipline program is successful in one school district, it will be available to a neighboring school district) are not considered a conflict of interest.
In submitting their list of external evaluators, PERO candidates should include not only evaluators at academic institutions, but also evaluators outside of academia, with enough names to guarantee some degree of privacy to each type of evaluator. As is the case for all candidates, the majority of the five letters from external evaluators, regardless of whether they are from within or outside academia, must come from the unit’s list. In sum, for PERO candidates, there must be at least three external evaluators at academic institutions along with two evaluators from outside of academia. Whether evaluators in these two categories come from the candidate’s or unit’s list, is up to the unit.
Invitations to external evaluators outside of academia must be made using the template letter created for this purpose (see PERO: Template Letter for External Evaluators Outside of Academia in the attachments to Provost Communication 9). The template letter should be modified to reflect the type of publicly engaged research conducted by the candidate with a brief but concrete description of the type of dossier being shared with the evaluator. The procedures outlined in section II.C.10 of Provost Communication 9 for solicitation of external evaluators at academic institutions must be followed in that invitations must be neutral and indicate that the confidentiality of the evaluator’s remarks will be protected to the extent possible within the law. As illustrated in the template letter, the invitation letter must include guidance on the provision of concrete feedback so that the letter is informative to promotion and tenure committees. By using the template letter, the external evaluator should write a letter that is appropriate for the promotion and tenure process. The letter from the external evaluator should show how the candidate has made a societal impact.
As is the case for external evaluators at academic institutions, the unit must include the qualifications, including titles and current affiliation, of all external evaluators from outside academia in the promotion papers. The unit should also explain why each external evaluator from outside academia was chosen—that is, what qualifies them to evaluate the candidate—and report any direct relationship (e.g., collaborator) between the evaluator and candidate. When the contacted individual declines to serve as an evaluator, their name must be included with the list of external evaluators (see Outline of Promotion Dossier Section VII.B in Provost Communication 9), and the reason for declining the request should be provided, if one is given. A copy of the letter or letters of solicitation must be in the promotion dossier. (If the same letter was sent to several different individuals, only one of the letters of solicitation need be submitted.)
Preparing the PERO MOU
All decisions about PERO must be documented in a Memorandum of Understanding (MOU; see the PERO MOU template and samples in the attachments to Provost Communication 9). The MOU must be created using the PERO MOU template. The MOU is required to detail the products and metrics reflecting societal impact; it should be as detailed and specific as possible. Candidates must share their draft MOU with the PERO Advisory Committee for feedback. The MOU should be revised to be responsive to the feedback; the candidate is required to detail how they responded to the feedback with a clear and detailed rationale for any guidance they decided not to follow. The feedback summary provided by the Committee needs to be attached to the MOU. The candidate is expected to submit the final MOU in the required format to their EO and Dean for approval. The approved MOU, and any approved addendums, becomes part of the candidate’s dossier.
The PERO Advisory Committee will provide feedback to candidates and EOs about the appropriateness of PERO versus the traditional promotion and tenure process, the range of products and metrics used to evaluate societal impact, and the types of external evaluators. Regarding the MOU specifically, the PERO Advisory Committee must review and provide feedback about each MOU before it is signed by the candidate, EO, and Dean. The Committee will not vote, approve, or endorse a MOU nor will it evaluate the candidate’s research regarding its quantity or quality. The PERO Advisory Committee is designed to provide guidance to the candidate and EO; the candidate must address the committee’s feedback in their MOU.
Guidance for Faculty Members Deciding Whether to Take PERO
The decision about whether to select PERO involves consideration of a variety issues, often making it complex. The questions below are provided to help faculty take these issues into account in making the decision. In answering each question, faculty should consider their research program as a whole—if only a small portion of their research is publicly engaged and/or it is their teaching and/or service that is largely publicly engaged, PERO is not appropriate. In addition, if the impact of the large majority of the scholarly and/or creative products can be evaluated with traditional metrics, PERO is likely unnecessary. PERO may be a good fit for candidates answering in the affirmative to many of the questions below.
- Does the research generate significant new transferable public knowledge—that is, knowledge that can be used outside the specific setting in which it was produced?
- Can the knowledge be used by others outside of the specific context in which it was studied?
- Does the research significantly draw on your scholarly expertise?
- Does the research meaningfully address a community or societal problem, need, concern, issue, or interest?
- Are the discipline’s traditional review mechanisms insufficient to evaluate the impact of the work? Note. The impact of some of the work must be able to be captured by traditional metrics (see above).
- Does the work have a significant measurable direct impact on the public at the local, state, national, or international level? For example, did the work contribute to a new policy or legal decision, improve access to needed services or other resources, enhance how an organization works, or improve the mental health of those the work is designed to help?
- Is review of the work’s impact by stakeholders outside academia critical to the evaluation of its success? And is the work publicly available so that it can be reviewed by external stakeholders as part of the promotion and tenure process?
- Is the research a collaboration with community or other public partners—for example, the partner identified the need for the research or is involved in the development of the methods?
- Did the work take significant relationship-building with external partners to maximize its quality and impact?
- Does the success of the research depend on community buy-in?
- Can the work be sustained in the community? Is the work done in a way that the community can continue the program and/or policy developed by the work?
Checklist for PERO
Because PERO has three unique elements that are required for the promotion and tenure process, we provide a checklist for these elements so candidates and EOs can ensure they have followed all the instructions:
- PERO Memorandum of Understanding—must be reviewed by the PERO Advisory Committee. Once the MOU has been revised according to the Committee’s feedback, it must be signed by the candidate, the EO, and the Dean at least two calendar years prior to the candidate’s submission of their materials for promotion. Use the template PERO MOU found in the attachments to Provost Communication 9.
- Two letters from external evaluators outside of academia—use one of the PERO: Template Letters for External Evaluators Outside of Academia (see attachments to Provost Communication 9) to request these evaluations.
- Three letters from external evaluators in academia—use the PERO: Template Letter for External Evaluators in Academia (see attachments to Provost Communication 9) to request these evaluations.
Candidates and EOs should also ensure they discuss the following three issues, beginning with the creation of the PERO MOU or earlier:
- The types of non-traditional scholarly and/or creative products the candidate’s work will generate and the rationale for why these products are appropriate. The products are entities (e.g., documents or performances) with metrics that reflect societal impact; they cannot include steps in the public engagement process, such as indicators of collaboration.
- The types of metrics appropriate to evaluate the societal impact of these non- traditional products. The metrics must speak to the societal impact of the research with a rationale for why these metrics are meaningful.
- The types of experts outside of academia appropriate to evaluate the societal impact of the candidate’s work; this discussion should not focus on specific individuals but rather the type of expertise and experience that is appropriate. The PERO Advisory Committee may provide feedback about the types of experts listed in the MOU.
Frequently Asked Questions About PERO
If a candidate takes PERO, do they need to have scholarly publications and/or grants? Or can all their work be non-traditional with non-traditional metrics to evaluate it?
All promotion and tenure candidates need to generate scholarly and/or creative products considered traditional in their discipline. However, candidates conducting publicly engaged research may also have products considered nontraditional if such products are the conduits through which their work has societal impact. Thus, PERO candidates may have fewer products considered traditional in their discipline, but their work as a whole, is expected to demonstrate societal impact.
Is research in the context of entrepreneurship or work with private companies appropriate for PERO?
Only to the extent that research in these contexts meets the definition of public engagement as described in the PERO Guide (see also section II.A.1 of Provost Communication 9), would it be considered publicly engaged research.
How can candidates who do a substantial amount of publicly engaged service, but not publicly engaged research, be recognized for their work?
Publicly engaged service should be discussed in the optional service statement, which can be up to three pages of the dossier. It should also be listed in the service section of the dossier under “public engagement, outreach, and/or extension” (see section II.C.6.a and Candidate Activity Outline Form sections IV.A.1 and V.C of Provost Communication 9).
Is it appropriate for the EO or their designee to have additional conversations to explain or educate external evaluators outside academia about the promotion and tenure process?
Similar to communication with external evaluators at academic institutions, communication with external evaluators outside academia should largely be restricted to the formal letter requesting the review to ensure the process remains unbiased. Of course, there may be correspondence around technical issues such as deadlines, format, conflicts of interest, or confidentiality. If an external evaluator has questions about the review process that go beyond these kinds of technical issues, the evaluator should be referred to the next level of leadership (e.g., the College Dean’s office).
Can the letter to external evaluators at academic institutions explain PERO and the different metrics being used to evaluate the candidate’s work?
Section II.C.10 of Provost Communication 9 details the additional language that can be used in letters to external evaluators at academic institutions. A template letter including this language is also available (see PERO: Template Letter for External Evaluators in Academia in the attachments to Provost Communication 9).
Does the feedback from the PERO Advisory Committee need to be addressed in the MOU?
The MOU should include the feedback given (verbatim) from the PERO Advisory Committee. The MOU should also include whether the candidate used the feedback and, if not, a strong rationale for not using the feedback from the PERO Advisory Committee.
What are some examples of non-traditional products and related metrics?
Non-traditional products will vary depending on the candidate’s area of research. However, some examples of non-traditional products are white papers, testimonies to Congress, policy briefs, public reports to organizations, replications of an intervention program, museum exhibits, documentaries, editorials published in major media outlets, websites, community-wide exhibits of research findings, amicus briefs, action briefs, and public videos. This is an illustrative, rather than exhaustive, list.
The candidate must also identify the metric related to the product. The metric can help showcase the societal impact of the product. Metrics will be individualized to the candidate’s scholarship and the products yielded by it. However, some examples of metrics are citations of a white paper by Congress and legislation; references to a Congressional testimony by legislators and advocacy groups, integration of policy briefs in legislation, number of citations of public reports, number of people or organizations replicating an intervention program, attendance of museum exhibits, number of viewers of a documentary; hit rate for a website, attendance of community-wide exhibits or events, extent of changed practices due to action briefs, and number of views of public videos. Again, this list is meant to be illustrative rather than exhaustive
What are some non-examples of non-traditional products and related metrics?
There are many activities that reflect the process of partnering with the community which are critical to public engagement. However, these activities do not represent products or metrics of societal impact and thus cannot be used for PERO. Such activities include, but are not limited to, MOUs with community partners and evaluations of the candidate’s partnership with a community agency. Products that cannot be made publicly available due to proprietary reasons and/or non-disclosure agreements also cannot be used as part of the evaluation in PERO.