# Assessment FAQs

Starting in Fall 2018, each time a Pathways course is taught, instructors will collect and report assessment data. Instructors are responsible for completing the first four steps of the Pathways assessment process.

- Instructors teaching a Pathways course should first consult the official course proposal to determine which core and integrative concepts and student learning outcomes are addressed in the course.
*Instructors are responsible for assessing all of the Pathways concepts and student learning outcomes addressed in the official course proposal.* - Instructors should identify a piece of student work (i.e. a direct measure of student learning) to assess each of the Pathways student learning outcomes. One of the key features of Pathways assessment is that each student learning outcome needs to be assessed independently.
- Instructors should use the respective Pathways Rubric(s) to interpret student work and determine the extent to which a student has gained the knowledge, skill, ability, or competency articulated in each student learning outcome.
- Instructors will use the Pathways Assessment Reporting Form to report the number of students who are
*below competent*,*competent*, and*above competent*for each student learning outcome to the Institutional Effectiveness unit in the Office of Analytics and Institutional Effectiveness.

Assessment data should be reported for *each section* of a Pathways course every fall semester, spring semester, or summer term that it is taught. If a course has 10 different sections, assessment data will need to be submitted for all 10 sections. Assessment data should be submitted for every section regardless of the classification of the instructor (i.e., data will need to be reported for sections taught by tenured faculty, adjunct faculty, graduate teaching assistants, etc.). If an instructor is teaching multiple sections of a course, data from each of the instructor’s sections should be reported.

If a lab course is separate from a lecture course with a different course number (e.g., CHEM 1035 [lecture] and CHEM 1045 [lab]) and has been approved for Pathways credit separately, the lab course should be assessed independently from the lecture course.

This decision is currently up to individual departments. Each section of a course may use the same assessment measures if desired, but this is not required.

This decision is currently up to individual departments. Some departments may opt for each instructor teaching a Pathways course to submit a separate report form, while other departments may choose to combine data from multiple sections of a course and submit them in one report form. When data from multiple sections are aggregated in one report form, all of the sections that contributed data should be listed.

For courses that have been approved as part of a Pathways *grouping*, assessment data will only need to be submitted for the final course in the grouping. For Pathways courses that are part of a *sequence*, data from each course in the sequence will need to be reported each time the course is taught.

Instructors should use direct measures of student learning (i.e., student work) to assess Pathways concepts and student learning outcomes. Direct measures allow for the assessment of student achievement related to specific knowledge, skills, abilities, or competencies. Direct measures of student learning include:

• Written exams or quizzes

• Oral exams or quizzes

• Multiple-choice questions

• Essays

• Capstone projects

• Observations of performances

• Oral presentations/debates

• Research papers or projects

• Artwork/exhibits

• Poster sessions

• Case studies

• End-of-course papers or projects

• Audio recordings

• Portfolios

• Films

• Lab reports

• Observations of teaching

• Online asynchronous discussions

Although the type of student work assessed is up to the instructor, the work should be a required course activity that all students complete. It is recommended that instructors select student work from later points in the semester so that students have more opportunities to obtain the knowledge, skills, abilities, or competencies outlined in the Pathways student learning outcomes.

If appropriate, a single piece of student work can be used to assess multiple Pathways concepts/student learning outcomes as long as different components or aspects of the assignment are used to measure the different student learning outcomes. If an instructor uses a quiz or exam to assess multiple student learning outcomes, each student learning outcome should be assessed with a unique set of quiz/exam questions that aligns with the student learning outcome.

For Pathways instructors who are interested in using group work as a direct measure of student learning, instructors are strongly encouraged to assess each student in the team individually rather than giving one rating for the entire group since high-performing students can mask the poor performance of other team members.

Instructors are not required to collect or report indirect evidence of student learning such as student surveys, course evaluations, or focus groups.

Pathways instructors should only report data for work that is received. If student work is not submitted, it should not be rated. For example, if 30 students are enrolled in a course but only 27 students submit the assignment (or component of the assignment) that the instructor is using to assess a specific Pathways student learning outcome, then the instructor would only report results for the 27 students from which the instructor received work.

Instructors should use the Pathways Assessment Reporting Form to report assessment data from their Pathways course(s). When complete, please submit your Pathways Assessment Reporting Form using the Pathways Assessment Data Collection Google Form. As an alternative, the reporting form may be submitted to Molly Hall in the Institutional Effectiveness unit in the Office of Analytics and Institutional Effectiveness.

Instructors may submit assessment data from their Pathways courses whenever data collection for the semester is complete. Deadlines for submitting data are as follows:

- Fall semester: January 31
^{st} - Spring semester: May 31
^{st} - Summer session I: July 31
^{st} - Summer session II: August 31
^{st} - Winter Session: Instructors are not currently required to submit Pathways assessment data for Winter Session courses

#### 11. Do Pathways instructors need to submit samples of student work along with their assessment data?

While all Pathways instructors are invited to submit samples of student work along with their assessment data, submitting student work is not required at this time. Instructors who choose to participate in working sessions to discuss the assessment data collected and how the Pathways curriculum may be improved may volunteer to provide samples of student work for these working group sessions.

Assessment data will be used to evaluate and improve the Pathways curriculum, not evaluate individual instructors or courses. Submitted data will be aggregated at the student learning outcome level, ensuring that the data are not student, instructor, or course identifiable. At the end of each academic year, the Institutional Effectiveness unit in the Office of Analytics and Institutional Effectiveness will create a summary of assessment data for all Pathways student learning outcomes and concepts that will be disseminated by the Office of General Education.

For Pathways courses with more than 20 students enrolled, instructors may choose to report data for a *minimum* sample of 20 randomly selected students. For courses with multiple sections, a minimum of 20 randomly selected students should be assessed ** per course section.* **All students in the course should complete the same assignments, but only the work of the sampled students would need to be reviewed by the instructor using the Pathways Rubrics and reported via the Pathways Assessment Reporting Form.

Although the Pathways implementation plan calls for a ** minimum **sample of 20 students, the larger the sample size, the more reliable and valid any conclusions from the data will be. Larger sample sizes are associated with a smaller margin of error. Instructors who would like to have a 10% margin of error with a 95% confidence level will need to collect data from the following number of students per course:

Population/Class Size |
Sample Size |

20 students or less | All students |

25 students | 21 |

30 students | 24 |

40 students | 29 |

50 students | 34 |

75 students | 43 |

100 students | 50 |

150 students | 59 |

200 students | 66 |

250 students | 70 |

300 students | 73 |

350 students | 76 |

400 students | 78 |

500 students | 81 |

Instructors are encouraged to submit data for as many students as possible. For instructors utilizing Opscan forms or Canvas for exams/quizzes, it may be easier to submit data for all students in a course than to report data for a sample of students.

*** Alternative data collection and reporting option for large, multi-section courses that use the exact same Pathways assessment measures across sections**

For large-enrollment courses that have multiple sections AND use the exact same Pathways assessment measures in each section, departments may instead choose to collect Pathways assessment data from a larger random sample of students pulled from the entire population of students enrolled in the course (i.e., across the multiple course sections). Departments choosing this option should select a sample large enough to obtain a 10% margin of error or less with a 95% confidence level. Below are sample size guidelines for courses that would like to utilize this option.

Population/Class Size |
Sample Size with a 10% Margin of Error |
Sample Size with a 5% Margin of Error |

600 students | 83 | 235 |

800 students | 86 | 260 |

1000 students | 88 | 278 |

1200 students | 89 | 292 |

1400 students | 90 | 302 |

There are multiple options for creating a random sample of students in a Pathways course. Two options described in more detail here are (1) a random number generator, and (2) Excel.

**Using a Random Number Generator:**

Random number generators are available on the internet (e.g., http://stattrek.com/statistics/random-number-generator.aspx) or as smartphone applications (e.g., Random Number Generator Plus). These programs allow instructors to enter the number of random numbers they need to create a sample (e.g., 50 random numbers for a sample of 50 students), as well as minimum/maximum values and the option to eliminate duplicate entries.

For example, an instructor would like 73 random numbers to be generated to create a sample of 73 students from a class with 300 students. The minimum value should be set at 1 to represent the first student on the class roster and the maximum value set at 300 to represent the last student on the class roster. Duplicate entries should not be allowed in order to prevent numbers/students from being selected more than once. The program then generates a list of random numbers (e.g., 2, 4, 9, 10, 19, 23, 26, etc.). The instructor's sample would be comprised of the students from the class roster that correspond with the random numbers generated (e.g., student #2, student #4, student #9, student #10, student #19, student #23, student #26, etc.).

**Using Excel:**

To create a random sample of students using Excel, follow the steps below. This example is for an instructor who has 500 students in a course and would like to generate a random sample of 81 students.

- Within the Excel spreadsheet that includes your class roster, add a new column and name it Random_number
- In the first cell underneath the heading row, type =RAND()
- Press "Enter" and a random number will appear in the cell

- This number will be a decimal between 0 and 1 - Copy and paste that cell into the cells below in the Random_number column so that each row contains a random number. Note: You may notice that numbers within previous cells change during this process. Step 5 will lock the random numbers in place.
- Select the entire Random_number column

a. Click COPY

b. Click PASTE VALUES

c. This step replaces the random number formula with the generated value - Use the SORT option to custom sort the records by the Random_number column

- Select “Smallest to Largest” order and click OK - Once sorted, choose the first 81 students listed. These students would comprise a random sample of 81 out of 500 students.

(Adapted from: https://www.surveymonkey.com/mp/random-sample-in-excel/. A three-minute tutorial entitled “How to Create a Random Sample in Excel” is available on YouTube here: https://www.youtube.com/watch?v=q8fU001P2lI.)

Two options that instructors may consider using are (1) replacing the students who withdraw, do not submit assignments, etc. in the sample, or (2) select a larger sample of students initially to account for attrition. Instructors are encouraged to do the best they can to submit data for a minimum sample of 20 students.

Pathways data will be used for multiple purposes. Most importantly, Pathways assessment data will be used to inform and improve the Pathways curriculum to enhance student learning at Virginia Tech.

Pathways data will also be used to fulfill requirements for SACSCOC, Virginia Tech’s regional accreditor, and SCHEV, the State Council of Higher Education for Virginia. SACSCOC requires each accredited college/university to create and measure student learning outcomes associated with its general education curriculum and demonstrate that it is using these findings to seek improvement. SCHEV has developed a new policy on Student Learning Assessment and Quality in Undergraduate Education that requires each college/university in Virginia to report assessment data related to six competency areas, including Critical Thinking, Written Communication, and Quantitative Reasoning. Data from selected Pathways concepts and student learning outcomes will be reported to SCHEV to fulfill this requirement.

No. Pathways assessment is part of the teaching and learning process at Virginia Tech and instructors do not need to obtain informed consent from students to assess their work for Pathways. Since Pathways assessment data are used for institutional improvement – not to contribute to generalizable knowledge – IRB approval is not required.

Pathways assessment data will be shared with SACSCOC, Virginia Tech’s regional accreditor, and the State Council of Higher Education for Virginia (SCHEV). However, “releasing data to accrediting agencies in order to present evidence of improvement of student learning does not constitute dissemination of research results/data, and therefore does not require IRB approval.”^{1}

^{1} Virginia Tech’s Institutional Review Board Website