Running Assessment Without an Institutional Research Office

If one person is running assessment, the goal is not completeness, it is a defensible evidence base that can be produced every year without burning out. That means automating the repeatable parts, stopping work that does not change a decision, and collecting enough evidence to show patterns in student learning, program quality, and improvement over time.

Start with the decisions you must support

Before collecting anything, name the few decisions the institution actually makes with assessment results. For a small college, that is usually one of these: whether to revise a program, how to respond to accreditors, where to focus student support, or whether an internship, practicum, or capstone is meeting expectations.

If the data do not inform one of those decisions, they are optional. A one-person office cannot maintain a broad dashboard of every possible metric and still keep the work current. The most common mistake is building a system that looks comprehensive but is too fragile to sustain after one busy semester.

Automate the routine, not the interpretation

Automation should handle tasks that are repetitive, stable, and easy to validate. Good candidates include:

Enrollment, credit, and completion pulls from the student information system
Standard survey distribution and reminder emails
Basic file naming, timestamping, and storage
Simple aggregates, such as response rates by program or term
Routine exports for accreditation templates

What should not be automated without review is interpretation. A 78 percent satisfaction rate means something different if it came from 9 responses in one program than if it came from 240 responses across four cohorts. Automation can assemble the table, but a person still needs to decide whether the result is credible.

For a small college, the best time saver is usually not a complex dashboard. It is a template that produces the same report every term with the same fields, so you are not rebuilding the structure from scratch each cycle.

Ignore data that do not change a decision

A one-person assessment function has to be selective. Ignore data when any of the following are true:

You cannot act on the result within a year
The same question is already answered by another measure
The sample will be too small to interpret responsibly
The collection process requires manual work that is disproportionate to the value of the result
The result would be interesting, but not useful for planning or reporting

This is hard for institutions that feel pressure to show breadth. But breadth without use creates dead work. A small college does not need 40 indicators if 10 of them are the ones faculty review, deans use, and accreditors ask for.

A useful rule is to keep measures that are tied to a recurring action. If a metric is never discussed in a committee, never used in a program review, and never appears in a report to trustees or accreditors, it probably does not belong in the core set.

Build the minimum viable evidence base

The minimum viable evidence base is a small set of measures that can answer three questions:

Did students have the intended experience?
Did they demonstrate the intended learning or competency?
What changed because of the findings?

For many institutions, that can be done with four evidence streams:

1. Participation and completion data

Track who participated, in what, and whether they finished. In experiential learning, that might mean internship placements, capstone enrollment, or practicum completion. This is not enough to prove quality, but it tells you whether students had access and whether the program operated as planned.

2. Student self-assessment

A short self-assessment can show perceived growth over time, especially when it uses a stable rubric or competency framework. The limit is obvious, students are not neutral observers of their own performance. Still, self-assessment is useful when it is paired with other evidence rather than treated as proof by itself.

3. Supervisor or faculty ratings

External ratings are often the most credible evidence for applied learning because they come from the people who observed the work. Even a simple 4-point scale on a few clearly defined competencies can be more useful than a long narrative form that only one supervisor completes well.

4. A documented action log

Assessment is not just the finding, it is the response. Keep a brief log of changes made, who approved them, and what evidence prompted the change. This is the part that often gets lost when one person is doing the work. If an accreditors asks what happened after the data review, the action log should answer in one page.

Keep the rubric short and repeatable

Long rubrics create uneven scoring, especially when supervisors are busy. A better approach is a short rubric with clearly observable behaviors. For example, instead of asking whether a student demonstrates “professionalism,” ask whether the student arrives prepared, meets deadlines, communicates delays early, and follows workplace norms. Those behaviors can actually be rated.

The same logic applies to surveys. A six-item instrument completed consistently is usually better than a twenty-item form that produces thin data and low response rates.

The point is not to lower expectations. It is to reduce ambiguity so that the evidence you collect is comparable across terms and supervisors.

Use reporting that matches your capacity

If there is no institutional research office, reporting has to be simple enough to finish before the next cycle starts. A workable annual package usually includes:

One summary page of response rates and participation counts
One table of results by program or cohort
One short narrative on trends and limitations
One list of changes made or planned

That is often enough for internal use and far more realistic than a polished multi-tab report that only gets completed once.

If the institution needs accreditation-ready reporting, keep the same core data and format it into the required structure once. Do not build separate systems for each audience unless there is a clear reason. Duplicate reporting is where small offices lose time.

Be explicit about limits

Small institutions often have small samples, uneven participation, and missing data. Do not hide that. State the limits directly: number of respondents, response rate, number of supervisors, and whether results are based on self-report, observed performance, or both.

That honesty increases credibility. A report that says “12 internship supervisors rated student performance, response rate 86 percent, results are directional rather than statistically generalizable” is more trustworthy than a report that implies certainty it does not have.

The same is true for trend data. If the measure changed last year, or if the cohort size is tiny, say so. The minimum viable evidence base is not a claim of precision, it is a record of disciplined observation.

A practical example

A small liberal arts college with one assessment coordinator might do this each year for internships: pull enrollment and completion data from the registrar, send a short student self-assessment at the midpoint and end, collect a supervisor rating on five competencies, and record one improvement action from each program. That gives the institution access, growth, external observation, and follow-through without asking one person to manage a full analytics shop.

When institutions want a more structured way to gather student self-assessment plus supervisor and faculty feedback on competencies, a tool such as the Career Readiness Report can reduce the manual burden, but the reporting logic should still stay simple and limited to the measures the institution will actually use.

Keep the system small enough to survive next year

The right standard for a one-person assessment operation is not whether the system is ideal. It is whether it can be repeated by the same person next term, after committee meetings, accreditation work, and a full inbox.

If a measure is hard to collect, hard to explain, and hard to act on, drop it. If a measure is easy to repeat, tied to a decision, and understandable to faculty, keep it. That is the difference between having assessment files and having an evidence base.