Federal- and state-mandated criteria for accountability and assessment of student competence led to an 'assessment movement' in higher education during the 1980s.|1~ Legislative mandates emerged in response to critiques of student and institutional performance, coupled with declining revenue sources for public education.|1-3~ As a result of the reform movement in higher education, more than 80% of American colleges and universities implemented assessment measures.|4~ The National Association of State Universities and Land Grant Colleges suggests the focus of institutional program and student outcomes assessment be directed toward increasing effectiveness of academic programs and improving student learning and performance.|5~ The federal focus on assessment prompted all six of higher education's regional accrediting bodies to revise procedures and place more emphasis on assessment as a form of institutional accountability.|6~ Since accreditation is now linked directly to assessment, all institutions of higher education will be required to engage in student assessment activities.
ASSESSMENT IN HIGHER EDUCATION
Numerous methods and techniques have been identified as appropriate approaches in the assessment process.|7~ Locally developed achievement measures in the form of comprehensive examinations, recording of student performances, and simulation exercises have been used successfully as assessment strategies. These activities can be administered on a before-and-after basis to allow a more comprehensive view of student progress.|8~
Other ways of collecting assessment data that may reflect the quality of student preparation include surveying external groups such as alumni, employers of graduates, internship preceptors, and educational institutions that former students might later attend.|1,9~ Advisory councils comprised of individuals from these groups can provide expertise in developing assessment instruments and surveys.|10~
Self-report data collection in the forms of surveys, inventories, interviews, panels, and journals allow students to assess their own progress in the program. Self-assessment can prove beneficial to students by allowing them to be more self-guided, as well as providing significant information concerning the program and curriculum.|11~
Program quality can be assessed through pass rates on professional licensing examinations such as the National Athletic Trainers Association Certification Exam and the National Teacher Exams.|7~ Use of nationally standardized examinations such as the Collegiate Assessment of Academic Proficiency (ACT/CAAP) as an assessment method probably is more common to institutions than most other assessment tools.|12~
Collecting samples of student course work in the form of portfolios has been used as an assessment tool. Examples of portfolio components include examinations, research or term papers, academic writings, video tapes of student performances, and course projects or assignments. Faculty members as well as employers, intern preceptors, and advisory council members review and analyze portfolios to assess student progress toward institutional or departmental goals.|7,13,14~
ASSESSMENT IN HEALTH EDUCATION
The Role Delineation Project Curriculum Framework, developed by the National Task Force on the Preparation and Practice of Health Educators,|15~ assists in standardizing educational goals and objectives for preparation of entry-level health educators. The Framework has been used to assess and revise graduate and undergraduate curricula at a number of universities.|16~
Using a self-assessment tool based on the seven responsibilities for entry-level health educators, Schmidt and Beall|17~ compared perceptions of students in a professional school and community health preparation program prior to and following their field experience. Hayden|18~ used a self-assessment instrument to survey senior health education majors' perceived competencies as they prepared to take the certification examination. At East Carolina University, the Framework assessed the most important responsibilities and areas of perceived graduate success and deficiency as evaluated by practicing alumni of the School and Community Health Education program, agency employers, and field work student preceptors.|19~ Knight|19~ reports that the role delineation competencies, when adapted to the needs of various constituencies and when responsive to institutional mission, are useful supports in program assessment and curricula planning.
ASSESSMENT AT EASTERN ILLINOIS UNIVERSITY
The Dept. of Health Studies at Eastern Illinois University is developing a comprehensive plan for assessing the professional preparation program. A variety of assessment methods have been used and additional methods are being pilot-tested as part of the department's comprehensive assessment plan. The assessment strategies include: focus group interviews with graduating seniors, Departmental Advisory Council review of health education competencies, assessment of the competencies of undergraduate student interns by the internship preceptors, exit interviews with graduating seniors, alumni surveys, and portfolio assessment.
The comprehensive pilot assessment plan will determine if the professional preparation program adequately addresses responsibilities and competencies required of entry-level health educators. Meaningful curriculum evaluation and reform can be implemented based on assessment results.
Focus Group Interview
One of the first assessment strategies implemented involved a focus group interview. Participants were identified from students expected to graduate fall semester 1991 and spring semester 1992. Of 42 students, 24 randomly selected graduating seniors were invited to attend the focus group interviews, and nine participated in the interviews.
A self-assessment survey instrument and a group consensus survey instrument were developed from statements listed in Responsibilities and Competencies for Entry-Level Health Educators.|15~ The seven areas of responsibility in the Framework are: Responsibility I -- assessing individual and community needs for health education, Responsibility II -- planning effective health education programs, Responsibility III -- implementing health education programs, Responsibility IV -- evaluating effectiveness of health education programs, Responsibility V -- coordinating providing of health education services, Responsibility VI -- acting as a resource person in health education, and Responsibility VII -- communicating health and health education needs, concerns, and resources.
Ratings for these instruments ranged from a high of very well able to perform, to a low of 1, poor ability to perform. These instruments provided the basis for dialogue in the focus groups. During the interview, students completed the self-assessment survey instrument regarding their personal perceived competence for each of the seven responsibilities. After completing the individual survey, students collaborated on the group consensus survey instrument and recorded their perceptions on degree of exposure to the responsibilities and degree of competence in the responsibilities.
Mean scores were calculated for both the self-assessment responses and the group consensus responses. Responsibility I produced the highest score of 4.18, while responses to Responsibility V represented the lowest at 3.15 on the self-assessment instrument. Mean scores of the group consensus for perception of the 'degree of exposure' to the seven responsibilities ranged from a high of 4.25 for Responsibility I to a low of 2.60 for Responsibility V. Group consensus responses for' degree of competence' for each of the responsibilities ranged from a mean score of 4.25 for Responsibility I to a low of 2.75 for Responsibility V.
Department Advisory Council
The Advisory Council for the Health Studies Dept. includes 16 individuals from the community employed in fields related to health education. At a regular meeting of the Advisory Council in fall 1991, council members completed the Assessment of Health Education Competencies Questionnaire.|20~ The questionnaire, based on the Role Delineation Project Curriculum Framework, assesses perceived importance of each competency and sub-competency for all health educators and the use of each competency and sub-competency by individuals in their particular position. Strong agreement emerged that the competencies included in Responsibilities I, II, III, IV, VI and VII were important to all health educators. Strong agreement also existed that the competencies included in Responsibilities I, II, III, IV, V, and VII were used in their present professional setting. Weak agreement emerged regarding use of the competencies included in Responsibility VI.
Internship Preceptor Evaluation
As part of a capstone experience, student majors complete an eight-semester-hour internship with a community health agency. Prior to the internship, the curriculum prepares students to perform in all seven areas of responsibility for entry-level health educators.
Internship site preceptors for students enrolled in the internship during spring and summer 1992 semesters completed a survey instrument regarding performance of student interns in relation to the seven areas of responsibility. Fifty-three survey instruments were sent to preceptors, with 36 instruments (67%) returned. The instrument used a four-point scale ranging from 4, very competent, to 1, not competent. Means for the seven areas of responsibility fell in a narrow range from a high of 3.32 for Responsibility II to a low of 3.0 for Responsibility IV.
Other Assessment Activities
Each semester, graduating seniors participate in an exit interview with the department chairperson to allow students to evaluate their academic experiences in the program. Questions are asked concerning the students' overall satisfaction with their training, as well as students' perceptions of the quality of relationships with the faculty, opportunities for professional involvement, and program strengths and weaknesses. Alumni surveys are conducted every two years to ascertain whether the program has adequately prepared students for their present positions. Alumni are asked to evaluate skills and qualities acquired through the program that are most beneficial in their jobs and to identify competencies that need to be addressed or emphasized.
Future Assessment Activities
The next phase of the comprehensive assessment plan will pilot-test a portfolio system of assessment. Student volunteers are gathering samples of their core course work for inclusion in a portfolio. Examples include papers, essays, program proposals, curriculum guides, grants, presentation outlines, written exams, and research papers. Portfolios will be reviewed and analyzed by members of the Department Assessment and Curriculum Committees to determine progress of the students and the program in developing expertise in the responsibilities and competencies of the entry-level health educator. This information, along with other assessment feedback, will be used to further revise and redirect curriculum.
IMPLICATIONS OF THE ASSESSMENT
After reviewing the completed assessment data, the Health Studies Dept. has determined that the professional preparation program is adequately preparing students to meet entry-level competencies for Responsibilities I, II, III, IV, and VII. Further consideration will be given to revising or redirecting the curriculum to include material to prepare students to meet the competencies required for Responsibilities V and VI. The greatest consensus regarding adequacy of the students' preparation emerged among internship preceptors, suggesting the possibility that either the competencies integral to Responsibilities V and VI are deemed less important in professional positions for the entry-level individual or that student competence is considered adequate in these areas for entry-level positions.
By using the Role Delineation Project Curriculum Framework, health education professional preparation programs can collect relevant and informative assessment data. The assessment approaches taken at Eastern Illinois University provided valuable information and direction for curricula planning and evaluation. These assessment methods can be adapted at other institutions and implemented without straining staff and fiscal resources.
References
1. Hutchings P, Marches T. Watching assessment: Questions, stories, prospects. Change. 1990;22(5):12-38.
2. Bloom A. Closing of the American Mind. New York, NY: Harper and Row; 1987.
3. Boyer E. College: The Undergraduate Experience in America. New York, NY: Harper & Row; 1987.
4. El-Khawas EH. Campus Trends, 1990. Washington, DC: American Council on Education; 1990.
5. National Association of State Universities and Land Grant Colleges and Universities. Statement of Principles on Student Outcomes Assessment. 1988.
6. Wolff R. Assessment and accreditation: A shotgun marriage? Assessment 1990: Application and Renewal. Washington, DC: The American Association for Higher Education; 1990.
7. Commission on Institutions of Higher Education. Assessment Workbook. Chicago, Ill: Commission on Institutions of Higher Education; 1991.
8. Spicuzza F. Two-part examination at University of Tennessee at Knoxville. Assessment Update. 1990;4(2):5-7.
9. Williams C, Rizzo D. Cross-validation approach at Kean College. Assessment Update. 1990;4(2): 1-7.
10. Ewell P. Assessment, Accountability, and Improvement: Managing the Contradiction. Washington, DC: American Association for Higher Education; 1987.
11. Boud D. Three principles for good assessment practices. The New Academic. 1991;1(1):4-5.
12. Gray P. Campus profiles. Assessment Update. 1991;3(2):6.
13. Forrest A. Time Will Tell: Portfolio-Assisted Assessment of General Education. Washington, DC: American Association for Higher Education; 1990.
14. Larson R. Using portfolios to assess the impact of a curriculum. Assessment Update. 1991 ;3(2):9-11.
15. National Task Force on the Preparation and Practice of Health Educators, Inc. A Framework for the Development of Competency-Based Curricula for Entry-Level Health Educators. New York, NY: National Commission for Health Education Credentialing, Inc; 1985.
16. McMahon J, Bruess C, Lohrmann D. Three applications of the Role Delineation Project 1985 Curriculum Framework. J Sch Health. 1987;57(7):247-278.
17. Schmidt G, Beall S. Role delineation competencies and the entry-level health educator. Health Educ. 1988:45-47.
18. Hayden J. Preparing for certification: Health educators' perceived competencies. Health Educ. 1990;21(3):41-42.
19. Knight E. Curriculum planning for professional education: Meeting national standards and local needs. J Health Educ. 1991; 22(4):226-232.
20. Patterson S. Assessing the validity and levels of importance of the role delineation competencies as defined by the National Task Force on the Preparation and Practice of Health Educators, Inc., as perceived by selected health educators in Illinois, unpublished thesis. Carbondale, Ill: Southern Illinois University: 1989.
Marietta Deming, PhD, CHES, Associate Professor; Kathleen Doyle, PhD, CHES, Professor; and Susan Woods, MS, Associate Professor, Dept. of Health Studies, Eastern Illinois University, Charleston, IL 61920. This article was submitted January 28, 1993, and accepted for publication April 26, 1993.
