Evaluation and Testing

Projects
E-TEAM (Educational Training, Evaluation, Assessment and Measurement)
College of Continuing Education
University of Oklahoma

Here is a sampling of our projects:

Great Expectations Evaluation
The Educational Training, Evaluation, Assessment and Measurement Department (E-TEAM) in the College of Continuing Education at the University of Oklahoma designed and conducted a quasi-experimental study to determine if students in classrooms of teachers implementing Great Expectations demonstrate greater gains in achievement on a standardized student achievement test during the school year as compared to students in demographically similar classrooms not implementing Great Expectations (GE). Principals, teachers, students, and parents were surveyed to examine attitudes and beliefs relevant to implementation of GE practices and other aspects of the environment that could affect student achievement. Participating classrooms were observed to document implementation of GE Practices, and administrative data was collected on referrals, absences, and suspensions. This study used scientific methodologies that meet the silver standard of research identified by the What Works Clearinghouse.

Growing With Mathematics Evaluation
This study was a national cross-site evaluation comparing gains in math achievement for students in grades K-5 receiving or not receiving instruction using the McGraw Hill Growing with Mathematics Curriculum (GWM). The University of Oklahoma's Educational Training, Evaluation, Assessment and Measurement (E-TEAM) Department designed and implemented a study to investigate the impact of using the GWM series of materials and curriculum. The study examined gains in student mathematics achievement in K-5 classrooms using the GWM program during the 2003-2004 school year compared to gains in student mathematics achievement in demographically similar classrooms not using the GWM materials. Differences in teacher attitudes toward their mathematics curricula, training, and classroom situation were also explored. This study used scientific methodologies that meet the silver standard of research identified by the What Works Clearinghouse. This evaluation was funded by McGraw Hill, Inc.

Reading Success Network Evaluation - RSN is a national multi-site project designed to improve student reading achievement by developing a network of teacher coaches who promote effective classroom reading instruction. Fifteen comprehensive technical assistance centers funded by the U.S. Department of Education provide RSN training to support early literacy in schools. The E-TEAM recently completed a national cross-site evaluation comparing gains in K-3 student reading in schools implementing RSN to students in demographically comparable schools not implementing RSN. The evaluation also includes examining how aspects of RSN implementation affect student outcomes, teacher behaviors, and classroom instruction.

Center for Early Childhood Professional Development
-Apprenticeship Program Evaluation - The E-TEAM is providing process and outcome evaluation to document the effects of an early childhood apprentice program
-Educator Performance Evaluation - The Center for Early Childhood Professional Development (CECPD) hires independent contractors to provide training programs to caregivers who work in licensed child care settings in Oklahoma. The E-TEAM conducts site visits to observe a sampling of the consultant educators using a rubric developed to assess educator instructional strategies. The evaluation also includes a self-administered student survey instrument that was developed to provide a measure of student outcomes. The data is analyzed and reported to guide educator training programs, provide feedback to educators and provide information for program curriculum development.

Early Steps to Literacy (ESTL)
The ESTL program prepares early childhood educators to foster language and literacy competencies in low-income children. The E-TEAM is responsible for the developing and maintaining the project's website and a LISTSERV discussion group to facilitate communication between participants. High-speed scan-able forms were developed by E-TEAM staff, which greatly improve reliability of the data. E-TEAM staff scan the data into a database and implement data quality edits to ensure the data is clean and complete.

Oklahoma Crash Outcomes Data Evaluation System (CODES) - CODES is an ongoing project that links data from motor vehicle crashes and health outcomes data (e.g. inpatient hospitalizations, death, EMS, etc) for use in motor vehicle safety research and public policy development efforts. CODES is funded by the Oklahoma Highway Safety Office and The National Highway Traffic Safety Administration.

Department of Human Services
-Job Profiling Project - The E-TEAM worked with the Child Support Enforcement Certification Program to develop a child support specialist job profile for use in developing the certification program. Focus groups and work groups were used to identify tasks performed and the knowledge and skills needed for doing child support enforcement within six job areas: case initiation, locate, establishment, finance, enforcement and interstate. Generalists who perform tasks in more than one job area were also included in the process. A researcher from the OU E-ETEAM also shadowed child support specialists in each job area and observed the tasks performed. Child support specialists also completed an online survey developed from the identified knowledge, skills and tasks for each job area. The Child Support Certification Job Profile Report was published June 2003.
-Key Indicators Planning Document
-Outcomes and Outcome Measurement Management Document
-Child Support Enforcement Customer Assistance Center Evaluation
-Center for Early Childhood Professional Development Annual Report
-Supervisor Academies I-VI Evaluation Reports and comparisons
-Office of Field Operations Division Web-based survey - track training, viewing formats, and curriculum.

Oklahoma City Community-wide Mental Health Needs Assessment
The E-TEAM is conducting a county-wide mental health assessment in Oklahoma City, Oklahoma for the Oklahoma County Mental Health Association. A phone survey was conducted using a stratified random sample of residents in Oklahoma County, to include special populations, e.g. the homeless, African Americans, American Indians, Hispanic Americans, and Asian Americans. The purpose of the assessment is to determine unmet mental health needs throughout the county. These data will be used to guide programming, funding, policies, and legislation. This project is funded by the Oklahoma City Community Foundation with additional support from the Division of Public and Community Services within the College of Continuing Education at the University of Oklahoma and in partnership with the Oklahoma County Mental Health Association, the Oklahoma Department of Mental Health and Substance Abuse Services, the Department of Psychiatry at the University of Oklahoma Health Sciences Center, the Oklahoma Department of Rehabilitation, and the Oklahoma Health Care Authority.

Oklahoma City Public Schools
-Gaining Early Awareness and Readiness for Undergraduate Programs - GEAR UP- Collection and reporting on qualitative and quantitative data collected via direct observation, record analyses, interviews, and surveys of parents, students and teachers to determine if programs providing college preparation for high-risk inner city youth results in increased numbers of students applying and admitted to college. This evaluation is funded by the Oklahoma City Public School District.

Eagle's Wings Program Evaluation - Process and outcome evaluation focusing on adolescent resiliencies.

The Effects of Operation Aware Oklahoma Program On Students in Grades Three Through Nine

Oklahoma Learn & Serve America
-Community-Based Projects Evaluation Report
-School-Based Projects Evaluation Report

Oklahoma Rural Elementary Schools: A Descriptive Study - Qualitative and quantitative evaluation of educational experiences in Oklahoma rural school districts.

Operation Aware of Tulsa Evaluation Report

E-TEAM | Public and Community Services | College of Continuing Education | University of Oklahoma |

	`Great Expectations Evaluation` The Educational Training, Evaluation, Assessment and Measurement Department (E-TEAM) in the College of Continuing Education at the University of Oklahoma designed and conducted a quasi-experimental study to determine if students in classrooms of teachers implementing Great Expectations demonstrate greater gains in achievement on a standardized student achievement test during the school year as compared to students in demographically similar classrooms not implementing Great Expectations (GE). Principals, teachers, students, and parents were surveyed to examine attitudes and beliefs relevant to implementation of GE practices and other aspects of the environment that could affect student achievement. Participating classrooms were observed to document implementation of GE Practices, and administrative data was collected on referrals, absences, and suspensions. This study used scientific methodologies that meet the silver standard of research identified by the What Works Clearinghouse.
	`Growing With Mathematics Evaluation` This study was a national cross-site evaluation comparing gains in math achievement for students in grades K-5 receiving or not receiving instruction using the McGraw Hill Growing with Mathematics Curriculum (GWM). The University of Oklahoma's Educational Training, Evaluation, Assessment and Measurement (E-TEAM) Department designed and implemented a study to investigate the impact of using the GWM series of materials and curriculum. The study examined gains in student mathematics achievement in K-5 classrooms using the GWM program during the 2003-2004 school year compared to gains in student mathematics achievement in demographically similar classrooms not using the GWM materials. Differences in teacher attitudes toward their mathematics curricula, training, and classroom situation were also explored. This study used scientific methodologies that meet the silver standard of research identified by the What Works Clearinghouse. This evaluation was funded by McGraw Hill, Inc.
	`Reading Success Network Evaluation` - RSN is a national multi-site project designed to improve student reading achievement by developing a network of teacher coaches who promote effective classroom reading instruction. Fifteen comprehensive technical assistance centers funded by the U.S. Department of Education provide RSN training to support early literacy in schools. The E-TEAM recently completed a national cross-site evaluation comparing gains in K-3 student reading in schools implementing RSN to students in demographically comparable schools not implementing RSN. The evaluation also includes examining how aspects of RSN implementation affect student outcomes, teacher behaviors, and classroom instruction.
	`Center for Early Childhood Professional Development` -Apprenticeship Program Evaluation - The E-TEAM is providing process and outcome evaluation to document the effects of an early childhood apprentice program -Educator Performance Evaluation - The Center for Early Childhood Professional Development (CECPD) hires independent contractors to provide training programs to caregivers who work in licensed child care settings in Oklahoma. The E-TEAM conducts site visits to observe a sampling of the consultant educators using a rubric developed to assess educator instructional strategies. The evaluation also includes a self-administered student survey instrument that was developed to provide a measure of student outcomes. The data is analyzed and reported to guide educator training programs, provide feedback to educators and provide information for program curriculum development.
	`Early Steps to Literacy (ESTL)` The ESTL program prepares early childhood educators to foster language and literacy competencies in low-income children. The E-TEAM is responsible for the developing and maintaining the project's website and a LISTSERV discussion group to facilitate communication between participants. High-speed scan-able forms were developed by E-TEAM staff, which greatly improve reliability of the data. E-TEAM staff scan the data into a database and implement data quality edits to ensure the data is clean and complete.
	`Oklahoma Crash Outcomes Data Evaluation System (CODES)` - CODES is an ongoing project that links data from motor vehicle crashes and health outcomes data (e.g. inpatient hospitalizations, death, EMS, etc) for use in motor vehicle safety research and public policy development efforts. CODES is funded by the Oklahoma Highway Safety Office and The National Highway Traffic Safety Administration.
	`Department of Human Services` -Job Profiling Project - The E-TEAM worked with the Child Support Enforcement Certification Program to develop a child support specialist job profile for use in developing the certification program. Focus groups and work groups were used to identify tasks performed and the knowledge and skills needed for doing child support enforcement within six job areas: case initiation, locate, establishment, finance, enforcement and interstate. Generalists who perform tasks in more than one job area were also included in the process. A researcher from the OU E-ETEAM also shadowed child support specialists in each job area and observed the tasks performed. Child support specialists also completed an online survey developed from the identified knowledge, skills and tasks for each job area. The Child Support Certification Job Profile Report was published June 2003. -Key Indicators Planning Document -Outcomes and Outcome Measurement Management Document -Child Support Enforcement Customer Assistance Center Evaluation -Center for Early Childhood Professional Development Annual Report -Supervisor Academies I-VI Evaluation Reports and comparisons -Office of Field Operations Division Web-based survey - track training, viewing formats, and curriculum.
	`Oklahoma City Community-wide Mental Health Needs Assessment` The E-TEAM is conducting a county-wide mental health assessment in Oklahoma City, Oklahoma for the Oklahoma County Mental Health Association. A phone survey was conducted using a stratified random sample of residents in Oklahoma County, to include special populations, e.g. the homeless, African Americans, American Indians, Hispanic Americans, and Asian Americans. The purpose of the assessment is to determine unmet mental health needs throughout the county. These data will be used to guide programming, funding, policies, and legislation. This project is funded by the Oklahoma City Community Foundation with additional support from the Division of Public and Community Services within the College of Continuing Education at the University of Oklahoma and in partnership with the Oklahoma County Mental Health Association, the Oklahoma Department of Mental Health and Substance Abuse Services, the Department of Psychiatry at the University of Oklahoma Health Sciences Center, the Oklahoma Department of Rehabilitation, and the Oklahoma Health Care Authority.
	`Oklahoma City Public Schools` -Gaining Early Awareness and Readiness for Undergraduate Programs - GEAR UP- Collection and reporting on qualitative and quantitative data collected via direct observation, record analyses, interviews, and surveys of parents, students and teachers to determine if programs providing college preparation for high-risk inner city youth results in increased numbers of students applying and admitted to college. This evaluation is funded by the Oklahoma City Public School District.
	`Eagle's Wings Program Evaluation` - Process and outcome evaluation focusing on adolescent resiliencies.
	`The Effects of Operation Aware Oklahoma Program On Students in Grades Three Through Nine`
	`Oklahoma Learn & Serve America` -Community-Based Projects Evaluation Report -School-Based Projects Evaluation Report
	`Oklahoma Rural Elementary Schools: A Descriptive Study` - Qualitative and quantitative evaluation of educational experiences in Oklahoma rural school districts.
	`Operation Aware of Tulsa Evaluation Report`