OU Home

E-TEAM at The University of Oklahoma College of Continuing Education designs research and evaluations to help organizations understand and use information and data to solve real-world problems. Our well-rounded evaluations help you apply fresh thinking and original solutions to complex problems.

At E-TEAM , evaluations are viewed as a dynamic process to help improve programs as they are developed and implemented. Our experienced research team will work with you to identify your evaluation needs and clarify the central questions you would like to answer.

E-TEAM will then plan a systematic evaluation using appropriate measures and tools to collect data to answer those evaluation questions. We help you monitor progress with regular feedback, technology solutions, evaluation findings, and recommendations.

All E-TEAM evaluations are designed to be in compliance with nationally mandated standards for conducting research involving human participants. Our goal is to help you gather the information you need to make informed decisions and improve program outcomes.

E-TEAM researchers and evaluators have extensive experience in:

  • Research and evaluation design
  • Quantitative and qualitative methods
  • Data collection and analyses
  • Reporting and report automation
  • Cost/benefit analyses
  • Needs assessments
  • Survey development and administration
  • Database design and development
  • Secure web data implementations
  • Locating relevant funding opportunities
  • Grant writing and consultation
  • Training

Early Literacy Quick Assessment (ELQA)

In collaboration with the Center for Early Childhood Professional Development , E-TEAM developed Early Literacy Quick Assessment software that facilitates formative progress monitoring of early literacy skills. Children's early literacy skills are assessed at the beginning of the school year and again at intervals throughout the school year so that teachers are able to 1) identify children at risk for not meeting early literacy benchmarks, and 2) differentiate instruction to prepare young children to enter kindergarten with the necessary language and literacy skills to benefit from formal school instruction.

Teachers use the ELQA data to plan instruction that is differentiated to address each child's learning needs. Assessment data also helps teachers pinpoint what is and is not working in their curriculum so they may modify instruction to optimize student learning. The ELQA can provide data to guide flexible grouping instruction for all students in the classroom. An example of how ELQA data can be used to to examine the effects of an intervention, see the Energizing Readiness Curriculum Study Report . For more information, see the ELQA website .

Parents as Teachers Evaluation

E-TEAM designed and conducted a quasi-experimental study to determine if students of parents who participated in the Parents as Teachers (PAT) program demonstrate greater gains in literacy skills compared to demographically-similar students whose parents did not participate in the PAT program. Data were collected from parents on family and student demographics and on literacy promoting behaviors. PAT parents were also surveyed regarding their opinions of the PAT program. PAT program data was collected from the OKCPS PAT office on program participation. School readiness data were collected from teachers. Student literacy skills were assessed across three years: during fall of their pre-kindergarten year, spring Pre-K, spring kindergarten and spring 1st grade. The English and Spanish versions of the Phonological Awareness Literacy Screening (PALS) assessment and the Peabody Picture Vocabulary test (PPVT) were used to assess literacy skills and vocabulary. View the PAT Evaluation Report Executive Summary.

Great Expectations Evaluation

E-TEAM designed and conducted a quasi-experimental study to determine if students in classrooms of teachers implementing Great Expectations demonstrate greater gains in achievement on a standardized student achievement test during the school year as compared to students in demographically-similar classrooms not implementing Great Expectations (GE). Principals, teachers, students and parents were surveyed to examine attitudes and beliefs relevant to implementation of GE practices and other aspects of the environment that could affect student achievement. Participating classrooms were observed to document implementation of GE Practices, and administrative data was collected on referrals, absences, and suspensions. This study used scientific methodologies that meet the silver standard of research identified by the What Works Clearinghouse.

Oklahoma Preparing Readers and Initiating Developmental Excellence Evaluation

Oklahoma PRIDE (Preparing Readers and Initiating Developmental Excellence) was one of 30 Early Reading First grants awarded in 2003 across the country. The goal of the grant was to create Centers of Excellence in three elementary schools in south Oklahoma City and one community-based child development center. The focus of Oklahoma PRIDE was to provide teachers with the knowledge and skills to ensure that children obtain early literacy skills and to support teachers in predominately Spanish-speaking schools and low-income families. Throughout the Oklahoma PRIDE project, teachers, teacher assistants and paraprofessionals were provided intensive research-based, early-literacy professional development in the form of ongoing classes, seminars, institutes and coaching/mentoring.

Reading Success Network Evaluation

Reading Success Network (RSN) was a national multi-site project designed to improve student reading achievement by developing a network of teacher coaches who promoted effective classroom reading instruction. Fifteen comprehensive technical assistance centers funded by the U.S. Department of Education provided RSN training to support early literacy in schools. E-TEAM conducted a national cross-site evaluation comparing gains in K-3 student reading in schools implementing RSN to students in demographically-comparable schools not implementing RSN. The evaluation also included an examination of how aspects of RSN implementation affected student outcomes, teacher behaviors, and classroom instruction.

Sooner Teachers Advocating Literacy to Kids Evaluation

The goal of Sooner Teachers Advocating Literacy to Kids (TALK) was to transform existing early-education programs into programs of excellence that provided high-quality early education to young children, especially those from low-income families. The purpose was to prepare young children to enter Kindergarten with the language and literacy skills necessary to ensure school success. The U.S. Department of Education’s four-million-dollar award supported the Sooner TALK project, a collaboration of the Center for Early Childhood Professional Development, Oklahoma City Public Schools, three private childcare centers in Oklahoma City, and the Department of Instructional Leadership and Academic Curriculum in the College of Education to improve children's early reading and language skills.

McGraw Hill Growing With Mathematics Program Evaluation

E-TEAM conducted a national cross-site evaluation comparing gains in math achievement for students in grades K-5 receiving or not receiving instruction using the McGraw Hill Growing with Mathematics Curriculum (GWM). E-TEAM designed and implemented a study to investigate the impact of using the GWM series of materials and curriculum. The study examined gains in student mathematics achievement in demographical similar classrooms not using the GWM materials. Differences in teacher attitudes towards their mathematics curricula, training, and classroom situation were also explored. This evaluation was funded by McGraw Hill, Inc.

Mid Continent Comprehensive Center Evaluation

E-TEAM currently conducts the Mid Continent Comprehensive Center (MC3) external evaluation. This evaluation assesses the extent to which MC3 is making progress towards meeting its performance measures. The MC3 evaluation is utilization-focused and participatory in nature, incorporating a strong and ongoing feedback loop for continuous quality improvement. The evaluation is designed to assess program efficacy based on the congruence of MC3 goals and objectives with MC3 technical assistance (TA) service delivery. The evaluation focuses on program processes and TA service outcomes to determine MC3’s progress in implementing the processes and strategies necessary to achieve its objectives and outcomes detailed in the Work Plan. View the MC3 Evaluation Report Executive Summary. \

The following projects are selected examples of E-TEAM’s work in the area of Behavioral Health:

Care Management Evaluation

E-TEAM is evaluating a new cross agency, statewide care coordination program for children and youth with the most intense needs and their families. Partner agencies include the Oklahoma Department of Mental Health and Substance Abuse Services, Oklahoma Health Care Authority, Department of Human Services, and Office of Juvenile Affairs. The project is designed to create resources through redirecting funding from residential to community-based services, and will strengthen local Systems of Care efforts to integrate mental health and substance abuse services for children, youth, and their families to remain in their own communities. View Care Management Report Executive Summary .

Systems of Care Evaluation

E-TEAM performs administrative and analytical duties to manage the design and implementation of the Oklahoma Children’s Systems of Care program. Oklahoma Systems of Care (OSOC) is a coordinated network of community-based services and supports that are organized to meet the challenges of children and youth with serious mental health needs and their families. E-TEAM conducts site visits at each of the active OSOC sites, maintains and improves the OSOC Youth Information System (YIS) at Systems of Care sites and offers ongoing technical support for use of that data system. E-TEAM uses evaluation findings to inform stakeholders of OSOC project development efforts, including improvement of management procedures, adoption of new system and service policies, and attaining new sources of public and private financing.

Keeping Our Tribes Alive Evaluation

E-TEAM served as an external evaluator for Keeping Our Tribes Alive (KOTA) in assessing the effectiveness of a school-based curriculum in preventing and reducing youth verbal and physical aggression and violence, and the precursors that lead to youth violence. KOTA was a federally-funded grant by the Substance Abuse and Mental Health Services Administration with the purpose of preventing and reducing youth verbal and physical aggression and violence. KOTA was a collaboration of the American Indian Institute and E-TEAM at the University of Oklahoma as well as representatives of public schools, law enforcement, the judiciary, the youth shelter, and juvenile services in Tecumseh, Oklahoma. In addition to pre- and post-test data, E-TEAM conducted formative assessments to ensure program fidelity and to provide the program staff with information to make program improvements.

The following projects are selected examples of E-TEAM’s work in the area of Business Processes:

Job Profiling Project

E-TEAM worked with the Oklahoma Department of Human Services Child Support Enforcement Certification Program to develop a child support specialist job profile for use in developing their certification program. Focus groups and work groups were used to identify tasks performed and the knowledge and skills needed for doing child support enforcement within six job areas. Generalists who perform tasks in more than one job area were also included in the process. An E-TEAM researcher shadowed child support specialists in each job area and observed the tasks performed. Child support specialists completed an online survey developed from the identified knowledge, skills and tasks for each job area. A Child Support Certification Job Profile Report summarized the findings.

Oklahoma Department of Rehabilitation Services Project

E-TEAM worked with the Oklahoma Department of Rehabilitation Services (DRS) to conduct a study to estimate the return on the investment of public resources by DRS in vocational rehabilitation services. The study was designed to estimate the net impact of DRS vocational rehabilitation services on consumers’ later employment and earnings and to assess the benefits and costs associated with DRS vocational rehabilitation services. In the study’s report, “ Evaluating Rehabilitation Services in Oklahoma: An Analysis of Program Impacts and of Benefits/Costs ,” E-TEAM noted the following key findings: DRS services increased employment and earnings; these increased employment and earnings were broad-based; and the benefits of DRS services outweigh the costs, and the program pays for itself multiple times.

The following projects are selected examples of E-TEAM’s work in the area of Public Health and Safety:

Tobacco Free Zone Evaluation

E-TEAM is evaluating a project funded by the OSU Seretean Wellness Center that attempts to reduce tobacco use in low-income housing in three sites in Tulsa, Oklahoma. Along with reducing tobacco use, the project attempts to eliminate second-hand smoke exposure, make 10 percent of the residence buildings non-smoking, and increase the awareness of the Oklahoma Tobacco Help Line.

Oklahoma City Community-wide Mental Health Needs Assessment

E-TEAM conducted a county-wide mental health assessment in Oklahoma City, Oklahoma for the Oklahoma County Mental Health Association. A phone survey was conducted using a stratified random sample of residents in Oklahoma County, to include special populations—homeless, African Americans, American Indians, Hispanic Americans, and Asian Americans. The purpose of the assessment was to determine unmet mental health needs throughout the county. These data were then used to guide programming, funding, policies, and legislation.

Crash Outcome Data Evaluation System (CODES)

E-TEAM used probabilistic matching to link data from motor vehicle crashes and health outcomes data (e.g. inpatient hospitalizations, death, EMS, etc) for use in motor vehicle safety research and public policy development efforts. CODES was funded by the Oklahoma Highway Safety Office and The National Highway Traffic Safety Administration.

American Education Corporation Center for Public Management
Oklahoma Department of Human Services Oklahoma Highway Safety Office
Northeastern State University Eagle Ridge Institute
Center for Early Childhood Professional Development Oklahoma State Regents for Higher Education
American Indian Institute Oklahoma City Public Schools
Mental Health Organization of Central Oklahoma Wright Group McGraw-Hill
Oklahoma Department of Mental Health Total Life Counseling (TLC)
Center for Early Childhood Professional Development Oklahoma Professional Development Program
ORC-Macro Western Heights School District
Edvance Mid Continent Comprehensive Center
National Center for Disabilities Education and Training Oklahoma Department of Education
U.S. Department of Education U.S. Office of Juvenile Justice and Delinquency
Saxon Publishing U.S. Department of Health and Human Services
Oklahoma Health Care Authority Oklahoma State University Seretean Wellness Center
U.S. Department of Transportation Substance Abuse and Mental Health Services Admin.
Oklahoma Parent Information and Resource Center (PIRC)

Evaluation and Testing

The Evaluation and Testing Department at the University of Oklahoma, College of Continuing Education has merged with the E-TEAM. If you need to contact us, please e-mail Milton Collier mcollier@ou.edu or by phone at (405) 325-7186.

ScanForms

Services include:

  • Survey Development
  • Custom scannable form development
  • Scanning
  • Custom report automation and development

Belinda Biscoe, Ph.D. - Director

Belinda holds a Ph.D. in psychology. She is also a certified prevention specialist and a certified director for drug and alcohol programs. She has 20 years of experience with school- and community-based programs, including higher education as an administrator, researcher, and program developer. Belinda worked for eight years in the Oklahoma City Public School District as a senior research associate and administrator for Support Programs which included bilingual education, homeless education, safe and drug-free schools, Johnson O'Malley, Indian Education, Community, Title I Education, Early Childhood, Family Literacy, etc. She has developed competitive grants that have resulted in funding for innovative programs in technology, early childhood, education for homeless youths, parenting education, literacy, substance abuse prevention and treatment, bilingual education, health and nutrition, and school-based foreign language curricula. She is also co-founder of a community-based non-profit substance abuse prevention and treatment agency in Oklahoma City. In addition to her professional work, Belinda serves on numerous community boards involved with issues related to social issues, the family and youth. Her eclectic background provides a creative out-the-box approach to research.

 

Senior Research Associates

Geneva Strech, M.Ed., M.H.R. - Associate Director

Geneva has two Master's degrees from The University of Oklahoma—one in Adult Education and one in Human Relations. She has a Bachelor of Arts degree in English and History from the University of New Mexico. Geneva has been with The University of Oklahoma since 1996 and has been responsible for the management and implementation of several federally-funded programs. She has more than 20 years’ experience in developing programs for youth and adults. She has coordinated large-scale evaluations, designed survey, observation, and interview instruments, coordinated data collection, and authored evaluation reports. She has served as a project director, program monitor, curriculum developer, and grant writer for various projects. She has presented at national events on education issues such as curriculum development, technology as a learning tool, lifelong learning and adult learning. She is also a veteran and served as a Sergeant in the U.S. Army.

 

John Vetter, M.A.

John Vetter, MA, a Senior Research Associate for E-TEAM since 2002, received his undergraduate degree from the University of Oklahoma in Sociology and a master's degree in History.  John has served as the lead evaluator for Oklahoma Systems of Care since 2003 and developed the Youth Information System (YIS), a dynamic user interface with data transformation capabilities.  The YIS is a secure, web-based application which provides real-time access to evaluation and program monitoring data to state management and local site leadership.  The YIS has become invaluable as a management tool in the ability of OKSOC site management and staff to identify, better serve, and document positive outcomes for individuals with critical needs. John has presented multiple times at national meetings and workshops on E-TEAM’s work with the OKSOC evaluation.  John worked for several years in Albuquerque, New Mexico designing data collection and monitoring systems for substance abuse programs. John worked at the Center on Alcoholism, Substance Abuse and Addiction, affiliated with the University of New Mexico, where he created the Center's information system in support of a Target Cities grant from the Center on Substance Abuse Treatment. He planned and coordinated data collection for Albuquerque's network of substance abuse providers and handled all aspects of data reporting, to both CSAT and local governmental and university sponsors.

 

Carol Surber, M.A.

Carol Bridges Surber holds a Masters of Arts in Sociology from The University of Oklahoma. She was the director of The University of Oklahoma's Institute for Applied Social Policy. Carol also served as the lead external evaluator and principal grant writer for grantees of federally-funded programs, notably: seven Title III English Language Acquisition programs serving Native American and Spanish-speaking students; two Safe Schools/Healthy Students Initiatives creating community-wide partnerships to prevent violence and drug use among school-age children; two 21st Century Community Learning Centers for after-school support; a Grant to Reduce Access to Alcohol among underage students; and a National Science Foundation project to promote science and technology skills among youth with disabilities. Carol has worked with over 40 school districts in both rural and urban locations across Oklahoma as a consultant, researcher, and grant writer for over 25 years.

 

Research Assistants

José Diaz

José Diaz has a Master’s in Business Administration from the University of Arizona and two Bachelor’s degrees in Physics and Astrophysics from the University of New Mexico. José has three years of compliance and IT audit experience with the Pascua Yaqui Tribe in Tucson, Arizona (of which he is a tribal member). His background in sciences combined with business theory has opened opportunities for him to serve as a technology transfer research project team lead for Los Alamos National Laboratories and INAOE (Instituto Nacional de Astrofísica, Óptica y Electrónica) Puebla, Mexico. Currently, José is focusing on developing web applications for the Oklahoma Department of Mental Health Youth Information Systems. He has experience working with web development languages: VB.net, ASP, SQL Server, Javascript, and Actionscript(Flash). José is bilingual in English and Spanish.

Lisa White

Lisa holds a Bachelor's degree in early childhood education from The University of Oklahoma and is currently working toward her Master’s in Administrative Leadership.  Lisa worked as an early childhood educator for seven years. Lisa has been with the E-TEAM since 2009 and works with the Early Literacy Quick Assessment (ELQA), providing training and technical assistance with the current version and serving as a content resource for software updates and beta testing newly developed capabilities.  She also provides training and technical assistance to Youth Information System (YIS) users.  Lisa manages data collection and data quality control and conducts field interviews and classroom observations. In addition, she trains data collectors, supports data analysis, and assists in report development.

Katty Sze Yee Tong

Katty Sze Yee Tong holds a Bachelor’s degree in Management Information Systems and has been with the E-TEAM since 2011.  Katty previously conducted marketing analyses for several years for wholesale industry.  She provides software development support for the E-TEAM applications, including Youth Information System (YIS) and the Early Literacy Quick Assessment (ELQA).  Katty provides technical assistance and support for incoming queries and issues for YIS and ELQA end-users.  She also provides data analysis and report development for E-TEAM senior researchers.   

Kelly Phillips

Kelly has been with E-TEAM since 2006.  She previously worked as an early childhood educator for several years.  Her expertise in early childhood classrooms has made her a valuable resource for the Early Literacy Quick Assessment (ELQA), and she is working with the development team on the latest ELQA version.  Kelly provides technical assistance and support for incoming queries and issues for both ELQA users and Youth Information System (YIS) users.  Kelly serves as the point-of-contact for the OKSOC Monthly and Quarterly Reports.  Kelly also manages data collection, conducts interviews and observations, and assists with report development.

Sharon Strait

Sharon has been with the E-TEAM since 2008. She previously served as the office manager for a busy dental practice for 26 years and brings that experience in working with people and managing logistics to her work with E-TEAM.  Sharon serves as the point-of-contact for the Healthy Transitions Initiative and sits on several departmental committees.  She conducts field interviews and observations, manages data collection, and assists with report development.

 

Administration

Yuxin Qin, CPA

Yuxin Qin, a Certified Public Accountant in Oklahoma, holds a Master of Accountancy degree from the University of Oklahoma. Prior to joining ETEAM, Yuxin has 5+ years’ experience working in non-profit and industrial accounting environment. Yuxin oversees the development of budget, research grant/contracts and ensures compliance with all relevant university and/or agency regulations and restrictions.

Milton Collier

Milton Collier holds a Bachelor degree in Business Administration with an emphasis in Marketing from The University of Oklahoma. Milton has been with the E-TEAM since 2001 and serves as Operations Manager.  He liaises and networks with a range of stakeholders including customers, colleagues, and partner organizations, communicates with target audiences and manages customer relationships.  He designs and manages the production of E-TEAM materials, including marketing materials and research and evaluation reports.  Milton leads the team charged with the ongoing development and design of the E-TEAM web page.   

The Center for Early Childhood Professional Development (CECPD) and the Educational Testing, Evaluation, Assessment and Measurement Department (E-TEAM) within the University of Oklahoma OUtreach are offering a FREE, no-obligation, trial of The Early Literacy Quick Assessment (ELQA) software for PreK, Head Start and childcare providers.
ELQA website

The Early Literacy Quick Assessment (ELQA) is a software assessment tool that facilitates formative progress monitoring of early literacy skills. Children's early literacy skills are assessed at the beginning of the school year and again at intervals throughout the school year so that teachers are able to, 1) identify children at risk for not meeting early literacy benchmarks, and 2) differentiate instruction to prepare young children to enter kindergarten with the necessary language and literacy skills to benefit from formal school instruction.

Teachers use the ELQA data to plan instruction that is differentiated to address each child's learning needs. Assessment data also helps teachers pinpoint what is and is not working in their curriculum so they may modify instruction to optimize student learning. The ELQA can provide data to guide flexible grouping instruction for all students in the classroom. The ELQA has four sections described below.

Alphabet Knowledge

  • Uppercase Letters: Students are shown 26 uppercase letters and are asked to name each letter.
  • Lowercase Letters: Students are shown 26 lowercase letters and are asked to name each letter.
  • Letter Sounds: Students are shown 26 uppercase letters and asked for the corresponding letter sounds.

Concepts of Print

This test consists of ten items used assess the child's knowledge of aspects of reading. The teacher uses an engaging early level text and asks the child questions about the book (e.g. story title, where to begin reading, which direction to read, return sweep, word by word pointing, meaning of period, etc.). An image that displays letters, numbers, and words is used to determine if the child can point to a letter, a word and the first letter in a word.

Phonological Awareness

  • Rhyme Recognition: Students are asked to decide whether each of five pairs of words rhymes (e.g., cat and hat).
  • Rhyme Generation: Students are asked to generate a word that rhymes with five different words spoken by the teacher.

Vocabulary

  • Receptive Vocabulary: Students are shown a picture or sets of pictures and asked to point to the picture that represents the concept presented by the teacher. Concepts include: Body Parts, Directional Terms, Shapes, Ordinal Terms, Transportation, Community, Nature, Geography, Emotions, and Clothing.
  • Expressive Vocabulary: Students are shown 25 pictures and are asked to name the object shown in each picture. Concepts include: Animals, Body Parts, Clothing, Food, Household Objects, Nature, Common Objects, People, Tools, Toys, and Transportation.

ELQA Reports

The ELQA software generates individual student and classroom-level reports . The student report shows the students' scores for each subtest over time. The classroom report illustrates class progression over time and provides recommended flexible groupings according to the ELQA scores. The group reports are used to facilitate differentiated instruction. The individual reports can be used to help facilitation transition from one classroom to another, and also may be used to aid parent-teacher conferences.

Getting ELQA

STEP 1: SEND E-MAIL TO: inquiries@elqa.eteam.ou.edu IN YOUR E-MAIL, SIMPLY REQUEST TO BE SIGNED UP FOR THE 2013-2014 SCHOOL YEAR, FOR NO COST. FREE! TRY IT FOR THE ENTIRE SCHOOL YEAR. 

STEP 2: A FORM WILL BE RETURNED TO YOU TO FILL OUT, VIA E-MAIL. SIMPLE!

STEP 3: SEND THE FORM BACK VIA E-MAIL.

STEP 4: YOU WILL BE SET UP TO USE ELQA FOR THE SCHOOL YEAR – NO COST!

TECHNICAL SUPPORT AFTER ENROLLMENT: support@elqa.eteam.ou.edu

Financial Inquiries

Yuxin Qin yqin@ou.edu

(405) 325-7801

General Inquiries

Milton Collier mcollier@ou.edu

(405) 325-7186

1639 Cross Center Drive, Suite 308, Norman, Oklahoma 73019-2219


View Larger Map