Research Projects

Conduct scientifically-based research: High school ELL program effectiveness

Client: 

National Geographic School Publishing

The Challenge: 

As the population of students whose primary language is not English increases, there is a growing demand for English Language Learner (ELL) instruction at the high-school level. At the same time, buyers and users of such programs are demanding evidence of effectiveness before adoption. SEG’s client, a publisher of ELL curriculum and instructional programs, recognized the importance of scientifically-based research (SBR) in proving program effectiveness. As a leading provider of scientifically-based research services, SEG was asked to design and implement a controlled study of program effectiveness

The Solution: 

SEG designed and conducted a scientific effectiveness study of the client’s ELL instructional program. A quasi-experimental design was used to investigate program effectiveness. More than 100 schools participated in the study, with half of the classes using the Publisher’s instructional program, and the remaining half using traditional instructional practice. Analysis of Covariance was used to adjust for potential differences between the groups resulting from non-random assignment. The English language skills of both groups were measured at the beginning of the school year and at the conclusion of the school year. The proficiency of the two groups was compared statistically to determine if users of the program showed greater improvement. The effectiveness of the Program was described in a report for use by the Publisher and school’s considering the Program.

Data-Based Case Study: The Effectiveness of a Cognitive Science-Based Reading Program Targeted at Improving Neural Pathways

Client: 

John Burgess Education

The Challenge: 

Do students that use the Reading Performance System's exercises for the practice and development of foundational reading skills that strengthen neural pathways show significant gains in reading skills?

The Solution: 

During the 2010-2011 school year, John Burgess Education Technology conducted a year-long study of ethnically diverse students in Marion County, Florida, using the Reading Performance System (RPS) to evaluate the effectiveness of the RPS in improving student reading skills. SEG analyzed the improvement in reading skills attained by students using the Reading Performance System between the beginning and end of the 2010-2011 school year, as measured by two widely used measures of reading skills. The findings of the study demonstrate that students using the John Burgess Education Technology Reading Performance System Instructional Program show significant improvement in reading. The positive findings of this study need to be confirmed with larger samples of students and with more robust research designs employing a treatment-control group design.

The Effectiveness of a Developmental Reading & Writing Online Learning Digital Tool in Institutions of Higher Learning

Client: 

Cengage

The Challenge: 

Does Cengage Learning's Aplia, an online learning solution that that corresponds with specific Cengage textbook chapters for developmental reading and writing college students, maintain student learning interest and increase end of semester scores?

The Solution: 

MarketingWorks and SEG Measurement conducted a study of Aplia's effectiveness in developmental reading and writing classes in the fall of 2012. Over 750 students and 35 instructors at two and four year institutions of higher education were involved in a semester-long study which included interviews, biweekly feedback, qualitative and quantitative data. Students were compared in classes using Aplia vs. those not using Aplia, but with the same textbook.
In this national, semester-long study, Aplia was found to be an effective tool for improving student knowledge and skills. The results show that students in classes using Aplia showed greater growth in reading and writing knowledge and skills than students in classes that did not use Aplia

Evaluating the Effectiveness of a Preschool Music Program on Students' Language Arts and Literacy Skills

Client: 

Kindermusik International

The Challenge: 

Do students that use the ABC Music & Me program attain higher literacy and language arts skills than students who do not use ABC Music & Me?

The Solution: 

SEG Measurement, an independent research firm, conducted a national study of nearly 300 preschool children, aged three years through five years, to test the ability of Kindermusik International's curriculum ABC Music & Me affect on students' literacy development. During the study, students in both the treatment and control groups were tested with the Phonological Awareness Literacy Screening (PALS) Pre-K Assessment at both the beginning and end of the school year. Results indicated that students in the treatment group showed a 32% higher gain in PALS scores over the control group, indicating greater gains in student's literacy as a result of the curriculum

Evaluating the Effectiveness of a Supplementary Math Skills Resource Program on Elementary School Students

Client: 

Mentoring Minds

The Challenge: 

Do students that use Mentoring Minds' Motivation Math supplementary math skills program achieve do better in math than students that do not use Motivation Math?

The Solution: 

Over 300 students were included in the study across grades 3, 4, and 5. The study incorporated a pretest and posttest design with Treatment and Control Groups. The Treatment Group was composed of students in classes that used Motivation Math. The Control Group was composed of students in comparable schools that did not use Motivation Math. Students who completed both the pretest and the posttest were included in the analyses. The comparability of the pretest ability of the two groups was evaluated to ensure the posttest data could be compared. Students who used Motivation Math were found to have a higher mean posttest ability level than the students who did not use Motivation Math.

Examining the Effectiveness of Learning Management Digital Tools in Post-Secondary Education

Client: 

Cengage

The Challenge: 

Does Cengage Learning's CengageNOW, an online teaching and learning resource that provides instructors with learning management tools, essential class activities, and teaching and learning resources, deliver better student outcomes?

The Solution: 

MarketingWorks and SEG Measurement, two independent research firms, conducted a comprehensive study of CengageNOW’s effectiveness in the spring of 2013 using a variety of qualitative and quantitative methods. Pre- and posttests of the accounting knowledge and skills taught in the course were administered, and differences in end-of-course grades between CengageNOW users and non-users were examined as a basis for evaluating the effectiveness of CengageNOW. In addition, the 15 instructors and a sample of 27 students using CengageNOW provided feedback at the beginning and end of the study and bi-weekly throughout the semester. CengageNOW was found to be an effective tool for improving student knowledge and skills.

Examining the Effectiveness of Learning Management Digital Tools in Post-Secondary Education

Client: 

Cengage

The Challenge: 

Does Cengage Learning's CengageNOW, an online teaching and learning resource that provides instructors with learning management tools, essential class activities, and teaching and learning resources, deliver better student outcomes?

The Solution: 

MarketingWorks and SEG Measurement, two independent research firms, conducted a comprehensive study of CengageNOW’s effectiveness in the spring of 2013 using a variety of qualitative and quantitative methods. Pre- and posttests of the accounting knowledge and skills taught in the course were administered, and differences in end-of-course grades between CengageNOW users and non-users were examined as a basis for evaluating the effectiveness of CengageNOW. In addition, the 15 instructors and a sample of 27 students using CengageNOW provided feedback at the beginning and end of the study and bi-weekly throughout the semester. CengageNOW was found to be an effective tool for improving student knowledge and skills.

Improving Literacy Skills in Pre-K Students: a study of the impact of an early learning program on student academics and behavior

Client: 

Houghton Mifflin Harcourt

The Challenge: 

Pre K schools/classrooms are increasingly looking for solutions that will help students prepare for later academic work. Toward this end, HMH developed a new pre-k program, Splash into pre k to support the development of early childhood students. HMH was looking to evaluate the efficacy of Splash and was looking for a means of formative evaluation for use in continued improvement of Splash.

The Solution: 

SEG Measurement designed an evaluation for HMH to determine the effectiveness of Splash into Pre-K™ as a tool for improving student literacy skills and to provide formative information for use in improving the program. Using pre-post test design, SEG Measurement conducted and evaluation of the Splash program in approximately 15 preschool classrooms with approximately 300 children. Teachers provided feedback approximately every two weeks on program components and student academic development was measured using PALS (Phonological Awareness Literacy Screen) at the beginning and end of the year after receiving instruction. The results were then compared statistically

Improving Student Learning Through Online Tools: study of the effectiveness of a test preparation program for high school students

Client: 

USATestprep

The Challenge: 

Improving students’ academic skills, particularly in the core content areas of language arts and mathematics remains a critical part of the educational mission. As schools seek to develop student reading and mathematics skills Common Core Skills, they are looking for solutions that will assist them in achieving these goals. USATestprep offers a web-based solution to help schools in developing student academic skills. The client sought to validate the efficacy of the product for use in schools and to measure of the impact the program has on student learning and performance.

The Solution: 

SEG designed and conducted a scientific effectiveness study for USATestprep integrating both qualitative and quantitative data for the evaluation. Student achievement gains were evaluated through Stanford 10 tests as well as state-based test results. The study also explored potential differences in growth between boys and girls and among students of different ethnic backgrounds. A quasi-experimental design was used to compare two groups of students, similar in ability. Based on a year-long, multi-state study, students using USATestprep, to improve their academic skills, showed substantially greater growth in both Reading and Mathematics than students who did not use USATestprep. The study also found that USATestprep is equally effective for boys and girls and for students of different ethnic backgrounds.

Improving Students' Science and Knowledge of Skills: A Study of an Augmented Reality Library Program

Client: 

SK Telecom

The Challenge: 

3D Library is an instructional tool enhanced by augmented reality, combining written books with computer-delivered 3D animation. The client, SK Telecom, wanted to know if this type of instructional tool would increase science knowledge and skills as compared to students in classes that used only their books or in classes that did not use any of the 3D Library Program components.

The Solution: 

The study examined the effectiveness of the 3D Library Program by comparing the growth in science knowledge and skills among three sets of students, 1) using 3D Library with both the books and computer-based animations, 2) to a comparable group of students who used only the books and 3) a control group that used none of the 3D Library components. We compared the gains made by students in all three study groups, controlling for any initial differences in science knowledge and skills.
Students who used both the 3D Library Program books and the computer-delivered augmented reality animations (Treatment Group 1) were compared to both those students who used only the books
from the 3D Library Program and to the Control Group of students that did not use any of the 3D Library components. We used a statistical procedure known as analysis of covariance (ANCOVA), to provide a more accurate comparison of growth among groups.