Research Projects

Conduct Product Effectiveness Research: Higher Education Science Textbooks

Client: 

Wiley Publishing

The Challenge: 

Textbooks continue to play a major role in instruction at institutions of higher education depend on textbooks. Textbook selection is a critical component of the instructional process. Faculty and other members of the higher education community want to be confident that the textbooks they select for their courses are effective. Wiley, a publisher of higher education textbooks recognized the importance of scientifically-based research in the selection process. As a leading provider of research services, SEG was asked to design and implement a controlled study of the effectiveness of their Visualizing Series textbooks. The Wiley Visualizing Series relies heavily on visual content and is grounded in decades of research that suggests that students learn better with words and pictures than with words alone.

The Solution: 

SEG designed and conducted a scientific effectiveness study of Wiley’s Visualizing Series of textbooks. A quasi-experimental design was used to investigate effectiveness of three different science and social science titles.. Eight institutions of higher education reflecting different institutional characteristics (e.g., size, geography) participated in the study. At each participating institution, a set of classes using the Wiley Visualizing Series (treatment group) and a set of classes not using the Wiley Visualizing Series (control group) were identified. At the beginning of the semester each student in both the treatment and control groups was administered an assessment of the content in the content area represented by the title; all students were again assessed using the instrument at the conclusion of the semester. Analysis of Covariance was used to compare the knowledge levels of the two groups, adjusting for potential differences between the groups present at the beginning of the study. The results demonstrating that students who used the Wiley Visualizing Series learned more was described in a report for use by the Publisher and institutions considering adopting the Wiley Visualizing Series.

Conduct an Analysis of Teacher Effectiveness Using Student Growth Measures

Client: 

Northwest Georgia Regional Educational Service Agency

The Challenge: 

It is necessary to provide feedback to teachers regarding performance beyond that which can be provided through classroom observation. One method for providing additional performance information is to evaluate the amount of student growth shown for each teacher compared to a benchmark. Challenges in data management across districts often prohibit the ability to generate these types of analyses in a timely and cost effective manner.

The Solution: 

SEG provided data management, analysis, and reporting services to produce teacher effectiveness data for all teachers in the Northwest Georgia RESA. First, SEG provided data management services to use scheduling records to identify the core reading, language arts, and mathematics teachers for each student throughout the school year. Second, SEG provided data management services to create data sets including all students who have pre (prior Spring) and post (current Spring) CRCT test scores in one or more of reading, language arts, or mathematics. Then, SEG combined this schedule and performance information and conducted Analysis of Covariance (ANCOVA) for each grade and subject using teacher as a random factor and pretest score as the covariate. Using ANCOVA allows for the data to be analyzed as if all of the students started at the same point. This provides a more fair comparison of student growth across teachers. Reports were provided for teachers in grades 3 through 8 in reading, language arts, and mathematics for each district and school. In addition, a report interpretation guide including cautions and limitations was provided.

Conduct a scientifically-based research study of the effectiveness of an online, animated, e-learning application

Client: 

BrainPOP

The Challenge: 

It is widely believed that delivering educational content online can be a very valuable component of education. Moreover, there is widespread recognition that this educational content can be enhanced through animation. While these beliefs are widely held, there was little scientific evidence documenting the effectiveness of BrainPOP’s animated k-12 movies. BrainPOP asked SEG to help in obtaining this evidence.

The Solution: 

SEG designed and implemented a project to identify scientific evidence supporting the effectiveness of the BrainPOP application. First, SEG researched and wrote a White Paper describing the scientific research supporting the use of multimedia applications. Second, SEG designed and conducted a scientific effectiveness study of the BrainPOP’s science movies. A quasi-experimental design was used to investigate program effectiveness. More than 50 classrooms in several schools participated in the study, with half of the classes using BrainPOP, and the remaining half delivering instruction without the benefit of BrainPOP. Analysis of covariance was used to adjust for potential differences between the groups. The reading and science skills and affect toward school of both groups were measured at the beginning of the school year and at the conclusion of the school year. The proficiency level of the two groups was compared statistically to determine if students using BrainPOP showed greater improvement. The effectiveness of the Program was described in a written report for use by BrainPOP and school’s considering the Program.

Case Study: Integrating Technolgy in a Small, Rural School

Client: 

Atomic Learning

The Challenge: 

When faced with this challenge, Seiling selected Atomic Learning as a central component for its technology professional development. The online delivery provided an easy-to-use, cost-effective solution for helping teachers learn technology.

The Solution: 

When faced with this challenge, Seiling selected Atomic Learning as a central component for its technology professional development. The online delivery provided an easy-to-use, cost-effective solution for helping teachers learn technology.
Marquita Seifried provides a good illustration of how Seiling has benefitted from the adoption of the Atomic Learning online technology training solution. When faced with a technology challenge, Marquita runs to Atomic Learning. “I use it a lot if there is a technology tool I need to learn”, said Ms. Seifried. “I particularly like the Web 2.0 tutorials. I have used a wide range of units and modules.” Marquita is confident that Atomic Learning has helped improve student learning. “I believe that the students whose teachers use Atomic Learning do better academically than students whose teachers do not take advantage of the Atomic Learning tools."

Case Study - Providing Technology Integration Professional Development for a Suburban School District

Client: 

Atomic Learning

The Challenge: 

Independent School District 197 has always prided itself on providing students with a sound, comprehensive education that enriches their lives and prepares them for their future. But like most schools, ISD 197 faces a significant technology challenge. ISD 197 recognizes that teachers are the key to developing 21st century skills in students. Therefore, they must have access to up-to-date technology and the appropriate training and technical support to take advantage of it.

The Solution: 

Margaret Hinke has been helping ISD 197 address this challenge. She is a Technology Integration Specialist with ISD 197. Margaret has been with the district for about 5 years, but provided training in the airline industry for 30 years before coming to the district. Margaret holds a BA in Psychology and a Masters degree in Instructional Design. Margaret sees the importance of technology training and how Atomic Learning can help fulfill this need."Atomic Learning provides an important bridge to ISD 197 teachers helping them get the knowledge and skills they need to reach 21st century learners." -- Margaret Hinke.

A Study of the Effectiveness of Imagine Learning on Student Reading Achievement

Client: 

Imagine Learning

The Challenge: 

Do students in grades 2 through 5 who use Imagine Learning show greater gains on the Scantron Performance Series Reading Assessments than comparable students in the same district who do not use Imagine Learning?

The Solution: 

The design of this study included a treatment and control group with pre and post testing. The treatment group included students in grades 2 through 5 in six schools within a large California school district who used Imagine Learning between the pre and posttest. The control group included students also in grades 2 through 5 in six comparable schools within the district that did not use Imagine Learning. Students in grades 2 through 5 who used Imagine Learning between December 2012 and June 2013 showed greater gains in reading ability than students who did not use Imagine Learning.

A Study of the Effectiveness of BrainPOP, a Web-Based Animated Instructional Tool Multimedia Learning Application

Client: 

BrainPOP

The Challenge: 

Do students using BrainPOP as part of their instruction show larger gains in Language, Reading Comprehension, Vocabulary and Science skills than a comparable group of students who do not use BrainPOP? Research has shown that the brain processes information using two channels: visual and auditory. Does this multimodal processing capability, as presented by the BrainPOP instructional tool enhance student learning?

The Solution: 

Using a quasi-experimental, pre-post design, this study compared the growth in Language, Reading Comprehension, and Vocabulary skills between students in classes using BrainPOP (Treatment Group) and a comparable group of students in classes that did not use BrainPOP (Control Group). Student growth in Language, Reading comprehension, Vocabulary and Science was measured by comparing scores from the Stanford 10 Achievement Test™, Abbreviated Battery (SAT 10), at the beginning of the second semester of the school year and end of the 2008-2009 school year. The study examined differences in growth among the total group of students and between students of different gender and ethnic backgrounds as well as whether or not they received free or reduced lunch.

A Study of ELL Students Using a Post Hoc, Pre-Post, Treatment Group Only Design

Client: 

Cengage Learning

The Challenge: 

The client had an instructional intervention program designed to provide intervention for English Language Learners (ELL) and Struggling Readers and asked for proof of effectiveness --Do ELL students show significant gains in Reading and Language skills after receiving instruction using Milestones?

The Solution: 

SEG conducted a study of the effectiveness of Milestones, an instructional program designed to provide intervention for English Language Learners (ELL) and Struggling Readers, examining the growth in Reading and Language skills for students enrolled in a class of 6th grade students in a Developmental Language Arts through ESOL Program in Florida. These students received instruction using Milestones (Treatment) and their Reading and Language proficiency growth was measured at the end of the 2007-2008 school year and then again after receiving instruction at the end of the 2008-2009 school year. Student Reading and Language skills were measured using the CELLA test required in Florida.

Conduct research examining the use and effectiveness of an augmented-reality based science application

Client: 

SKT

The Challenge: 

Science education is playing a critical role in maintaining US competitiveness in the world marketplace. There is a growing recognition that the teaching of science can be enhanced through technology. When SKT developed its new augmented reality enhanced science textbooks, they looked to SEG to help understand how the product was being used in the classroom and how effective the technology was in teaching science.

The Solution: 

SEG conducted a project to help SKT develop the prototype technology and conduct the research to explore how it would be used in the classroom and whether or not it was effective. First, SEG helped SKT develop the product and deploying it in the schools. Second, SEG designed and conducted a scientific effectiveness study of SKT’s augmented reality product. A quasi-experimental design was used to investigate program effectiveness. Classrooms in four states participated in the study, with some the classes using the SKT product and others delivering instruction with just the book component or no supplementary materials at all. The science skills of the groups were measured at the beginning of the study and at the conclusion of study. The proficiency level of the two groups was compared statistically to determine if students using SKT showed greater improvement. The effectiveness of the Program was described in a written report for use by SKT.

Conduct Scientifically-Based Research: ELL Product Effectiveness

Client: 

Hampton Brown Publishers

The Challenge: 

The population of students whose primary language is not English is increasing rapidly and the demand for effective English Language Learner (ELL) instruction has grown. SEG’s Client, a publisher of supplemental curriculum for use with ELL students, believed their instructional program was effective and wanted to ensure that this claim is supported by scientifically-based research. As a leading provider of educational research services, SEG was asked to design and implement a scientific study of program effectiveness

The Solution: 

SEG designed and conducted a scientific effectiveness study of the Client’s ELL instructional program. An experimental design was used to investigate program effectiveness. Several schools participated in the study with half of the classes involved using the Publisher’s instructional program and the remaining half relying on traditional instructional practice. Both groups were measured at the beginning of the school year and were assessed again at the conclusion of the school year. The English language skills of both groups were compared statistically to determine if students in the Publisher’s program showed greater improvement. The findings were documented in a written report for use by the Publisher and by school’s considering the program.