Examining the effects of Phono-Graphix on the remediation of reading skills of students with disabilities: a program evaluation. Education & Treatment of Children - May 1, 2007
Stacy A. Endress, Holly Weston, Nancy E. Marchand-Martella, Ronald C. Martella, Julie Simmons
Abstract
The purpose of this program evaluation was to examine the effects of Phono-Graphix, an intensive tertiary-level reading intervention, on the basic reading skills of 16 students aged 6 to 17 years who received special education services. Instruction was delivered one-on-one or in small groups of two or three students. Instruction lasted 8 weeks. Students were pre- and posttested using the Woodcock-Johnson Psycho-Educational Battery-3rd Edition: Tests of Achievement (WJ-III ACH). Results of this program evaluation indicated that the reading skills of students with disabilities improved. Statistically significant gains were made by the elementary and upper elementary/secondary students. The effect size gains were all above .39. Results are discussed with regard to effective interventions for students with reading disabilities and implications for future research.
**********
In today's society, reading is a fundamental and necessary skill. Although reading is considered by many to be the most important skill to be mastered in education (National Institute of Child Health and Human Development [NICHD], 2000; Snow, Burns, & Griffin, 1998), it is also the area of academics in which children are most likely to encounter difficulties (Meese, 2001; Shapiro, 1996). In the United States, 80% of all children identified as learning disabled in schools struggle primarily with reading (Foorman, Fletcher, & Francis, 1997; Lyon, S. Shaywitz, & B. Shaywitz, 2003). Reading deficits are found in children from early elementary through junior high and high school. However, there are few programs that address reading skill deficits across a wide range of age groups.
Unfortunately, children who fail to learn such basic reading skills during the first 3 years of elementary school rarely catch up to their same age peers (Torgesen, 2002). Once a poor reading performance pattern is established, it is likely to continue on into higher grades without effective remediation (McCardle & Chhabra, 2004). If reading interventions are delayed until 9 years of age, approximately 75% of these children will continue to struggle with reading throughout high school and into adulthood (Lyon, 1997; Torgesen, 1998). Ironically, 90% of the children identified with reading disabilities struggle with the same basic reading skills involved in decoding at the single word level (Foorman et al., 1997). Decoding words centers on phonemic awareness. Phonemic awareness can be defined as the awareness that words, which are made up of letters, consist of sounds. Although this awareness is not critical for speaking and understanding others speak, it is crucial in developing reading and spelling skills in an alphabetic language. Phonemic awareness helps children learn to read and spell because it provides a direct link between the sounds and symbols that comprise the alphabetic code. For example, Kochnower, Richardson, and DiBenedetto (1983) found that when students with reading disabilities were matched on IQ with typical readers they demonstrated a distinct difference in their decoding skills. It seems that the students with reading disabilities were not as able to break and utilize the alphabetic code. Evidence indicates that the connection between sounds and symbols may not develop without intervention for some students and may be the most difficult component of reading instruction to master for those later labeled with a learning disability (Ball & Blachman, 1991). However, training in phonemic awareness can improve reading achievement. Effective interventions in phonemic awareness development have been demonstrated under a wide variety of conditions with a wide variety of learners including those with disabilities. For example, Lovett and Steinbach (1997) found that students with severe reading disabilities made gains in skills such as phonemic awareness when delivered 35 hours of direct, remedial instruction.
Along with phonemic awareness, research shows that blending and segmenting phonemes is particularly important in developing phonemic recognition skills (NICHD, 2000). Blending involves combining individual sounds, while segmenting requires sounds be addressed individually. Students identified with a reading disability have demonstrated significant differences in their skills in segmenting the sounds in words and blending sounds into words (Lovett & Steinbach, 1997; Stanovich, 1988). In fact, the lack of these core speech/sound skills has been considered a potential origin for many instances of reading disabilities (Lovett & Steinbach, 1997).
Some may argue that if such primary skill deficiencies are not addressed within a certain time span the individual may never make significant gains. However, it seems that these core deficits are responsive to direct and intensive intervention in a broad range of ages (Byrne, Fielding-Barnsley, & Ashley, 2000; Gersten, Keating, & Becker, 1988; Kochnower et al., 1983; Lovett & Steinbach, 1997). Though it is vital to intervene early when reading problems present in young students, it is just as important to recognize that it is never too late for older students to gain skills. Research does not indicate that there is any particular age in which reading skills level off and cannot continue to grow (Bruck & Treiman, 1992; Foorman, Francis, Fletcher, & Lynn, 1996). This information is relevant in light of the dearth of services for students with disabilities at the middle and high school level.
Explicit instruction has been shown to be efficacious in addressing core skill deficits such as letter-sound correspondence and blending and segmenting in a variety of age groups (Adams, 1990; Ball & Blachman, 1991; Lovett & Steinbach, 1997; NICHD, 2000; Snider, 1995). Reports from both the National Research Council (Snow et al., 1998) and the National Reading Panel (NICHD, 2000) emphasize that reading instruction in phonemic awareness and phonics strategies are best taught using explicit instructional techniques (Torgesen, 2002). In other words, the method in which reading instruction is delivered is as important as content. Explicit instruction is defined as "a systematic method of teaching with emphasis on processing in small steps, checking for student understanding, and achieving active and successful participation by all students" (Rosenshine, 1986, p. 34). Explicit instruction is designed with the underlying premise that all students can learn when clear and unambiguous teaching methods and materials are provided (Mercer & Mercer, 1993). Teachers model the specific tasks that students are expected to do, give opportunities to practice with teacher feedback, and then provide opportunities for students to apply the new skills independently over time. This instructional technique has been found to be effective when remediating reading deficits in students with learning disabilities and has been shown to be more efficient than unstructured or indirect methods (Somerville & Leach, 1988; Vaughn & Linan-Thompson, 2003). Several longitudinal studies have found explicit instruction to be effective in increasing rates of learning for students with disabilities, particularly in the area of reading (Gersten et al., 1988; Somerville & Leach, 1988). Since children with basic skill deficits in reading acquire skills more slowly than other children, teachers must do more in less time. The most efficient way for children to acquire basic reading skills is to be taught through explicit instruction (Armbruster, Lehr, & Osborn, 2003; NICHD, 2000; Vaughn & Linan-Thompson, 2003).
One reading intervention that incorporates components for effective reading instruction with an explicit instructional approach is Phono-Graphix (C. McGuinness & G. McGuinness, 1998). This program utilizes explicit instruction techniques and addresses the core skill areas of phonemic awareness, blending, segmenting, and letter-sound correspondence. This program is designed for use with children age 4 through adult non-readers. Phono-Graphix is a linguistically-based program that builds upon the primacy of sound knowledge in the process of reading development (C. McGuinness & G. McGuinness). Letter forms are taught in conjunction with sounds but not in terms of letter names. The rationale for this is that the letters do not make sounds, they represent sounds. Therefore, letter names are referred to in the program as "sound pictures" (C. McGuinness & G. McGuinness). Program lessons progress systematically and are structured to build on increasing mastery of skills. Initial lessons utilize explicit instruction to teach the basic alphabetic code (17 consonants and 5 vowels) through phonemic awareness, segmenting, and blending. Letter-sound correspondence is taught through phonics instruction that is direct, explicit, and always within the context of a word. After this "basic code" is mastered, lessons progress through the remaining alphabetic code, consonant digraphs, spelling alternatives, and multisyllabic words.
The developers of the Phono-Graphix program discuss numerous studies they have conducted that demonstrate program efficacy as a reading intervention; however, only one such study has been published. The results of that study indicated that considerable gains could be made through the use of the Phono-Graphix method (C. McGuinness, D. McGuinness, & G. McGuinness, 1996). In this study, the authors used a quasi-experimental design with a pretest/posttest comparison and no control group. Participants included 87 children aged 6 to 16, who had reading difficulties. Of these students, 35 had been previously diagnosed with a learning disability. They received 1 hr per week of one-on-one instruction for a total of 12 hrs of intervention. In addition, supplementary homework assignments were completed with parents. Results from the study reported overall average gains on the WJ-III subtests dependent measures of 13.7 standard score points in Word Identification and 19.34 points on Word Attack. These results suggest that Phono-Graphix may be an effective program in remediating reading skill deficits. However, given the paucity of empirical evidence to support this instructional program, further investigation seems appropriate.
The purpose of this program evaluation was to examine the effectiveness of the Phono-Graphix program as an intensive reading intervention. Of particular interest was how effective this program would be with a wide range of student age groups in a rural public school setting.
Method
Participants and Setting
Students who participated in this program evaluation attended school in a small, rural public school district in Washington State. The district included a total of 193 students in grades K through 12; 80% of the student population qualified for free or reduced price meals. Of the total student population, 14% qualified for or received special education services. Sixteen students who participated in this program evaluation were identified as having reading goals in their current special education service program and lagged one or more grade levels behind in reading. Students were divided into an elementary group (grades K-4) and an upper elementary/secondary group (grades 5-12). Students in the elementary group (n = 7) ranged in age from 6 years, 6 months to 10 years, 11 months (average age = 9 year, 0 months). Their disability categories included specific learning disability (n = 4) and developmentally delayed (n = 3). This group included 2 females and 5 males. Students in the upper elementary/secondary group (n = 9) ranged in age from 11 years, 10 months to 17 years, 6 months (average age = 13 years, 2 months). Their disability categories included specific learning disability (n = 7) and other health impairment (n = 2). This group included 3 females and 6 males.
The district implemented a three-tier intervention model to prevent and remediate reading disabilities. This three-tier model consisted of primary, secondary, and tertiary levels of instruction (Vaughn & Linan-Thompson, 2003). At both the primary and secondary levels students received progressively more intensive reading instruction within the classroom. At the tertiary level, special education services were provided and students participated in an intensive reading intervention (Phono-Graphix, described next). Students who received this intervention were placed in small groups (one-on-one or two to three students) based on skill level as measured by Woodcock- Johnson Psycho-Educational Battery-Third Edition: Tests of Achievement (WJ-III ACH) (Woodcock, McGrew, & Mather, 2001) pretest scores. The subtests administered included Letter-Word Identification, Passage Comprehension, Word Attack, and Comprehension (described under Dependent Variables and Measures). Students received instruction from a special education teacher with 20 years of experience, two educational assistants both with 2 years of experience, and a special education student teacher.
Curriculum and Materials
The district selected Phono-Graphix (C. McGuinness et al., 1996) as the tertiary intervention given its inclusion on Washington State's adoption list for scientifically-based reading interventions. This program emphasized phonemic awareness, decoding skills through segmenting, blending and sound manipulation, and alphabetic code knowledge. Phono-Graphix emphasizes an explicit instruction model whereby the instructor initially modeled the desired skill, the instructor and students practiced the skill together, and the students practiced the skill independently. Error correction within the program was direct, immediate, and paired with explicit feedback from the instructor. This program consists of 40 lessons and was delivered over an 8-week period. Instruction is divided into four different levels. These levels included the following:
Level I. The focus is teaching the "Basic Code" (common sounds represented by only one letter) through segmenting (arrange letters to form three-letter words spoken by instructor), mapping (instructor says sound and student writes the corresponding letter), segmenting/blending (instructor says word and student segments sounds and then blends sounds to form word), sound manipulation (instructor reads real or nonsense words and student arranges tiles to form words), and story reading (pre-taught words are incorporated into stories read orally by student). The "Basic Code" consists of 17 consonants and 5 vowels. The focus is on words that follow the consonant-vowel-consonant (CVC) pattern.
Level II. Instruction in the "Basic Code" continues with remaining consonants and consonant digraphs. The concept of "sound pictures," the program's approach to sound-to-print correspondence, is taught rather than letter names. Activities as this level include the same activities as in Level I. Additional activities include a game (sound bingo game based on the sounds being taught that combines the skills of phonemic awareness and code knowledge), increased story reading using more complex text, and spelling practice (sounds, words).
Level III. The "Advanced Code" is taught with remaining consonant digraphs and phonographs. Instruction includes mapping (student says sounds and writes them for various spellings of targeted sounds), sound sorting (sounds are printed on cards and student sorts them based on commonalities including shared letters/sounds), word analysis (student underlines multi-letter sounds in words), sound search (student searches for pre-taught sounds in stories), scratch-sheet spelling (student spells words using various spelling options and is then asked to ascertain which spelling is correct for these words), and oral reading of decodable text.
Level IV. Instruction includes a focus on "Multi-Syllabic Words" up to five syllables with multi-syllable analysis (student uses letter tiles to show word chunks presented orally by the instructor), controlled reading (student reads word chunks on worksheets as well as connected text), and process spelling (student reads word, says how many syllables are in word, underlines important sound spellings, and spells word from memory once it is removed).
Dependent Variables and Measurement
Two dependent variables were measured in this investigation: student reading skills (measured by the WJ-III ACH) and instructor perceptions of the program's effects (measured by a post-program survey).
WJ-III ACH. All students were assessed before (within 10 school days) and after (within 5 school days) the implementation of the program using the WJ-III ACH. The WJ-III ACH is an individually administered standardized test of achievement. The test is composed of 14 subtests constructed to assess academic performance levels in areas of reading, mathematics, written language, and general knowledge. The WJ-III ACH has two forms (A and B) that were used as the pre- and posttest measures, respectively. Five subtests were administered in this program evaluation: Letter-Word Identification, Reading Fluency, Passage Comprehension, Word Attack, and Reading Vocabulary. These subtests comprise three reading clusters: Broad Reading, Basic Reading, and Comprehension. The standard score produced by each participant on each of these clusters was used as the dependent measure.
The Broad Reading cluster composite score is comprised of the following three subtests: Letter-Word Identification, Reading Fluency, and Passage Comprehension. In the Letter-Word Identification subtest items range from requiring participants to identify single letters to reading words. In the Reading Fluency subtest items are timed. Participants' reading speed of simple statements of four or five words and their ability to determine the accuracy of the statement are assessed. In the Passage Comprehension subtest reading for understanding is assessed. Initial test items involve matching a rebus to an actual picture of an object. Test items progress in difficulty from multiple-choice options of short phrases that represent a picture to questions requiring a keyword be added to a passage for it to make sense.
The Basic Reading cluster composite score is comprised of two subtests: Letter-Word Identification and Word Attack. The Word Attack subtest measures sound-to-print recognition (phonics) and word structure analysis. Test items include creating a sound for specific letters and reading nonsense words aloud that are phonetically consistent with regular patterns in the English language.
The Reading Comprehension cluster composite score is comprised of two subtests: Passage Comprehension and Reading Vocabulary. The Reading Vocabulary subtest assesses word reading and understanding vocabulary by producing appropriate definitions. Synonyms, antonyms, and word analogies are addressed.
Social validation survey. Instructors involved in the implementation of the Phono-Graphix program completed a post-program survey. The survey used a Likert scale from 1 (strongly disagree) to 5 (strongly agree). The survey was comprised of four questions pertaining to the impact of the program on the students' reading performance.
Experimental Design
A pre-experimental design comprised of one group with a pretest and posttest (Martella, Nelson, & Marchand-Martella, 1999) was used to evaluate the effects of an 8-week implementation of Phono-Graphix. Data were grouped for evaluative purposes based on elementary (ages 6-10 years) or upper elementary/secondary (11-17 years) standing.
Procedures
Instructor training. The special education teacher, educational assistants, and special-education student teacher involved in administering the intervention were trained to use Phono-Graphix. The training was conducted by a certified Phono-Graphix trainer with 6 years of experience in using the program. The training course consisted of 35 hrs of instruction over 5 days.
During the first day of training, trainees received a structural blueprint of instruction based on the nature of the code and the cognitive skills required to read and spell using the code. The concept of pictures of sounds, the skills of segmenting, blending and phoneme manipulation, and the code of 143 "sound pictures" (letters) that Phono-Graphix uses was introduced. Trainees practiced activities that they would later be able to use in their intervention groups and discussed how to integrate these concepts into literature the students would use for free reading. The second day of training focused on mastering the "Basic Code." Presentation of the interactive lessons was conducted in the morning. Practice with the interactive lessons and question and answer time comprised the afternoon session. The third day of training focused on the "Advanced Code." The morning session focused on a presentation of the interactive lessons while the afternoon was dedicated to practicing with the materials. The fourth day of training focused on "Multi-Syllabic Words" with a presentation of techniques and materials in the morning and practice in the afternoon. The fifth day of training focused on the diagnostic components of Phono-Graphix. In-program tests were presented. Standardized reading tests, error pattern analysis, as well as case management and moderating the flow of instruction and/or remediation were discussed. An exam was provided to the instructors at the end of the session. This exam included 41 multiple-choice, short answer, and essay questions addressing aspects of the week-long training program. The purpose of the exam was to demonstrate competency in both clinical and classroom teaching of the principles and practices taught in the course. This exam was scored and kept by the Phono-Graphix authors.
Phono-Graphix Intervention
Students from both groups received 8 weeks of Phono-Graphix instruction. Instruction consisted of 2-hr blocks and was delivered Monday through Friday. The same instructors always worked with the same students throughout the program; again, instruction was delivered one-on-one or in small groups (two to three students). The same types of activities were used throughout the 8-week intervention period. Error correction techniques were direct, immediate, and paired with explicit feedback from the instructor. Specific praise was used when a student performed well or corrected a previous error. Students progressed through four levels of the Phono-Graphix program (see Curriculum and Materials section for details on instructional activities). Activities remained the same whether instruction was provided one-on-one or in small groups (e.g., the instructor took turns with individual students when round-robin reading was conducted or served as a "student" when choral reading was required).
Mid-way through the program students were allowed to choose a story from Harcourt materials to read at the end of each session. A choral round robin format was used with each student and the instructor in turn taking a paragraph to read.
At least once per week during the 8-week intervention, each instructor was observed conducting the program by the reading specialist and/or study authors for feedback purposes. Observers provided oral feedback on how the sessions were conducted following the session.
Data Analysis
The performance of the students in each group was evaluated both prior to and following the Phono-Graphix program intervention. To examine the change in student reading skill, the Broad Reading, Basic Reading, and Reading Comprehension clusters of the WJ-III ACH were provided before and after the intervention. The performance of students across pretest and posttest administrations of the WJ-III ACH reading clusters were then compared using a t test for correlated means. The effect size measure, standardized mean difference (SMD), was used to indicate observed differences between the pretest and posttest performance on the WJ-III ACH reading cluster subtests. Effect size values are considered independent of sample size and scale of measurement. The SMD was calculated by subtracting the pretest mean from the posttest mean and dividing the result by the pretest standard deviation.
SMD was also calculated for gains made from pre- to posttest in comparison to the WJ-III normative group sample. (Note: If the sample made equivalent gains to that of the normative group [which would be expected for typically achieving students], the standard score difference would essentially remain the same.) The SMD was calculated by subtracting the sample pretest mean from the normative group mean (i.e., 100) and dividing the result by the normative group standard deviation (i.e., 15); this calculation was repeated using the sample posttest mean. This calculation represents how divergent the sample was from the normative population and indicates the amount of progress toward the normative score that has been attained.
Results
WJ-III ACH Reading Clusters
Each student completed all four levels of the program during the 8-week intervention. Thus, comparisons were not made based on the differential progress of each student. The means, effect sizes (SMD), and standard deviations from pretest and posttest administrations of the WJ-III ACH reading cluster subtests were calculated for each group separately.
Elementary group. As shown in Table 1, the mean gain was 4.71 standard score points from pretest to posttest for the Broad Reading cluster, 4.57 standard score points for the Basic Reading cluster, and 6.14 standard score points for the Reading Comprehension cluster. To determine the statistical significance between the pretest and posttest administrations of the WJ-III ACH reading cluster tests, paired samples t tests were conducted. The t test for the Broad Reading cluster was statistically significant, t (6) = 3.89, p<.008. The SMD between pretest and posttest of the Broad Reading cluster was .43. The t test for the Basic Reading cluster was not statistically significant. However, the SMD between the pretest and posttest of the Basic Reading cluster was educationally significant at .57. The Reading Comprehension cluster t test was not statistically significant. However, the SMD for the Reading Comprehension cluster was educationally significant at .42.
In comparing the differences between students and the WJ-III normative group sample, the SMD was calculated to determine if the students' skills had improved and the gap narrowed from pretest to posttest scores when compared to the normative group. In all test clusters students narrowed the gap between their pretest scores and the normative test average. In the Broad Reading cluster, the SMD for the pretest was -1.32 indicating that the mean was at or below 91% of the normative group's scores. The SMD for the posttest was -1.01; thus, the posttest mean was at or below 84% of the normative group's scores. These data indicate that the students made a gain of 7% on the normative group.
In the Basic Reading cluster, the pretest SMD was -.83 (i.e., at or below 79% the normative group's scores), while the posttest SMD was -.53 (i.e., at or below 70% of the normative group's scores). Therefore, the students gained 9% toward the normative mean.
In the Reading Comprehension cluster, the pretest SMD was -1.24 indicating that the pretest mean was at or below 89% of the normative group's scores. The SMD for the posttest mean was -.83; therefore, the posttest mean was at or below 79% of the normative group's scores. This result demonstrates a 10% gain on the normative group.
Upper elementary/secondary group. As shown in Table 2, the mean gain was 5.56 standard score points from pre- to posttest for the Broad Reading cluster, 8.22 standard score points for the Basic Reading cluster, and 8.33 standard score points for the Reading Comprehension cluster. To determine the statistical significance of differences between the pretest and posttest administrations of the WJ-III ACH reading cluster tests, paired samples t tests were conducted. The t test for the Broad Reading cluster was statistically significant, t (8) = 2.98, p<.01. The SMD effect size between pretest and posttest of the Broad Reading cluster was .39. The t test for the Basic Reading cluster was statistically significant, t (8) = 3.79, p<.005. The SMD effect size for the Basic Reading cluster was .61. The t test for the Reading Comprehension cluster was statistically significant, t (8) = 2.52, p<.03. The SMD effect size for the Reading Comprehension cluster was .53.
In examining the differences between students and the normative group sample, the SMD was calculated to determine if the students' skills had improved and the gap narrowed from pretest to posttest scores when compared to the normative group. In all test clusters the gap narrowed between pretest scores and the normative test average. In the Broad Reading cluster the pretest SMD was -1.56 indicating that the pretest mean was at or below 94% of the normative group's scores. The posttest SMD was -1.18 indicating that the posttest mean was at or below 88% of the normative group's scores. These data indicate that the students made a gain of 6% on the normative group.
In the Basic Reading cluster the pretest SMD was -1.11 (pretest mean was at or below 86% of the normative group's scores), while the posttest SMD was -.56 (posttest mean was at or below 71% of the normative group's scores). Therefore, there was a gain of 14% on the normative group.
In the Reading Comprehension cluster the pretest SMD was -1.15 indicating that the pretest mean was at or below 87% of the normative group's scores. The posttest SMD was -.59 indicating that the posttest mean was at or below 72% of the normative group's scores. This finding demonstrates a 15% gain on the normative group.
Social Validation Survey
The survey consisted of four questions on a Likert scale from 1 (Strongly Disagree) to 5 (Strongly Agree). Of the four instructors involved in implementing the Phono-Graphix program, all responded "Strongly Agree" (Mean = 5.00) when asked if the program increased the student's reading performance. Three of the four responded "Strongly Agree" and the fourth responded "Agree" (Mean = 4.75) to the continued use of Phono-Graphix as part of the special education program and all four supported the inclusion of Phono-Graphix in the general education reading program. When asked if students demonstrated improvement in reading in content areas, one instructor responded "Strongly Agreed," two noted "Agreed," and one responded "Neutral" (Mean = 4.00).
Discussion
The purpose of this program evaluation was to examine the efficacy of an 8-week intensive reading intervention using Phono-Graphix (C. McGuinness et al., 1996). This intensive tertiary-level intervention aligns with recommendations made by the Vaughn Gross Center for Reading & Language Arts (2005) for up to 2.5 hours of focused reading intervention per day for students with the most severe reading problems. Implementation of this program was conducted in a rural public school setting in special education classrooms. The 16 participants were students, aged 6 to 17, with reading goals included in their individual education programs. Results of the WJ-III ACH Broad Reading, Basic Reading, and Reading Comprehension clusters demonstrated students made gains in all three clusters. In the elementary group a statistically significant gain was made in the Broad Reading cluster. This finding indicates that students' core skills of phonemic awareness as well as reading fluency improved. In the upper elementary/secondary group, statistically significant gains were made in all three reading clusters. The results indicated that not only did students' fundamental phonemic awareness skills of blending and segmenting improve but their overall skills in comprehending printed materials increased as well.
In examining the results it is important to consider the effect sizes for the WJ-III ACH reading clusters. Often in educational research effect sizes do not reach the levels required for statistical significance due to small sample sizes. However, Adams and Engelmann (1996) found that educational significance may be met if the performance increase is at .25 of a standard deviation. Thus, a SMD effect size of .25 or greater would be considered to have educational significance. For the elementary group SMD effect sizes exceeded the minimum level in all clusters (see Table 1). The core skill areas assessed in the Broad Reading and Basic Reading clusters reflect improvement with a .43 and a .57 standard deviation increase from pre- to posttest for the elementary group.
For the upper elementary/secondary group, results from each reading cluster showed that the SMD effect size exceeded the minimum level (see Table 2). The upper elementary/secondary group showed even greater improvement in the Broad Reading and Basic Reading clusters with a .39 and .61 standard deviation increase from pre- to posttest, respectively.
Importantly, the Reading Comprehension cluster effect size in both groups reflected skill growth. A standard deviation increase of .42 was found in the elementary group, and a standard deviation increase of .53 was found in the upper elementary/secondary group.
The results of this program evaluation seem to support Lovett and Steinbach's (1997) assertion that lack of core skills such as sound/letter correspondence may be at the heart of many reading disabilities. These results also support the findings of Byrne et al. (2000) that these core deficits are responsive to intervention in a broad range of ages. Findings also support the work of Vaughn and Linan-Thompson (2003) on explicit instructional formats as effective tools in remediating reading deficits. Gains made by participants were similar to those made in a previous study using Phono-Graphix as the intervention (C. McGuinness et al., 1996). However, overall gains were not as robust as those found in the C. McGuinness et al. study for the elementary group. Gains in the upper elementary/secondary group were closer to those found by C. McGuinness et al. It should be noted that the C. McGuinness et al. study only reported subtest gains rather than reading cluster gains. The current program evaluation found mean gains for the elementary students of 4.71 standard score points from pretest to posttest for the Broad Reading cluster (Letter-Word Identification, Reading Fluency, and Passage Comprehension subtests) in the elementary group. A gain of 4.58 standard score points was found for the Basic Reading cluster (Letter-Word Identification and Word Attack subtests). In the C. McGuinness et al. study gains on the Letter-Word Identification subtest and the Word Attack subtest were 13.7 and 19.3, respectively. In the upper elementary/secondary group the current program evaluation found gains of 8.22 points in the Basic Reading cluster. However, the C. McGuinness et al. study found standard score gains of 15.10 for Letter-Word Identification and 22.21 points for Word Attack in the 10-16 year age bracket. Regardless of the discrepancies in gains demonstrated in the two investigations, the Phono-Graphix program has been shown to remediate reading deficits at various age and grade levels. These results become more relevant given the dearth of research validated reading interventions appropriate for a wide range of students with special needs.
Though many of the gains demonstrated in this program evaluation are statistically significant, these students made gains on the WJ-III normative group in their performance of 7% to 10% for the elementary group and 6% to 15% for the upper elementary/secondary group rather than continuing to fall further behind as do most students in special education. As stated previously, given that the students in this evaluation received special education services, one would expect their standard score difference to be stable or to worsen. Just the opposite occurred. This finding is particularly important when one considers that the goal of special education is to reintroduce the student into the general education curriculum.
Although overall changes were seen, there was considerable variability among the students. For example, two students in the elementary group (students 3 and 4) had decreases in their basic reading performance while other students showed gains. In fact one student (student 5) had a 13-point improvement. Upon consulting with educational staff, it seems as if students 3 and 4 were in the process of shifting from their whole word or sight word strategy learned before the program (i.e., identifying words from memory) to a decoding or sound/letter strategy. Additionally, it is possible that known words were presented in the pretest (Form A), whereas more novel words were presented in the posttest (Form B) where the sound/letter strategy was required.
Post-intervention instructor survey results indicated that the program increased students' reading performance. Instructors also felt that the program could supplement general education reading curriculum as well as become a permanent part of the special education program. These results reflect the level of instructor support this program generated. These responses are supported by the fact that all students returned to their regular reading programs (i.e., Harcourt or Read Well [grades K-6]) or literature classes (grades 7-12).
Despite the gains made by students in this program evaluation, several limitations were present. A pre-experimental one group pretest posttest design was used. This design did not include a control group and did not use random assignment when designating students to groups. Unfortunately, with no control group as a comparison group it is impossible to be certain that the improvement demonstrated by students can be solely attributed to the intervention. However, it would be difficult to randomly assign students due to the skill grouping necessary for instruction. Future studies should incorporate a control group to increase experimental control and show that this intervention program is significant when compared to special education students not receiving the program.
Second, the environment in which this investigation was conducted coupled with the small number of students limits the ability to generalize the results to other populations and/or to other school environments. It would be beneficial to examine if this program is effective in larger schools in urban areas and with a larger number of students. Examining different disability categories independently would also be beneficial.
Third, this program evaluation lacked a fidelity measure regarding the consistency of program instruction and the utilization of a within program progress measure. However, instructors attended a week-long training on all aspects of Phono-Graphix implementation. The trainer observed instructors as they practiced various aspects of this program. Additionally, the reading specialist and study authors conducted several observations and provided feedback to the instructors based on what they saw. In the future, it would be beneficial to gather formal data on instructional fidelity to ensure that all instructors followed the same protocol throughout program implementation.
Fourth, mid-way through the program, Harcourt materials were made available to the students. The students were allowed to choose a story to read at the end of each session. Although likely minor, the addition of the Harcourt materials makes it somewhat more difficult to attribute the improved reading performance only to the Phono-Graphix program. In other words, the Harcourt materials were essentially part of the independent variable. It would be important to determine whether or not the Harcourt materials had an effect on the students' performance in future investigations.
Finally, the primary measure used in this investigation (i.e., WJ-III) may not have been sensitive to changes in reading skill for an 8-week intensive intervention program. Curriculum-based measures of oral reading fluency and/or word identification fluency should be used in future investigations. Even so, changes were seen with the use of the WJ-III.
In summary, results of this program evaluation suggested that instruction that is explicit, structured, and targets core skills such as phonemic awareness, segmenting, and blending, holds much promise in remediating reading deficits for students with disabilities of various ages.
Acknowledgment
Preparation of this manuscript was supported in part by a grant from the Office of Superintendent of Public Instruction (OSPI), Olympia, Washington. Opinions expressed do not necessarily reflect the position of OSPI, and no endorsement should be inferred.
References
Adams, M. J. (1990). Beginning to read: Thinking and learning about print. Cambridge, MA: The MIT Press.
Adams, G. L., & Engelmann, S. (1996). Research on Direct Instruction: 25 years beyond DISTAR. Seattle, WA: Educational Achievement Systems.
Armbruster, B. B., Lehr, F., & Osborn, J. (2003). Put reading first: The research building blocks for teaching children to read, kindergarten through grade 3. Jessup, MD: National Institute for Literacy.
Ball, E. W., & Blachman, B. A. (1991). Does phoneme awareness training in kindergarten make a difference in early word recognition and developmental spelling? Reading Research Quarterly, 26, 49-66.
Bruck, M., & Treiman, R. (1992). Learning to pronounce words: The limitations of analogies. Reading Research Quarterly, 53, 292-307.
Byrne, B., Fielding-Barnsley, R., & Ashley, L. (2000). Effects of preschool phoneme identity training after six years: Outcome level distinguished from rate and response. Journal of Educational Psychology, 92, 659-667.
Foorman, B. R., Fletcher, J., & Francis, D. (1997). A scientific approach to reading instruction. Retrieved June 27, 2004 from http://ldonline.org_/ld_indepth/reading/cars.html
Foorman, B. R., Francis, D. J., Fletcher, J. M., & Lynn, A. (1996). Relation of phonological and orthographic processing to early reading: Comparing two approaches to regression-based, reading-level-match designs. Educational Psychology, 88, 639-652.
Gersten, R., Keating, T., & Becker, W. (1988). The continued impact of the direct instruction model: Longitudinal studies of follow-through students. Education and Treatment of Children, 11, 318-327.
Kochnower, J., Richardson, E., & DiBenedetto, B. (1983). A comparison of the phonic decoding ability of normal and learning disabled children. Journal of Learning Disabilities, 16, 348-352.
Lovett, M. W., & Steinbach, K. A. (1997). The effectiveness of remedial programs for reading disabled children of different ages: Does the beneficial increase for older children? Learning Disabilities Quarterly, 20, 189-210.
Lyon, G. R. (1997). Congressional testimony: Report on learning disabilities research. Committee on Education and the Workforce. Washington, DC.
Lyon, G. R., Shaywitz, S. E., & Shaywitz, B. A. (2003). Defining dyslexia, comorbidity, teachers' knowledge of language and reading: A definition of dyslexia. Annals of Dyslexia, 53, 1-14.
Martella, R., Nelson, R., & Marchand-Martella, N. (1999). Research methods: Learning to become a critical research consumer. Needham Heights, MA: Allyn & Bacon.
McCardle, P., & Chhabra, V. (2004). The voice of evidence in reading research. Baltimore, MD: Paul H. Brookes.
McGuinness, C., & McGuinness, G. (1998). Reading reflex: The foolproof Phono-Graphix method for teaching your child to read. New York: Simon & Schuster.
McGuinness, C., McGuinness, D., & McGuinness, G. (1996). Phono-Graphix: A new method for remediating reading difficulties. Annals of Dyslexia, 46, 73-96.
Meese, R. L. (2001). Teaching learners with mild disabilities: Integrating research and practice (2nd ed.). Belmont, CA: Wadsworth/Thomson Learning.
Mercer, C. D., & Mercer, A. R. (1993). Teaching students with learning problems (4th ed.). New York: Merrill/Macmillan.
National Institute of Child Health and Human Development. (2000). Report of the National Reading Panel. Teaching children to read: an evidence-based assessment of the scientific research literature on reading and its implications for reading instruction: Reports of the subgroups (NIH Publication NO. 00-4754). Washington, DC: U.S. Government Printing Office.
Rosenshine, B. V. (1986). Explicit teaching and teacher training. Journal of Teacher Education, 38, 34-46.
Shapiro, E. S. (1996). Academic skills problems: Direct assessment and intervention (2nd ed.) New York: Guilford Press.
Somerville, D. E., & Leach, D. J. (1988). Direct or indirect instruction? An evaluation of three types of intervention programme for assisting students with specific reading difficulties. Educational Research, 30, 46-53.
Snider, V. E. (1995). A primer on phonemic awareness: What it is, why it's important, and how to teach it. School Psychology Review, 24, 443-455.
Snow, C. E., Burns, M. S., & Griffin, P. (1998). Preventing reading difficulties in young children. Washington, DC: National Academy Press.
Stanovich, K. E. (1988). Explaining differences between the dyslexic and the garden-variety poor reader: The phonological core variable-difference model. Journal of Learning Disabilities, 21, 590-612.
Torgesen, J. K. (1998). Catch them before they fall: Identification and assessment to prevent reading failure in young children. American Educator, 22, 32-39
Torgesen, J. K. (2002). The prevention of reading difficulties. Journal of School Psychology, 40(1), 7-26.
Vaughn, S., & Linan-Thompson, S. (2003). What is special education for students with learning disabilities? The Journal of Special Education, 37, 140-147.
Vaughn Gross Center for Reading & Language Arts. (2005). Introduction to the 3-tier reading model (3rd ed.). Austin, TX: The University of Texas System/Texas Education Agency.
Woodcock, R. W., McGrew, K. S., & Mather, N. (2001). Woodcock-Johnson III: Tests of Achievement. Itasca, IL: Riverside.
Table 1 WJ-III ACH Reading Cluster Scores for Elementary Students
Group Broad Reading
Student Size Disability (1) Age (2) Pre Post Dif
1 2 DD 6-6 72 77 5
2 3 SLD 9-1 90 91 1
3 2 DD 6-10 69 74 5
4 3 SLD 10-4 66 77 11
5 3 SLD 9-10 94 96 5
6 3 SLD 10-11 89 91 2
7 3 DD 9-8 84 88 4
Mean 9-10 80.14 84.71 4.71*
SD 10.79 8.66
Pre-Post
SMD .43
Wj-III
SMD -1.32 -1.01
Basic Reading Reading Comphrension
Student Pre Post Dif Pre Post Dif
1 77 84 7 76 76 0
2 94 98 4 92 96 4
3 82 79 -3 70 75 5
4 80 78 -2 57 78 28
5 97 110 13 100 97 -3
6 95 100 5 87 90 3
7 87 95 8 88 94 6
Mean 87.42 92.00 4.57 81.42 87.57 6.14
SD 8.01 11.98 14.64 9.18
Pre-Post
SMD .57 .42
Wj-III
SMD -.83 -.53 -1.24 -.83
(1) DD = Developmental Disability; SLD = Specific Learning Disability
(2) In years and months (e.g., 6-6 = 6 years and 6 months)
* p <.008
Note. SD = Standard deviation; Pre/Post SMD = Standardized Mean
Difference effect size for difference between pre- and posttest
assessment scores; WJ-III SMD = Standardized Mean Difference effect size
for comparison with normative group.
Table 2 WJ-III ACH Reading Cluster Scores for Upper Elementary/Secondary
Students
Group Broad Reading
Student Size Disability (1) Age (2) Pre Post Dif
1 3 SLD 11-10 86 86 0
2 1 SLD 17-6 41 60 19
3 3 sld 11-7 88 91 3
4 3 SLD 13-0 83 87 4
5 3 SLD 12-3 75 81 6
6 3 OHI 11-10 77 82 5
7 3 OHI 11-10 83 86 3
8 3 SLD 16-8 72 81 9
9 1 SLD 16-8 72 81 9
Mean 13-2 76.66 82.33 5.56*
SD 14.41 8.94
Pre-Post
SMD .39
Wj-III
SMD -.56 -1.18
Basic Reading Reading Comphrension
Student Pre Post Dif Pre Post Dif
1 95 96 1 91 92 1
2 50 72 22 46 76 30
3 94 105 11 100 106 6
4 90 96 6 93 91 -2
5 80 84 4 76 90 14
6 84 96 12 78 88 10
7 83 94 11 84 86 2
8 84 87 3 86 87 1
9 84 87 3 91 104 13
Mean 83.33 91.55 8.22** 82.77 9.11 8.33***
SD 13.52 9.44 15.69 9.16
Pre-Post
SMD .61 .53
Wj-III
SMD 11.11 -.565 -1.15 -.59
(1) SLD = Specific Learning Disability; OHI = Other Health Impairment;
(2) In years and months (e.g., 11-10 = 11 years and 10 months)
*p<.01, **p<.005, ***p<.03
Note. SD = Standard deviation; Pre/Post SMD = Standardized Mean
Difference effect size for difference between pre- and posttest
assessment scores; WJ-III SMD = Standardized Mean Difference effect size
for comparison with normative group.
Citation Details
Title: Examining the effects of Phono-Graphix on the remediation of reading skills of students with disabilities: a program evaluation.
Author: Stacy A. Endress
Publication: Education & Treatment of Children
(Magazine/Journal)
Date: May 1, 2007
Publisher: Thomson Gale
Volume: 30
Issue: 2
Page: 1(20)