Realizations

School Achievement Indicators Program

Between 1993 and 2005, the Council of Ministers of the Education, Canada (CMEC) administered the School Achievement Indicators Program (SAIP), a cyclic mechanism of pan Canadian evaluation concerning the achievement of the students in mathematics, in reading and writing and in science. For a long time, Ministers for Education recognize that the achievement of the students as regards courses of study constitute generally a valid indicator of the efficiency of an educational system. That is why ministries of Education take part for two decades in a range of studies on the achievement of the students. On the international plan, by the company of the CMEC as well as in an individual way, provinces and territories of Canada collaborate in diverse evaluative activities, some those of the Organization for Economic Cooperation and Development (OECD), the international Evaluation of the school achievement and the international Association for the evaluation of the school achievement (IEA). With the aim of studying the performance of the students of Canada and reporting it, the CMEC created in 1989 the School Achievement Indicators Program. In a protocol of agreement signed in December, 1991, the Ministers agreed to estimate the performance of the students of 13 and of 16 year-old in reading and writing and in mathematics. In September 1993, they decided to add it the evaluation of the achievement in sciences. The information collected thanks to the evaluations of the SAIP allows provinces and territories to establish their priorities in education and to plan improvements in their programs.

Several activities take place within the framework of this process of evaluation of the achievement of the students. One has sharing of the responsibilities between the Secretarial department of the CMEC, the staff of the various ministries of Education and the consortium formed by representatives there resulting from provinces and territories of Canada. Technical works must be realized to assure the processing of information and the statistical analyses. A public report and a technical report are published following every evaluation.

Micro Centre Pythagore Inc. provided support in evaluation of learning to the Council of Ministers of the education, Canada since the introduction of this pan Canadian program of evaluation. He assisted the coordinator of the project in the realization of certain technical activities (choice of the types of statistical compilations to be retained, selection of the samples of schools that must participate in the operation, the examination of the instruments used to the pursued objectives, the support for the writing of the reports of evaluation).

Micro Centre Pythagore realized certain technical works within the framework of SAIP evaluations. We make a brief description below.

The softwares developed by Micro Centre Pythagore were used in order to complete several activities taking place in this evaluation. And so the samples of schools that must participate in the SAIP were chosen by means of EduStat software. At the time of the field testing of the instrumentation, the data were often captured by means of computing tools developed by Micro Centre Pythagore. The data collected within the framework of these evaluations were treated by means of EduStat in the purpose to exercise the quality control of the compilations which can be realized by the co-workers of the CMEC. Micro Centre Pythagore played an active role in doing several technical works.

Micro Centre Pythagore saw entrusting the realization of works of data analysis collected by means of tests containing multiple-choice items. Items analyses bringing in the Items Response Theory were executed for the evaluation of the mathematics and the science. Also the results were examined to identify the potential sources on the bias with regard to the one or the other group of evaluated students. To achieve these tasks, university researchers were associated and Informative programs necessary for the realization of the works were developed.

SAIP set up several activities on quality control touching the available data, the marking of the administered tests and the statistical compilations. Micro Centre Pythagore proposed some of these mechanisms and assured the statistical analyses of the available data. We can enumerate some of the mechanisms set up to verify the coherence of the data and the rigor of the adopted procedures. There was usually double marking of a sample of students' copies as well as marking by all the markers of a sample of students' same copies. To be able of realizing these operations of quality control, computing tools adapted to the peculiarities of the SAIP were developed.

Micro Centre Pythagore saw confiding the responsibility for compiling the data collected in certain evaluations of the SAIP for the preparation of the public report. Statistical tables were produced as well as graphic illustrations. In the driving of these works, it was necessary to respect the design appropriate for this activity of evaluation: attribution of the levels of performance, calculation of the confidence intervals, the attribution of weights taking into account the size of every estimated population.

There was realization of several works within the framework of the preparation of the technical reports published by the CMEC. These compilations mainly touched the quality control of the available data and the reliability of the marking of the works realized by the evaluated students. There were other analyses touching questionnaires administered to students, to teachers and to principals. And so distributions of frequencies were produced to report answers supplied by the contacted persons. There were also links established between the answers to these contextual questions and the performance on the students.

Finally, the whole available information and produced reports was integrated into banks of information to assure the filing of the data. The access to this information is facilitated by the development of an interface of easy communication of use.

Let us note that the last SAIP evaluation was administered in 2004. Another program, the PCAP (Pan-Canadian Assessment Program) took over. During the year 2005-2006, the Council of Ministers of the Education, Canada (CMEC) decided to bring modifications to the SAIP program to integrate better the evaluations of the learnings into the other evaluations realized by the Canadian jurisdictions, among others the PISA. This program of evaluation leans on the objectives, the purposes and the results of the curricula of participating Canadian authorities. It also reflects the conclusions of the researches and the exemplary practices in the field of the various disciplines taught in the high schools of Canada.

During the year 2006-2007, students' samples were selected in reading (major part of the evaluation), in mathematics and in science (these two disciplines form the minor components). Micro Centre Pythagore was associated with various technical works, in particular the choice of samples and the processing of data collected during the field testing of the instruments of evaluation. This firm also saw entrusting works connected with the preparation of the rules of data capture collected in the schools associated to this evaluation. Micro Centre Pythagore exercised a quality control of the statistical compilations which were realized by a firm of data processing.

Email address: info@mcpythagore.com