Realizations

Students’ Evaluation of Ontario primary schools
in language and in mathematics

Education Quality and Accountability Office (EQAO) received as mandate to heighten the level of responsibility and contribute to stress the quality of the education in Ontario. To get this, it leads evaluations and inquiries based on objective, reliable and relevant data and it spreads at the appropriate time to the public the obtained information and recommendations to improve the educational system. And so the tests in reading, writings and mathematics, administered to the students of Grades 3 and of 6, are based on the expectations and the contents of programs in reading, in writing and in mathematics, established in the curriculum of Ontario. These tests get individual and global data on the performance of the students. Every pupil who makes the test in reading, writing and mathematics receives an individual report. The EQEO also publishes results for the province, the school councils and the schools.

Between 1996 and 1998, Micro Centre Pythagore Inc. acted as sub-contracting party for the firm National Computer System NCS) for the implementation of this system of evaluation and for the administration of the first cycle of this program of evaluation. Micro Centre Pythagore assured the realization of certain technical works. We remind briefly the tasks then confided to this Canadian firm.

Field testing of the mechanism of evaluation

During year 1996, there was field testing of this new program of evaluation in Ontario. Micro Centre Pythagore were associated to all the steps of this experiment. First, there was examination of the instrumentation prepared to realize these evaluations in Grades 3 and 6. Three subjects were aimed (reading, writing and mathematics); the tests had to be available in both official languages. These tests were administered to students' samples being a part of schools chosen by Micro Centre Pythagore to participate in this field test.

There were four sessions of marking touching the available classes and the linguistic groups. Micro Centre Pythagore assured the constitution of databases from files supplied by NCS following the optical character reading of coding forms prepared for that purpose. Afterward, there were statistical compilations of the available data. The obtained results were presented to the persons in charge of the EQAO. Information sessions allowed the staff concerned to understand better reports supplied to make the best decisions for the continuation of the operations. To realize the required works, it was necessary to adapt computing tools accordingly.

Throughout the progress of the sessions of marking, the staff of Micro Centre Pythagore acted as observer of the process to exercise a quality control of the operation and suggest improvements to introduce for the official administration. Several control procedures of the quality of the marking were organized: the group marking, the double marking and the preparation of profiles of marking for the various markers.

Micro Centre Pythagore prepared a varied set of statistical reports to communicate the quality of the experimented instrumentation. The persons in charge of the Office used this information to prepare the final version of the tests that will be administered next year. The information collected during the field testing also allowed to plan the sessions of marking to be organized during the administration of this evaluation with all the Grade 3 students.

Administration at Grade 3

Micro Centre Pythagore was entrusted several technical tasks within the framework of the data processing collected during the first administration of the tests for Grade 3 students. There were two centres of marking, one in Toronto for the English copies and the other one in Ottawa for the French copies; Micro Centre Pythagore had to plan two teams of staff specialized to be present in both centres. More than 120 000 English-speaking students and 6 000 French speakers were evaluated. We grouped together the markers (more than 1 200) in teams. Besides the results of the marking of the tests, it was necessary to treat the answers supplied by the students, the teachers, the principals and the parents to questionnaires prepared. Various control procedures of the quality were planned: group marking, multiple marking (the same copy marked on a daily base), and analytical correction.

Micro Centre Pythagore supplied, on a regular base, statistics reporting results of the marking in the course of progress of the session of marking. There was marking of group twice a day (pupil's same copy being marked by all the markers); Micro Centre Pythagore made available every day the results at these exercises with regard to all the markers, with regard to every team of marking and for every marker. Furthermore, it was possible to report the performance of the students evaluated in three disciplines on a daily base. Finally, it was possible to draw up the profile of marking of every marker to detect abnormalities which can affect the results attributed to the students.

Following the holding of the session of marking, Micro Centre Pythagore had to prepare the database containing the available information. There was fusion of information resulting from different sources: the results of the marking for three tests, information collected by means of the questionnaire to the pupil, those resulting from questionnaires administered to the teachers, to the principals  and to the parents. Several activities of quality control were then driven.

Micro Centre Pythagores produced several reports from the available databases. However, the works of analysis had to be realized to assure the preparation of the individual reports for students. Micro Centre Pythagore prepared databases allowing NCS to produce these reports intended to be quickly distributed after the administration of the evaluation.

Administration at Grade 6

This evaluation took place with samples of English-speaking students and French speakers. Micro Centre Pythagore selected schools that must participate in this activity. Mandates confided to Micro Centre Pythagore with regard to the data processing were similar to the works realized for the Grade 3. It is however necessary to note that, for this evaluation, there was no production of individual report for every student. However, all the control mechanisms of the quality retained for the evaluation of Grade 3 were applied for this operation. Micro Centre Pythagore thus saw to realize the preparation of databases, quality control of the available data and treatment of the information. Numerous statistical reports were produced within the framework of this activity of evaluation.

Email address: info@mcpythagore.com