Realizations

Survey on Educational Achievement – 
Republic of Congo

The Republic of Congo wishes to have a permanent team of national agents in the field of assessment of student achievement based on standardized tests. Consulting work including participation of Micro Centre Pythagore took place in years 2012 and 2013; these activities enabled the preparation and administration of a survey on learning achievement among a representative sample of grade 5 students.

The mandate entrusted to the consultant:

• Establish a permanent monitoring system of school learning.

• Collect data from representative samples of schools according to 5th grade elementary students.

• Ensure the sustainability of the Evaluation of Educational Achievement by the accompanying officials of the activity by experts in learning evaluation and data analysis.

The following services were provided to carry out the mandate:

• Training in evaluation and data processing for concerned personnel

• Support the development of data collection instruments

• Testing of the evaluation mechanism

• Training the national leaders in data entry and quality control

• Data processing and execution of quality control of these activities

• Realization of different statistical compilations used in the preparation of evaluation reports.

First, there has been development of tables of specification for evaluated disciplines. When a team engages in an operation of such assessment of student achievement, the team responsible for the activity must develop what is often called a document on specification. In the case of such evaluation, a key element of such a notebook is preparing the tables specification. They present an accurate synthesis or programs of the concerned evaluation and, for purposes of summative evaluation. It's like a portrait of the concerned curricula. Participants in the training mission were able to familiarize themselves with the technical aspects of tables of specification by developing those for French and Mathematics programs at grade 5 in Congo. It is a rigorous exercise that often takes time.

The fundamental task that had to make the participants linked to the work was to carry out the instrumentation to collect information in order to conduct the evaluation. The reference to such a task is the tables of specification.

A test usually consists of a set of items which are tasks that the student must meet or achieve. Participants were familiarized with instructions to take into consideration in order to have items formulated according to established conventions. Previously, they were made aware of a reality more and more present in the field of standardized tests, namely the joint grouping of items based on a theme also called scenario or evaluation situation or even center of interest. The team chose to work on this basis. The developed tests were conducted with respect to scenarios.

Work teams were formed to prepare Mathematics and French tests. Items banks were prepared. The EduStat software was used to computerize these banks.

The instrumentation required to conduct the Evaluation of Educational Achievement must also include a number of questionnaires. These allow to collectadditional information made in connection with the students' answers to tests. Participants were invited to work on three types of questionnaires: a questionnaire to the student, a questionnaire to their teacher and another to the principal.

There has been administering of the assessment in relation to the selected sample. The administration of the instrumentation is often entrusted to third parties. In the case of the realized sample evaluation in Congo, administering tests and questionnaires was organized by an organization made up of a coordination team, supervision team, regional officials, and technical advisors. They had the responsibility of leading all survey operations.

Tests and questionnaires were administered to schools part of the selected sample. Twenty pupils were selected in each school. The administrator, assisted by an observer who had the mission to monitor the progress of the operation in class and, if necessary, make adjustments.

Subsequently, there has been processing of the collected data. Initially, it was the coding of data and marking of response to items. In preparation for this, the training of team members was made in particular to practice the marking within recommended codification. Indeed, on testing, it is often possible that a test includes items other than multiple choice items as a "partial credit". In this case, the answers were corrected taking into account the instructions provided and in order to ensure the rigor of the operation. In the case of our evaluation, student responses contained in the test booklets were transcribed on coding sheets prepared for this purpose to facilitate the processing of data by an optical reader. The same method has been retained to handle questionnaires. Indeed, it can happen that certain information must be coded prior to analysis with a treatment program. Such codification is needed especially when the instrumentation includes questions to answer or constructed elaborate answers to questions.

Several data capture techniques can be used to capture the data collected using a test or a questionnaire. Sometimes responsible for assessment or investigation entrusted to a specialized firm with the task of data entry. In this case, it must specify the entry rules attached to the instruments used. It is also possible to use coding sheets that can be "read" by an optical reader. This is a procedure that minimizes errors due to typing and that accelerates the process. It is this technique that was used by the national evaluation team for this investigation.

After entering the information collected using a test or questionnaire, data files were prepared for use by a computer program in order to make statistical compilations. The compilation of the data collected is a very important step in conducting an evaluation. The results are used to confirm or refute the assumptions at the start of the process. But it is important to choose the statistical analysis techniques adapted to the operation to the needs and data type. The EduStat software was selected to perform all the statistical tables in the evaluation report and the technical report.

Email address: info@mcpythagore.com