Data Processing

Use of IT tools

Usually, several computer programs are used to ensure the achievement of technical work taking place in the context of a learning evaluation activity. Some software have been developed to perform specific tasks at each stage of development of an evaluation project. However, it must be able to ensure the links between these different tools in order to accomplish the planned work while respecting the criteria of rigor and diligence in their accomplishment. The choice of these programs must often be based on different phases of evaluation activity. However, regardless of the selected IT tools by the officials concerned, we often provide training or development activities affecting both the technical aspects of software operation and the choice as well as meaning of statistical indices adapted to the situation.

We briefly recall some technical works to be done by indicating the types of necessary IT tools. For illustration, we present EduStat software options that can facilitate the achievement of any planned activity. The following will be addressed:

• preparing the design;

• sample selection;

• preparation of instruments, that is to say the tests and questionnaires;

• quality control;

• preparation and use of databases;

• execution of statistical compilations;

• conducting statistical analysis;

• preparation of reports.

A.   Preparing design

First, it is important to define the specifications of the assessment, that is to say on what will be concerned the planned activity. There are several techniques to clearly define the area to be examined. It may be mentioned here preparing "specification tables" that guide designers to select the necessary tools in order to collect information (especially the tests and questionnaires). In addition to know the "field" to analyze, we must be able to establish the necessary links with the IT tools that will be used to ensure the realization of future technical operations. This is a very important planning step for selecting the software which includes all the parameters necessary to carry out necessary activities.

Thus EduStat software includes options to make the link between the estimate of the desired evaluation and data processing that will be necessary to realize the data collection. This involves activities related to the preparation of the database used to perform the various required operations affecting both data entry, quality control and processing of information. Thus the preparation of the database prior to transactions with EduStat provides managers a tool to make this more rigorous. This database already contain all the technical information relating to the planned evaluation: the format of the data collected, the identification of valid values, answers marking procedures, the attachment to the table of specification, among others.

B.   Selecting sample

Before the official administration of the evaluation mechanism, we must select schools and students participating in the evaluation of learning (mainly in the case of an "investigation"). Such a study is usually carried out on samples that must meet the standards laid down in order to make the "inference" statistics from the collected data, that is to say the application to all population the characteristics and results obtained using the selected sample. Several software can be used to accomplish this important task.

The data required for this activity are already present in the computer systems of departmental management responsible for maintaining the inventory of several features of the education system: school enrollment, staff, facilities, financial resources for example. Computer systems have usually been set up to maintain the information on the school system. Sometimes IT tools already available in the market are used; it may also be the presence of systems specifically developed to meet the administrative needs of the Department of Education. In all cases, we make sure to extract information required to be used as part of the selected sampling procedure. Ordinarily, creating files will be promoted in a "spreadsheet" (Excel, for example) to establish links between departmental databases and software that will be used to run the assessment activity.

Some software can accomplish the technical tasks related to the selection of schools, students or staff that can account join as part of the evaluation process. EduStat software includes options for establishing a sampling plan respecting the characteristics of the current evaluation and the completion of various tasks related to school selection. Thus, this software includes a menu which allows to select a sample conform to the most commonly using. First, we prepare a database that will be used for further operations. The sample selection is performed taking into account the choices made in relation to the proposed plans. Some options to produce lists, reports and files that can help managers to better manage the project. Finally, EduStat performs the calculation of participation rates and weights if the sampling plan chosen provides.

C.   Preparation of instruments

The instruments used in the framework of evaluation of learning must meet high quality standards. Usually, we will make the preparation with computer tools for word processing and preparation of graphic illustrations. There are in the market such software. It is also necessary that employees involved in these tasks have the skills to use these computer tools.

It may be necessary to provide the constitution of items banks for the preparation of tests that will be required to achieve the planned evaluations. Some specialized software can be used to develop such information banks. The chosen tool should be able to integrate computer files created during the preparation of the assessment instruments already available; we have to avoid having to do again the input of the information that is already available.

EduStat software has options for the development of items banks from databases on tests or questionnaires. Other options assist the staff in the management and use of a set of items banks or questionnaires in order to prepare quality instruments.

D.   Quality control

Throughout the course of an evaluation activity, it is necessary to have recourse to quality control mechanisms. The information collected and processed as part of this audit on the quality of both the administration of the procedure and its processing must be subject to various controls to ensure rigor and validity. It will be necessary to retain the necessary IT tools to accomplish these important tasks. Sources of information may be varied; it must be ensured to establish links between information that may come from different computer systems.

EduStat software includes many options that can assist managers throughout the course of an evaluation activity: marking the responses of those assessed, data entry, review metric qualities of assessment instruments, among others. EduStat offers several strategies to help managers in monitoring the quality of data files: frequency distributions, validation and comparison of a file containing a sample of records and reference file. Other options allow to control the quality of the marking the answers provided by the students as part of a test.

E.   Data entry

After preparing the data collection instruments, it is prudent to provide immediately the data entry mechanism that will be collected. Most data processors provide the information input options. We must select compatible technology according to computer systems already available in the administrative department in charge of the operation. As with other activities taking place in the course of learning evaluation, mechanism must be provided for the integration of the data entry to the operations taking place before the constitution of the data files and those that follow the entry of information.

EduStat software includes options to ensure data entry with the features of the developed instrument. We can choose from several modalities the best options to conduct evaluation based on available resources. It may be mentioned here at least three input options that provide, in varying degrees, control on this: i) single entry without validation, that is to say the input data without the intervention of the software to check the validity of the provided information; ii) single validated entry, that is to say the input data with software intervention when information provided does not respect the values defined as "valid"; iii) double entry, that is to say the input data by two different operators or even operation at two different times.

It will be important that files created at the time of data entry are easily usable by software that will be used for the processing of information. Several different software may be needed to perform the various compilations taking place in a learning evaluation mechanism.

F.   Preparation of databases

After entering the collected data, we have to create the databases necessary for the implementation of relevant statistical compilations. Several tasks must be performed to ensure the availability of databases that meet the peculiarities of the current activity. It may be noted here the extraction of information, merging databases or data files, creating files in relation to various formats. Again, there must be the use of flexible IT tools adapted to the specifics of the evaluation mechanism in progress.

EduStat software includes a series of options to achieve the most common activities associated to an evaluation activity. Several other programs can be very helpful to achieve very specific tasks.

G.   Execution of statistical compilations

When databases are available, it is possible to undertake a series of statistical compilations tailored to the gathered information. The information in the databases usually comes from different sources: i) the assessed student; ii) the teachers of these students; iii) the directors of selected schools;iv) the quality control mechanism (administration of the assessment and marking of responses). It is desirable to establish links between these different sources of information. It is also necessary to retain analytical techniques adapted to each data set.

It is possible that several IT tools may be used to achieve a specific task either the particularities of the selected plan. It is prudent to choose the software adapted to the possibilities of linking the information processed by each system. For example, it may be noted that the EduStat software includes compilations options affecting particularly the assessment of school learning and their relation to information from various sources (school context, information on students, teachers and directors, opinions or attitudes of respondents, etc.).

It is also necessary to perform analyzes to decide on the metric quality of selected instruments (items analysis, analysis of differential item functioning, for example). Specialized software can be used to accomplish this prior work analyzes before other compilations. Thereafter, we will carry out various compilations to get the necessary information to conduct the evaluation of school learning and favorable conditions for student achievement.

H.   Realization of statistical analysis

Computer programs used to perform various useful deemed statistical compilations provide staff involved in the evaluation several sets of statistical indices. We must make the interpretation in order to analyse the observed trends. Concerned staff must have a good understanding of the different coefficients calculated following the compilations realized. The meaning of these statistical indices will be presented and explained to the public of the assessment reports. We will also be able to provide clear explanations to the public affected by the results obtained.

I.    Additional work

It is often necessary to undertake the execution of additional works to respond to questions raised at the time of the analysis of available information. EduStat includes options (utilities) that may be useful to managers of research since many tasks are realized with personal computers. We can mention: sorting records, merging files or retrieving records, recodification of the values of a variable, identification of records having a same identifier. When additional statistical analyzis are needed, the manager may wish to use specialized software. He can then use EduStat to prepare the files needed to use the selected software.

J.    Preparation of evaluation reports

We must quickly produce the report or reports which will present the results and the collected information. It is important to provide the results of compilations people who have the mandate to draw up these reports. Usually there is the production of one or more technical reports that will be used as an information source or directly by the report writers. The processors of the selected information should produce statistical tables easily integrated in a report. Like many statistical software, EduStat generates reports in different formats that can be directly integrated into software like Word or Excel.

It may be noted that statistical reports produced by EduStat allow communication of results to interested staff. EduStat also produces graphs that can be easily integrated into a report. It is possible to produce such reports in different formats that can be operated using specialized software in desktop publishing and graphic illustrations. 

Email address: info@mcpythagore.com