Assessment Items

Items Banks

Several officials of the learning assessment administration think that it is useful and even necessary to develop "items banks" that will subsequently be used to prepare the instruments necessary to conduct an evaluation. The hardware part of such banks must meet high quality criteria and have been ranked relative to categories to make easy choices and adapted to encountered situations.

EduStat software includes options for creating and managing items banks. There are specialized software on the market for the creation and operation of items banks. Some software can even generate tests administered directly with the use of a computer.

EduStat software includes options to create items banks, to save relevant information in databases and use these databases to create tests. It is also possible to produce banks of questions that will be used to develop complementary questionnaires or opinion questionnaires.

Several persons may be associated to the creation, updating and using of items banks. While one person may perform all work associated to the items banks, it is possible to identify three categories of staff usually realizing some specific jobs. Here are summarily presented categories of stakeholders that may be associated with activities related to the preparation of items banks and their use.

A.  Preliminary work could be done by specialists or regular users of EduStat software. These specialists will be specifically responsible for the creation of EduStat databases as well as the integration of the items in the database.

B.  The inclusion of items and questions in Word files whose format meets the requirements of EduStat software should be entrusted to persons with knowledge of this word processing software.

C.  The creation of items and questions should be entrusted to specialists in evaluation of learning or achievement of complementary questionnaires or surveys. It is these people who will use the items banks according to options provided in EduStat. Here are the steps that should be retained for the creation and use of items banks or questions.

Step 1 – Prepare questions

First, we must create the items or questions that meet the design associated to an already available table of specification. In addition to the text of the question, we would have written, if deemed appropriate, administrative instructions and marking instructions. We will also need to indicate for each question, its attachment to the table of specification and possibly a difficulty index defined a priori by the specialist in measurement and evaluation.

Step 2 – Create the database

A limited number of items or questions on the same subject should be grouped in order to create an EduStat database. Some fields of information provided in such a database should be detailed to allow an optimal use of this information when preparing tests or questionnaires. This is in particular the following parameters of the database that will form the future banks of items:

• identification of the variable – This is the item number.

• description of the items or questions – Usually each variable is documented with a descriptive text; this text will be copied (but not printed in the tests produced thereafter) in the documents included in each  items bank.

• variable type – We have to indicate the type of variable: multiple-choice item, constructed responses items, text or illustration, survey question.

• registration of valid values – This information is primarily used if it is a constructed-response item.

• inclusion of the answer key – This information is primarily useful if it is a multiple-choice item.

• indication of already used – If appropriate, this information will reveal the items or matters already used.

• exclusion – It is possible to indicate that a variable should not be used during a given period.

• difficulty index – It is helpful to include a code that will inform on the difficulty of the item. The selected code will be available when creating a test.

• link with table of specification  – tests should be developed from a table of specification. EduStat software can manage up to three sets of links that is to say up to design three different ways to group items. This information will be displayed next to each item when creating a test.

• language – When create a database, we have to indicate the language of items in the case of a multilingual bank.

Step 3 – Input questions

EduStat software includes options to create items banks, to save relevant information in databases and use these bases to create tests. Operations can record questions and their integration into the database. After creating a database in order to develop items banks, we have to incorporate the texts belonging to each question in the database. Two approaches can be used:

• use of parameter editor;

• creating files for adding text in Word documents for their integration into the items bank.

Step 4 – Use the items banks

At the end of the process described above, all information linked to an items bank is included in an EduStat database. Here are the steps to follow to create a test from one or more items banks. This approach consists of four sets of activities:

• Select the items banks or questions banks;

• Select items or questions;

• Finalize the structure of the data collection instrument;

• Create the test or questionnaire.

A.    Select the items banks

As noted above, each item belonging to the selected banks is connected to the original database. Information on items (attachment to specification table, difficulty index) are saved in an EduStat database. Therefore, we must indicate the relevant databases in order to have the list of items belonging to this or these banks and the various features that are assigned to them.

By pressing the "Statistics" button, a report showing the link of items to the specification table is produced in relation to the format chosen (Word document or an Excel file).

By pressing the "Save Bank", there is production of an Excel file containing the following information on each item of active banks:

• Identification of the item;

• The connection to the first grouping in the specification table;

• The connection to the second grouping in the specification table;

• The connection to the third grouping in the specification table;

• The difficulty index.

B.    Choose items or questions

After selecting database connected to items banks, the list of available items is displayed in the main window. Information on each item can be found in different columns provided for that purpose. To select an item to be included in the test, we must click on the line number of the desired item. It is possible to view the item or add the item to the test in preparation. If we choose the option to add the item to the test, the identification of the item is displayed in the bottom window and it is no longer possible to select this item for the test preparation.

It is also possible to select the items that are part of the instrument in preparation by choosing the button "automatic selection". A table containing the number of items per scale is displayed. We must indicate the number of items to choose from; the software will randomly select items.

C.    Finalize the structure of the data collection instrument

A series of buttons to the right of the window with the structure of the test preparation can execute various operations. Thus it is possible to change the order of the questions in the test. It is also possible to save the structure of the test or questionnaire; this Excel file can later be used to continue operations, in particular the creation of a test.

D.    Create the test or questionnaire

We first must identify the file containing the test to produce. After pressing the button "Create the instrument" the program saves the test in an RTF file. Beforehand, we have to indicate the sections of the test to include in the document. The software offers the following choices:

• Student booklet;

• Administrative guide;

• Scoring guide. 

Email address: info@mcpythagore.com