In order for assessment tools to be considered consistent with the requirements for psychometric characteristics, they must have sufficient reliability and validity indicators. In this article, we will describe the battery validity of our tests.
Validity is the ability of method to measure exactly what it was created to measure. In order not to get confused in terms, we will take shooting as an analogy. The reliability of the method can be compared with the heap of the results, and the validity can be compared with the accuracy of the hit. As in the case of reliability, the assessment of test validity doesn't boil down to only one way. There are several aspects of validity and, accordingly, several ways to measure it: from subjective to accurate, mathematically sound.
Construct validity is the answer to the question of whether our test really measures exactly what is stated and what we expect from it. For example, using an intelligence test, we ask ourselves the question: does this test really measure intelligence? Or maybe it measures erudition? Or maybe it measures only one intelligence aspect - the ability to perform mathematical operations? If you apply this question to personality questionnaires, it will sound like this: do we really measure exactly the factors that we want to measure? To answer this question, we will resort to the use of some procedures that helps to obtain an answer about the level of construct validity.
Most commonly researchers use independently created tests which measure the same characteristics as the main test. After respondents complete two tests, it remains only to calculate the correlation between the indicators of the main test and the alternative test.
Another way is to test a group that stands out among the others by the parameter you need. For example, accountants are good at counting, architects have a well-developed abstract thinking, etc. If your test shows a significant difference between this group and other groups, then it really measures targeted trait.
Another way is to assess the severity of the measured trait in a certain group of people with the help of an expert assessment, and then let the people, which being evaluated, to fill out a questionnaire. If the opinions of experts coincided with the results of your test, then you can say that it has a high construct validity.
This is, perhaps, the most important indicator of the test effectiveness within business environment.
Criterion validity measurement allows you to answer the question of how much the test results correlate with the work efficiency. Note that low indicators of criterion validity can indicate both the low quality of the methods and the absence of relation between measured abilities and work efficiency. Correlation study is aimed to obtain reliable information regarding criterion validity indicators.
Employees of the company can be selected as an object of study; in this case, their tests results are compared with the performance indicators of their work. This kind of validity is called competitive validity.
Construct Validity of GREEN Battery «Interpretation of information»
To test the constructive (theoretical) validity, ONTARGET company conducted a study, which was aimed to compare the results of the “Interpretation of information” battery tests with the results of Psytech (British company) tests, adapted for Russian-speaking respondents. The study was conducted in 2013.
Table 1. Correlation of tests from battery "Interpretation of information" with tests of British company Psytech
Construct Validity of the SAPPHIRE Battery «Analysis of information»
One of validity indicators of test battery is the correlation coefficient between the results of each test. Despite the fact that the results of verbal and numerical tests from “Analysis of information” battery are significantly related (which follows from the assumption of presence of general intelligence factor), correlation indicates that tests measure fundamentally different abilities. The study was conducted in 2015.
Table 2. Internal correlations between ONTARGET battery tests.
Criteria Validity of RED Battery «Understanding of information»
In order to validate the tests, a study was conducted on the correlation of test results with the level of behavioral competencies development. Test results were compared with competency ratings obtained from assessment centers and development centers conducted by DeTech (Development Technologies Ltd.). The total sample of the study was more than 160 managers of various levels. The study was conducted in 2015.
Since different competency models were used in different centers, all competencies were combined into several large clusters. The following correlations were obtained (only correlation coefficients that are significant at 0.05 level are shown):
Table 3. Correlation of tests “Understanding of information” with the competencies of assessment and development centers
The results show the presence of correlations with varying degrees of severity between the test results and a number of competencies. It follows, that the numerical test is mostly related to the ability to analyze and solve problems, and the verbal test is related to commercial thinking.
Criteria Validity of SCARLET Battery «Administration»
One of validity indicators of test battery is the correlation coefficient between the results of each test. Despite the fact that the results of verbal and numerical tests from “Administration” battery are significantly related (which follows from the assumption of presence of general intelligence factor), correlation indicates that tests measure fundamentally different abilities. The study was conducted in 2015.
Table 4. Internal correlations between battery tests (Understanding Instructions and Working with Numeric Information)
Construct Validity of SCARLET Battery «Administration»
To verify the construct (theoretical) validity, ONTARGET company conducted a study, which was aimed to compare the results of the “Administration” battery tests with the results of Psytech (British company) tests, adapted for Russian-speaking respondents. The study was conducted in 2013.
Table 5. Correlation of tests from battery "Administration" with tests of British company Psytech