Validity studies of the deep questionnaire
Validity is one of the key factors demonstrating the effectiveness of a test or questionnaire. Validity measures the compliance of the results of a test or questionnaire with assigned tasks. This article presents data from recent studies (2018-2019). The psychometric parameters of the scales of the questionnaire comply with the criteria of the “Russian Standard for Personnel Testing”.
In these studies, we measured the criterion validity, which reflects how the results of the questionnaire of the employees of companies — our clients — correspond to their performance indicators, the ratings of managers, and other employee success parameters used by companies. We present below data confirming the correlation of the scores in DEEP questionnaire scales with the key performance indicators of employees in different positions. A positive correlation means that the higher the score on a particular scale, the higher the employee’s performance indicator; a negative correlation reflects the inverse relationship.
To calculate the correlations, we divided the employees being appraised into two groups: “the best” employees — with the highest performance indicators compared to the majority, “the worst” employees — with the lowest indicators. The group of “average” employees was not used in the calculations.
Based on the values of the correlation coefficients given in the article, it is easy to determine the criterion validity indicator — this is the module of the correlation coefficient of the performance indicators with individual scales.
Research 1. Large Russian bank, 2018
Here we rated the managers of additional offices. A total of 119 people were appraised. The personal appraisal from management was considered a performance indicator. As there were no numerical performance indicators, to calculate the correlations, we coded the individuals classified among the group of “the best” employees with number 1 and among the group of “the worst” employees with number 2.
Table 1. Correlation of the performance indicator with the DEEP questionnaire scales (managers of additional offices)
Significance level = 0.05
This table shows that the “best” employees are the best leaders (these people are more dominant, socially confident, structured), which corresponds to the presented position. The personality traits presented above help the managers of additional offices be “the best” among their colleagues.
Research 2. Russian Bank, 2019.
In this study we rated 37 underwriter employees whose responsibilities include the approval of loan applications, with due account of the risks for the bank. In this case the performance indicators were based on the correspondence of the number of approved loan applications with the target number and the number of defaults on the repayment of the loans. The “best” group included people with both high scores, and the “worst” group — with both low scores.
Table 2. Correlation of the performance indicator with the DEEP questionnaire scales (underwriters)
Significance level = 0.05.
Based on the scores in the table above, it can be concluded that use of the DEEP personality questionnaire enables a company to identify the people who will be successful as underwriters.
Study 3. Federal Distributed Retail Company, 2018.
We rated 547 retailers and 89 sales office managers, whose performance was appraised based on KPIs. As in the previous two cases, two groups of people were appraised: the “best” employees — with high KPIs and the “worst” employees — with low KPIs.
Table 3. Correlation of the performance indicator with the DEEP questionnaire scales (retailers)
These traits enable employees to achieve success in active sales and also in the attainment of the target set for the sales office.
Table 4. Correlation of the performance indicator with the DEEP questionnaire scales (heads of sales offices)
It is essential that office managers manage processes and people (function of the Dominance trait), implement the plan and instructions of senior management (function of the Obligation trait) and to be enough open for innovation and change (catalyzed by the low Conservatism trait).