The Use Of Assessments Across Borders
In the current global environment in talent assessment, we are confronted with the question: can I use these tests to assess specific aspects across people in different countries or from different cultures?
MHS is a digital company with worldwide reach and our assessments are ready to be administered wherever you can have a connection to the Internet, or at least access to an electronic device. Thus, it begs the question, can this test that is being administered in a Chicago head office provide valid reliable results when I am assessing, for example, emerging leaders in New Delhi, India or Johannesburg, South Africa? Beyond the array of translations and international (local and global) norms that we produce, we also advise to use caution in the interpretation of a test result given the context of the testee.
The Importance of Language
Matsumoto and Juang1 point out that “linguistic equivalence alone does not guarantee measurement equivalence” (p.47). That is, even when the words may have an exact translation or if they are etymologically equivalent, they may have different meanings. Words in the same language may have different meanings across geography or time. That is the reason we are constantly updating our criteria with inclusive language to foresee potential difficulties in adapting a test.
Administering Assessments Across Borders
According to Greenfield2, there are three aspects that are needed for a test to “travel freely” across cultures, between, or even within societies.
- Values and Meaning: An agreement on the merit or value of responses. Items should mean the same thing across cultures.
- The unit of evaluation3: Can you evaluate the construct by assessing the individual exclusively? Or is the construct something that is anchored in the society that the person belongs to? Your unit of evaluation in most tests is the individual. Notable exclusion: those that attempt to evaluate organizational climate or other group level traits.
- Clarity in communication: The goal of the questions needs to be understood universally. Would there be an issue with the format and context of questioning? Does the test administrator’s organizational level or social strata affect the responses of the testee?
As a coach, test administrator, manager, or decision maker, it is imperative that specific nuances of language and culture be considered when using an instrument validated for a specific population. Our norming procedures are designed to create a robust instrument for the North American market. By following U.S. Census criteria, we have captured the diversity inherent to that country. Individuals of multiple ethnicities, education levels, and job statuses are part of our reference samples.
We work with our consulting and training partners worldwide to provide assessments and solutions that will be adapted to the context where they are implemented. Not only are we constantly creating translations as well as creating norms or evaluating benchmarks, but we develop our instruments with a global vision.
We want you to know you can take our portfolio of products and services comfortably across the world.
Check out MHS’ Talent Assessments and Solutions to learn more about how to further develop and manage talent in the workplace. Our range of Talent Development assessment tools are ideal for use in coaching, organization and leadership development, selection, and succession planning.
Have questions? Get in touch with us.
References
1 Matsumoto, D., & Juang, L. (2016). Culture and psychology. Cengage Learning: Boston, MA
2 Greenfield, P. M. (1997). You can’t take it with you: Why ability assessments don’t cross cultures. American Psychologist, 52, 1115-1124. Her work relates to cognitive ability testing, but it applies to most psychological testing.
3 Greenfield refers to this aspect as “Knowing” as her work is in evaluation of abilities.
