Today there are many corporations and professional organizations that offer certification programs. Many companies and people in various industries seek these certifications.
Certification programs allow an entity to market expertise that certified individuals deliver when you interact with them in their professional capacities. For example, knowing that a medical professional is certified indicates that the individual has shown the expertise covered by the exam.
Prior to the advent of certification exams, there wasn’t a standard set of criteria to judge a service provider’s expertise. Word-of-mouth and direct experience with individuals was essentially how such decisions were made. Fortunately, there are now many different certification programs in place. For example, the Healthcare Quality Certification Board (HQCB) began publishing exams for healthcare professions in 1984. The Certified Professional Healthcare Quality (CPHQ) certification has met with great success.
“Since the first examination was administered by the HQCB in 1984, more than 15,000 professionals from a wide variety of educational and employment backgrounds have registered for the CPHQ examination, with more than 11,000 achieving certified status. There are currently over 6,800 active CPHQs in the United States and worldwide.” http://www.cphq.org/
Certification programs also are advantageous to the organizations that offer them. The time and money invested is most often recouped at a profit. Not only are these organizations able to charge for the exams, they are also able to develop and deliver the instruction related to the exam topics. All parties benefit from certification programs when they are done right. Furthermore, if your enterprise has competitors who offer similar products or services, you can improve the chances that your organization’s products and services will be selected if users are certified in how to use your offerings.
However, not all certification programs are the same, some are better than others. A certification exam should show that an individual is able to perform certain activities. Those activities should be those needed to solve a problem or address an opportunity likely to be encountered on-the-job. However, on occasion, questions on certification exams ask for some or even one of the steps, rather than testing for knowledge needed to solve the problem or take advantage of the opportunity.
Sometimes exams contain trick questions. Questions, which are intentionally written to confuse the test taker are not an effective evaluation of the test takers ability to do the activity. Trick questions only test the ability to take a test or to parse a sentence. This is a particularly poor design for non-native English speakers. It is important to keep the purpose of a certification exam in mind when developing or accepting items for inclusion on an exam. Certification exams certify that those who pass the exam can perform the tasks that are defined as being covered by the exam.
Test items should cover content an individual needs to know and do on the job. Sometimes items that have nothing to do with performing the role might also be tested. For example if the individual needs to drive a car from point A to point B asking questions about color of the interior of the car would not be relevant to the needed end result. However, questions about the rules of the road would be appropriate.
Generally, certification tests are developed with enough questions to populate several forms of the test. Test developers who are concerned with developing a sufficient number of test items sometimes write questions which test minor facts or information that normally would be looked up on the job. These types of questions are unfair to all parties. Test takers who are knowledgeable in the needed areas might not pass a poorly constructed exam that contains questions that cover small components of an overall process or activity. For example, one doesn’t need to test on the location of the gas tank on a specific car. Rather than testing to see if someone has memorized the location, it would be better to ask if they know about an arrow on the fuel gauge which points to the location of the gas tank.
My perspective is that a certification exam needs to be designed to evaluate the test takers ability to perform certain end results. Test questions on a small piece of the overall process are often easier to develop but don’t necessarily test the ability to do what is needed.
Just like other technical fields, there are procedures and best practices for the development of test questions. There are also conventions that lead to the development of items that are both valid and reliable.
Reliability and Validity
Measurement experts (and many educators) believe that every measurement device should possess certain qualities. Perhaps the two most common technical concepts in measurement are reliability and validity. Any kind of assessment, … must be developed in a way that gives the assessor accurate information about the performance of the individual. At one extreme, we wouldn’t have an individual paint a picture if we wanted to assess writing skills.
A. Reliability: Definition
• The degree of consistency between two measures of the same thing. (Mehrens and Lehman, 1987).
• The measure of how stable, dependable, trustworthy, and consistent a test is in measuring the same thing each time (Worthen et al., 1993)
For example, if we wish to measure a person’s weight, we would hope that the scale would register the same measure each time the person stepped on the scale.
…
B. Validity
1. Definition:
• Truthfulness: Does the test measure what it purports to measure? the extent to which certain inferences can be made from test scores or other measurement. (Mehrens and Lehman, 1987)
• The degree to which they accomplish the purpose for which they are being used. (Worthen et al., 1993)
For a test to be valid, or truthful, it must first be reliable. If we cannot even get a bathroom scale to give us a consistent weight measure, we certainly cannot expect it to be accurate. Note, however, that a measure might be consistent (reliable) but not accurate (valid). A scale may record weights as two pounds too heavy each time. In other words, reliability is a necessary but insufficient condition for validity. (Neither validity nor reliability is an either/or dichotomy; there are degrees of each.)
Most good certification exams begin with a blueprint. The blueprint defines the topics and sub-topics that are likely to be included on the exam. In some cases, a blueprint will be developed by a group of subject matter experts who are familiar with the knowledge and skills needed to perform the job. In some cases these are the experts who will also develop the test items. In other cases, a different group of subject matter experts will develop the questions. Blueprint development can also be based on the learning objectives from course offerings that cover exam topics. Once a blueprint is developed, it should be put out for review by subject matter experts. Most often these experts are performing the job roles and they are knowledgeable about what is needed to successfully carry out the needed activities.
The blueprint areas are usually weighted for importance and an associated number of items to be written are assigned to the topics to be covered. Publishing the blueprint and associated sample test items is usually done so that the candidates can prepare for the exam. Sample items for well-constructed exams are of the same difficulty and quality level as items the exam candidate might expect to find on the exam.
While processes vary and item development might be in workshop format, or submission by subject matter experts or another process, the end result is a certification exam that will allow individuals who pass the exam to present themselves as having the expertise that the exam certified.
Certification exams are developed and taken so that there is a standard. People who have passed the exam can say they have met this standard. It allows for selection of providers who have a credential that informs the “buyer” that they have engaged a professional for a role in which they have shown their expertise by passing an exam. An exam that has been put in place to test many people who want to establish they have expertise in a specific area.
Certifications are valuable and worthwhile. But a certification is only as good as the exam that produced it. Avoid these common pit falls and stay focused on the goal: an exam that tests the skills, behavior and knowledge needed to carry out the tasks of the role. In the next installment, I will discuss recommendations for updating and revising certification exams.
Blueprints for Learning 