Learning Design: Do it in Chunks

When learning programs are designed for the K-12 environment, the designers need to use a broad brush; they need to provide the learners with life skills in the topics they are addressing. University programs need to deliver instruction with a variety of examples. Again, the learners may need to apply the concepts and skills to a variety of different applications.

A corporate environment is different, and I believe the design and development of training needs to reflect how the learning materials are applied in a business situation.

In corporate situations most people attend or consume training to learn about what they will need to do on the job. Developers of corporate training events must identify the end results the learning event prepares people to deliver. In other words they need to identify what people will need to do on the job. Based on what the learners will need to do, learning events should be developed which tell the learner what they need to do, how they will do it and provide multiple opportunities for practice and feedback. They also need to tell the learner where they can go wrong and how to prevent or correct mistakes.

Delivering elearning via slides is counterproductive. Why use slides when the learning materials are being viewed on-line by the learner? Words, graphics, video etc. that assist learning are all excellent ways of delivering content that provides the learner with the knowledge they will need to carry out their job role responsibilities. Just be careful to ensure that the tools used provide learning rather than something nice to see but devoid of instructional relevance. The instructional materials should also include a table of contents and an index, both of which allow the learner to click and go right to the content they need.

All the instructional material should be chunked, delivering small pieces of content, examples, cautions for mistakes and practice and feedback, and a summary of what was covered. In general, each chunk of instructional content should be able to be consumed by the learner in a 5 – 7 minute period prior to an interactive exercise. Pointers to where the information was covered and also additional references should also be included with the exercises. The exercises should ask the learner to do something that they will need to do on the job. They should be asked to show their ability and to apply what they have learned. Memorization questions, unless memorization is needed on-the-job, should not be asked. A question which asks where information can be found is, however, appropriate.

There is much more that can and will be said in future posts on topics related to corporate learning events. Please comment and also let me know what else you would liked covered in these brief posts.

Blueprint for Certification, Part Two: Don’t throw the baby out with the bathwater

Many professions and fields offer certification exams. Generally speaking once someone takes the needed exams and receives certification in a given area the certification is only valid for a limited period of time. This is due to the fast pace at which things change. Almost every field of study has new information and tools added on a constant basis. Therefore as the capabilities needed to perform effectively in a field change it is necessary to update, revise and enhance the exams that certify people’s ability to perform specific roles. In today’s fast moving and fast changing environment certification exams are most often updated and revised because of product or service changes. In most cases a process similar to the development of the original exam is used to develop new versions of the exam. This is usually necessary because there are enough product changes to warrant a significant number of new exam questions. However, before discarding the old questions there is a lot of good data available which can inform the new development efforts.

Certification exam revision is best done following a structured approach. When updating an exam I recommend first identifying the items which are no longer valid due to changes in the product or service. Yes, that likely goes without saying. But, before you delete those items it may be worthwhile seeing how they performed. For those items that didn’t perform well you may be able to determine why they performed poorly and that information can be used to inform the new items that will be developed for the current update.

Even though certain questions may no longer be applicable to the new version of the product or service there is useful information available to the update effort. For example is it is likely that the existing exam contains several different types of questions on several different topics.  There may be questions that ask about functionality or features or it’s compatibility to other products. There may be simple multiple-choice questions or perhaps there is a situation presented where there are several questions that reference that scenario. A first pass may look at how the various types of questions performed. What percentage of each type of question performed well and what percentage did not perform well? Is there a wide spread between how well different types of items performed? This is important because an individual’s score should be based on their ability to perform the activities associated with the job role as opposed to their ability to parse questions or understand the language of the test.

If you are working with an exam that addresses several topics for example services or competitive products, business problems addressed by product or services or sales strategies; then questions can be grouped by type and topic and reviewed for how well they performed. They can also be reviewed using individual aspects such as a specific service or product.

Statistical analyses can be carried out and you can use the resulting data to make decisions. However, what I want to address here is an analysis of how well the items performed based on the goals of the exam. For me the key questions to be answered are, first what did you want the people who receive certification to be able to do. This information is in the exam blueprint.

Reviewing how well test takers performed on each segment of the exam will provide information on how well the people who passed the exam are able to deliver to those areas covered by the exam. The reputation of the certification program rests not only on how many people achieve the certifications but also, and I would argue, more importantly, on how well the people who have the certification credentials are able to perform against the exam objectives.

The second question for me is how well test takers performed relative to section weights. On the heavily weighted sections is the performance evidenced at the level needed? Did people who received certification pass each section? If not, are there people who received certification but did not pass a highly weighted section. Did they receive the certification because they performed well on questions from the less relevant sections. If there are many people in this category consider if you need to change the section weights and have more questions address content from the key sections.

The purpose of a certification exam is to deliver the same questions to a wide audience of people within the field. Some may not be native speakers of the language of the exam. The item should assess whether the test taker has the knowledge needed to carry out the activity rather than the ability to understand complex language. Certification exams are designed to determine if the test taker can do those activities for which the certification is being awarded. Other tools or activities should be used to determine an individual’s ability to interact and comprehend a client’s communications.

I have heard people make a case for testing an individual’s ability to determine the problem or opportunity in more of a “real world” situation where a customer or client provides a great deal more information or perhaps incomplete information. My belief is that there is nothing wrong with testing for this competency, however, I believe these competency should be tested with another test, one which is designed to test comprehension, language, questioning etc. separate from a technical, product or industry specific certification instrument. Essentially if both the technical and comprehension factors are tested in the same item then one doesn’t know where remediation is needed. One also doesn’t know how to revise the item. Therefore, I recommend only testing for a single aspect in each item.

In looking at the items that didn’t perform well, consider if the language used in the stem (the questions itself) was simple and direct without additional information or ambiguous language. Some people may say that a complex or ambiguous question is okay.   However, I don’t recommend using confusing or complex stems in certification exams for the reasons cited above.

Next I recommend looking at the items that are still valid. Determine how they are performing. Are revisions indicated for some or all of the response choices?  Below are some steps to take before deleting items that are not performing effectively but cover content that still needs to be included on the exam.

• Show poorly performing items to experts, are they clearly phased
• Contact people performing the job role to learn the relationship of items that didn’t work to the work being done
• Do the questions that are performing poorly test real world activity
• How narrowly focused are the questions, do they address the work of the wide range of people taking the exam
• Do the poorly performing items follow recommended guidelines

See part 1 for additional exam development recommendations.

Doing the easier tasks first allows a determination of how much more needs to be done. Even with the best processes and procedures there are items that will not perform well. It is important when updating an exam to identify the cause of the poor performance of all the items that had bad results even if they are no longer relevant. A lot can be learned about how to develop new items even from items that are no longer technically relevant.

Once you have assessed the language and made any needed revisions I recommend you revise the answer options. Re-phase the correct response and develop new distractors. Test questions that are used for an extended period of time are often remembered by test takers and shared with others getting ready to take the exam. Once the total set of questions is developed it is recommended that all questions new or revised go through the same rigorous review process as was used for the initial development effort.

Summary

Exam development and exam updates are most effective when they follow a systematic process for development. When undertaking exam revision it is recommended that just as when the initial development effort was undertaken that a systems approach be employed. Identify the needed end results, validate that it is the result needed to achieve the desired outcomes, and then look at what is needed, what exists and what must be either developed or modified. It is important to keep the end result in mind as all the pieces of the development effort are carried out. As it is likely that there will be multiple exam questions developed by a group of exam writers, the test development manager will need to ensure that all aspects of the blueprint are covered by a sufficient number of questions.

Although you have more information as a result of doing an analysis of the results from the existing exam, you still need to ensure a sufficient number of items (new and revised) are developed and reviewed and approved so that items can be deleted as needed. A new practice test should also be developed. It is also advisable to have a few items available for interim updates that might be required after the exam is published.

In summary conducting a through analysis of the performance of the items on the existing exam will allow for more informed decision making as you develop the next version of the certification exam.
Systematic review and revision will lead to a credible and effective certification exam that will result in people who will effectively perform the activities defined in the exam objectives.

Blueprint for Certification, Part One: Not All Certifications Are Created Equal

Today there are many corporations and professional organizations that offer certification programs.  Many companies and people in various industries seek these certifications.

Certification programs allow an entity to market expertise that certified individuals deliver when you interact with them in their professional capacities.  For example, knowing that a medical professional is certified indicates that the individual has shown the expertise covered by the exam.

Prior to the advent of certification exams, there wasn’t a standard set of criteria to judge a service provider’s expertise.  Word-of-mouth and direct experience with individuals was essentially how such decisions were made.  Fortunately, there are now many different certification programs in place.  For example, the Healthcare Quality Certification Board (HQCB) began publishing exams for healthcare professions in 1984.  The Certified Professional Healthcare Quality (CPHQ) certification has met with great success.

“Since the first examination was administered by the HQCB in 1984, more than 15,000 professionals from a wide variety of educational and employment backgrounds have registered for the CPHQ examination, with more than 11,000 achieving certified status.  There are currently over 6,800 active CPHQs in the United States and worldwide.”  http://www.cphq.org/

Certification programs also are advantageous to the organizations that offer them.  The time and money invested is most often recouped at a profit.  Not only are these organizations able to charge for the exams, they are also able to develop and deliver the instruction related to the exam topics.  All parties benefit from certification programs when they are done right.  Furthermore, if your enterprise has competitors who offer similar products or services, you can improve the chances that your organization’s products and services will be selected if users are certified in how to use your offerings.

However, not all certification programs are the same, some are better than others.  A certification exam should show that an individual is able to perform certain activities.  Those activities should be those needed to solve a problem or address an opportunity likely to be encountered on-the-job.  However, on occasion, questions on certification exams ask for some or even one of the steps, rather than testing for knowledge needed to solve the problem or take advantage of the opportunity.

Sometimes exams contain trick questions.  Questions, which are intentionally written to confuse the test taker are not an effective evaluation of the test takers ability to do the activity. Trick questions only test the ability to take a test or to parse a sentence.  This is a particularly poor design for non-native English speakers.  It is important to keep the purpose of a certification exam in mind when developing or accepting items for inclusion on an exam.  Certification exams certify that those who pass the exam can perform the tasks that are defined as being covered by the exam.

Test items should cover content an individual needs to know and do on the job.  Sometimes items that have nothing to do with performing the role might also be tested.  For example if the individual needs to drive a car from point A to point B asking questions about color of the interior of the car would not be relevant to the needed end result.  However, questions about the rules of the road would be appropriate.

Generally, certification tests are developed with enough questions to populate several forms of the test.  Test developers who are concerned with developing a sufficient number of test items sometimes write questions which test minor facts or information that normally would be looked up on the job.  These types of questions are unfair to all parties.  Test takers who are knowledgeable in the needed areas might not pass a poorly constructed exam that contains questions that cover small components of an overall process or activity.  For example, one doesn’t need to test on the location of the gas tank on a specific car.  Rather than testing to see if someone has memorized the location, it would be better to ask if they know about an arrow on the fuel gauge which points to the location of the gas tank.

My perspective is that a certification exam needs to be designed to evaluate the test takers ability to perform certain end results.  Test questions on a small piece of the overall process are often easier to develop but don’t necessarily test the ability to do what is needed.

Just like other technical fields, there are procedures and best practices for the development of test questions.  There are also conventions that lead to the development of items that are both valid and reliable.

Reliability and Validity

Measurement experts (and many educators) believe that every measurement device should possess certain qualities. Perhaps the two most common technical concepts in measurement are reliability and validity. Any kind of assessment, … must be developed in a way that gives the assessor accurate information about the performance of the individual. At one extreme, we wouldn’t have an individual paint a picture if we wanted to assess writing skills.

A. Reliability: Definition

• The degree of consistency between two measures of the same thing. (Mehrens and Lehman, 1987).

• The measure of how stable, dependable, trustworthy, and consistent a test is in measuring the same thing each time (Worthen et al., 1993)

For example, if we wish to measure a person’s weight, we would hope that the scale would register the same measure each time the person stepped on the scale.

…

B. Validity

1. Definition:

• Truthfulness: Does the test measure what it purports to measure? the extent to which certain inferences can be made from test scores or other measurement. (Mehrens and Lehman, 1987)

• The degree to which they accomplish the purpose for which they are being used. (Worthen et al., 1993)

For a test to be valid, or truthful, it must first be reliable. If we cannot even get a bathroom scale to give us a consistent weight measure, we certainly cannot expect it to be accurate. Note, however, that a measure might be consistent (reliable) but not accurate (valid). A scale may record weights as two pounds too heavy each time. In other words, reliability is a necessary but insufficient condition for validity. (Neither validity nor reliability is an either/or dichotomy; there are degrees of each.)

http://course1.winona.edu/lgray/el626/MandEtext3.html

Most good certification exams begin with a blueprint.  The blueprint defines the topics and sub-topics that are likely to be included on the exam.  In some cases, a blueprint will be developed by a group of subject matter experts who are familiar with the knowledge and skills needed to perform the job.  In some cases these are the experts who will also develop the test items.  In other cases, a different group of subject matter experts will develop the questions.  Blueprint development can also be based on the learning objectives from course offerings that cover exam topics.  Once a blueprint is developed, it should be put out for review by subject matter experts.  Most often these experts are performing the job roles and they are knowledgeable about what is needed to successfully carry out the needed activities.

The blueprint areas are usually weighted for importance and an associated number of items to be written are assigned to the topics to be covered.  Publishing the blueprint and associated sample test items is usually done so that the candidates can prepare for the exam.  Sample items for well-constructed exams are of the same difficulty and quality level as items the exam candidate might expect to find on the exam.

While processes vary and item development might be in workshop format, or submission by subject matter experts or another process, the end result is a certification exam that will allow individuals who pass the exam to present themselves as having the expertise that the exam certified.

Certification exams are developed and taken so that there is a standard.  People who have passed the exam can say they have met this standard.  It allows for selection of providers who have a credential that informs the “buyer” that they have engaged a professional for a role in which they have shown their expertise by passing an exam.  An exam that has been put in place to test many people who want to establish they have expertise in a specific area.

Certifications are valuable and worthwhile.  But a certification is only as good as the exam that produced it.  Avoid these common pit falls and stay focused on the goal: an exam that tests the skills, behavior and knowledge needed to carry out the tasks of the role.  In the next installment, I will discuss recommendations for updating and revising certification exams.

Preparing for a Job Interview

Candidates for positions need to be as prepared as possible for each interview session. Interviewers ask questions that determine the candidate’s qualifications. The successful candidate needs to answer the questions in a way that demonstrates their expertise. These statements may seem obvious, but keeping them in mind can illustrate effective ways to prepare for your interview. Putting yourself into the interview’s position and trying to determine what questions will be asked can enhance your preparation.

A review of each statement on the job description will allow you to identify the qualifications. This will give you an opportunity to prepare examples of how you meet the requirements and what you can offer. Once you have identified the qualifications, you can develop questions that might be asked and also practice answers to those questions.

Even if the interviewer doesn’t ask questions that lead to your prepared answers, look for an opportunity. Most interviewers will give you an opportunity to ask questions or share additional information. At this point, you can speak about your experience and expertise related to something on the job description that the interviewer didn’t cover.

The Job Description

Most job descriptions list three types of qualifications. Generally, the largest component lists the knowledge a candidate will need. Then there will also be a listing of skills and behaviors needed for the job. Example skills might be the ability to lift 35 pounds or collate and distribute handouts. Skills describe activities the hiring manager wants an applicant to be able to do.

Behaviors, also known as emotional intelligence, show how you will do your job. How you carry out your work and how you interact with others. Interviewers want to know if you will fit into their culture, and whether you will augment or enhance it.

Basically the hiring team is looking for two things:

• Can you do the job? That is, do you have the knowledge and skills to do the work?
• Do you have the emotional intelligence or behaviors that the organization is looking for in this role? The same organization generally will want different behaviors for different roles. Most often, the behaviors a good sales person needs are not the same behaviors a good engineer would display.

During a behavioral interview, the interviewer wants to learn about how you do things. During this type of interview, or segment of an interview, the focus is on how you perform your work.

Example Job Description

Position: File Clerk

Filing and pulling charts, going through papers, stocking, general helping around with the office. Some phone answering.

Need someone who is dependable, pleasant manner, self motivated, presentable and can multi task.

Interview Prep

Deconstruct the job description, identify each requirement, and construct questions related to each. This will give you an idea of the kind of questions the interviewer might ask. Prepare answers to these questions and you will be that much further ahead.

In reviewing the job description, you can identify the needed knowledge and skills and also the needed behaviors.

Knowledge and Skills:
Filing
Pulling Charts
Stocking Supplies
Answering Phones

Behaviors:
Dependable
Pleasant Manner
Self motivated
Multi tasker

In preparing for an interview, develop both the questions you think might be asked and also prepare answers. It is important to be aware that the interviewer may not ask any of these questions. However, in answering the questions that the interviewer does ask, keep the job requirements in mind. Respond to the questions and, where it makes sense, integrate the examples of how you have successfully done something similar to what is being asked.

You can practice by looking at job posting for jobs in your area of expertise. Identify the knowledge, skills and behaviors. Then develop questions and potential responses.

Behavioral Interviews

Some interviewers will ask you “past performance questions.” These questions provide the interviewer with information about how you carried out your job responsibilities previously. Research has shown that if what a person does is successful, they will do things the same way in the future.

During this type of interview, the interviewer will ask you to recount a time when you did something similar to what they will need you to do on the job. The interviewer will be focused on how you did your job.

Prep for a Behavioral Interview

• Review the behaviors listed in job posting
• Keep in mind experiences from within the last year to 18 months which show you demonstrating the desired behavior
  • Think about what you did, said, thought and felt during that time
• In responding to questions, DO NOT talk about what other people did. The interviewer is interested in what you did. You can acknowledge that others were involved with a statement like “there were many people involved in this activity, but here is what I did”
• You will likely be asked to talk about what you:
  • Did
  • Said
  • Thought
  • Felt
• Do NOT talk about what you usually do. They want you to remember what you did in that specific situation. They want to hear specifically about that time and place
• Pick something you remember well. They will ask for details about what you did, said, thought, and felt during that specific experience

“While job interviewing may not come naturally to everyone, the right preparation can make the difference between landing on your feet and hitting the street. Still, even the most experienced professionals often under-perform by overemphasizing their career progression and chronology or projecting only what they know, rather than focusing on the impact that their leadership has had on the organizations they’ve worked for and the lessons they’ve learned over time.

Today, the more senior the position, the more crucial “soft” skills are. Thus, there will be a high degree of probing throughout the evaluation process to gauge your emotional intelligence and social styles — characteristics that help distinguish a good manager from a true leader.

At this level, the most commonly used interview technique is known as behavioral competency. This type of interview focuses not only on your self-awareness but also on what is known as your learning agility — the ability to apply what you’ve learned through past adversity to overcome obstacles in the future.”

How to Master the Art of Executive Interviewing; by Tierney Remick

Summary

Be prepared for an interview, both with examples of what you know and how you applied your knowledge. Be ready to speak to your skills, how you demonstrated them, and your emotional intelligence along with examples of how you behaved in a specific situation.

Opportunity in Adversity – Demonstrating the Value of HR in a Down Economy

As HR professionals we know that we add real value to the businesses we support. In some cases, we know that line managers appreciate what we do. In some of our efforts we are viewed as partners. However, there are other cases when the line manager works with us because they believe they need to. But, they would bypass us if they could.

There are ways in which we can improve our image so that line managers see us as true partners working together to achieve business results. A down economy might just be the perfect opportunity for us to demonstrate the contributions we can make. Below are some ways to showcase our value.

1. Career Planning

There are likely some HR activities, which have been postponed or cancelled because of the down economy. This opens up time to investigate current business activities for areas where the HR team can be of assistance. Even if nothing has been removed, and even if there are fewer people on the team, this is an opportunity that should not be wasted.

Now is when the line manager may be more receptive to a well conceived and presented collaborative activity. For example, this may be an ideal time to offer an Individual Development Plan (IDP) workshop to upper level managers who do not have development plans in place. A half-day workshop with directions and feedback for developing their own plans might be appreciated. This can demonstrate the value we bring to them individually, and by extension, to their entire organization. Offering the workshop to upper level managers, without cost if need be, can demonstrate the value of HR. Once the manager has seen the value of the workshop, they may be willing to pay to develop members of their team. They may agree that “down turn time” is an opportunity to develop and maintain their teams.

2. Surveys

What services do you provide to the businesses? Are all your offerings being subscribed to effectively? Are there additional offerings or services that your HR team could make available? Organizational development activities might find receptive audiences as the businesses seek to find new ways to promote business. Sometimes a quick and simple electronic survey will uncover needs that might not be evident from day-to-day interactions.

When developing a survey, develop a mix of item types. Include multiple-choice, check all that apply, ranking, or other fixed response choice selections and just a few open-ended questions. Also, many people will only respond to questions that provide response choices. Open-ended questions require more time and effort to analyze. Keep the survey short with no more than 10 questions in total; with no more than 3 open ended items.

In developing your survey, think about the type of issues you are seeing, the type of services you can offer, and also look at the larger picture. The larger picture can be found in news reports, business publications and in speaking to colleagues, friends and relatives about issues and opportunities they are seeing.

There are several excellent and free survey tools that you can use. Survey Monkey is one that has a good reputation, as does Zoomerang. Both of these tools are available on-line for free. The tool you choose is secondary to determining what end result you want to achieve. Know what questions you want to ask before you develop the survey. Having data without a goal is not likely to result in meaningful results.

Also, you don’t want to collect data that doesn’t relate to what your organization can offer. Develop questions that relate to tools and services you can deliver. For example, your team may want to help the organization develop mentors. You would develop one or more questions about a mentoring program. You might want to ask which of several types of mentoring programs the respondent would be most likely to participate in. What would encourage them to participate in one, discourage them, etc?

3. Workforce Development / Focus Groups

If your organization has downsized, it might be an opportunity to help the organization effectively develop and deploy the remaining resources. You might offer to facilitate focus groups. The focus groups would help work groups identify their business goals, the organizational capabilities, and the steps needed to achieve goals. Setting up focus groups with representatives from senior levels and individual contributors should provide a view of organizational capabilities and needs. Start by gaining consensus on what the overarching goals are. Then, as a team, determine what needs to be done to achieve those goals. Next, identify the areas of expertise that exist within the group. Finally, identify areas where development might be needed for the organization to achieve its goals. Help the team to identify appropriate development activities.

4. Teamwork

Now, perhaps more than ever is a time when groups and organizations need to look at ways to work together towards common goals. Silos and hording key talent will not help turn an organization around. Getting the organization to stability and greater profitability may require a unified approach. It may be advantageous to set up a committee composed of key talent to address a set of specific issues and opportunities that impact the overall business.

Bringing together key thinkers and high potential members of several organizations to address problems and recommend steps for action might benefit all the organizations that participate. This is an activity that the HR team could lead. Beginning, perhaps, by inviting organizations to participate in a task force. This won’t be an easy “sell” but it might be worth the effort.

One potential way would be to let the head of the organization know that just a few groups have been selected to participate. You can explain that, by accepting the invitation, they are committing to identify a high potential member (HiPo) of the organization to participate. You can explain the value of the effort and offer to meet with them to explain the benefit to their organization. Let them know that having a group of HiPos address issues and opportunities often leads to successful results. Additional topics are also often discussed and plans are put in place to address them. Also, the HiPos will likely keep in touch with each other and share their expertise in the future.

A key task at hand is to help organizations understand that breaking down silos will benefit them even if a key resource moves to a different role. It may even be advantageous to have HiPos rotate from one organization to another. Sharing knowledge within a company is always better than maintaining silos.

“A report by Industry Week noted that silos are the biggest hindrance to corporate growth. Another study by the American Management Association revealed that 83 per cent of executives said silos existed in their firms. Of these, 97 per cent believed silos were counterproductive.
The silo mentality can lead to a number of serious communication issues. Managers may be overprotective of their department’s activities, even going so far as to act as gatekeepers of information that impacts the rest of the organization. Internally, this lack of co-operation can cause internal competition and even a complete breakdown in communication.”

Shared information helps firms dismantle their ‘silos’, Written by Dr. John McFerran; Saturday, 04 April 2009
Summary

A down economy is an opportunity to forge greater connects to the line organizations you support. It is a time when activities and program put in place can result in a win for both the line and the HR organization.

How to Get a Seat at the Table

We are experiencing an economic downturn that is worldwide in nature. Companies of every size are announcing employee lay-offs. In order to survive, most businesses have downsized their workforce. They need the remaining employees to be as productive and cost effective as possible. Agile employees and business processes are the order of the day. The economic outlook for 2009 may have glimmers of hope, but it is still volatile. This economic situation can be viewed as an opportunity. Learning and development professionals can demonstrate their value to their business organizations. L&D professionals can provide knowledge and tools that can increase employee ability to perform their jobs. Increased effectiveness can lead to better techniques for revenue generation or reduced costs.

Learning and development organizations need to focus on developing materials and tools tied to producing revenue or decreasing costs. The better aligned an L&D organization is to the core business; the more likely it is to develop products that businesses will be interested in adopting and using.

In addition to being closely tied to business strategies and goals, new learning events also need to be brief. They need to be developed and ready for delivery in short time frames. After the elimination of positions, the people who remain will have more responsibilities. They will need to learn more, and have less time for learning. Today it is all about doing more with less.

In the current environment learning materials need to be produced in a timely manner. If needed the learning materials can be enhanced or modified later. Learning events for profitable products or professional capabilities might need to undergo continual improvement. Revisions to less profitable offerings may not be warranted. If it is an offering that consumers want or need, organizations are generally more than willing to fund continual upgrades and enhancements. In the case of product or service offerings with short life cycles, investments need to be sized appropriately to the value and life of the profit-generating offering.

Most training developed for business and industry is only moderately effective. At the end of 2007, Training Magazine conducted an analysis of the U.S. training industry. In reporting their findings, they said

This year, the primary focus among training organizations is on increasing the effectiveness of their programs. … The importance of training effectiveness has ratcheted up significantly in the last year, as organizations look for ways to affect the transfer of learning and ultimately the bottom line.

NOVEMBER / DECEMBER 2007 training magazine www.trainingmag.com

What follows is an outline of the steps that lead to effective learning results.

• Once an assignment is received, conduct a training needs analysis to verify it is a learning problem. Also verify that the problem can be solved with a training solution. Google “conducting an instructional needs analysis” for more information on how to perform this step.
• Identify the audience for the learning event and what knowledge and capabilities they bring to the learning event.
• Define the desired end result. What should someone be able to do at the conclusion of the instruction
• Deconstruct and document each step that needs to be taken to achieve the end result.
• Identify and document where things can go wrong.
• Review the “can go wrong list” and add steps to achieve the end result. These are steps that should be taken before something goes wrong, but which are usually overlooked when the needed steps are identified.
• List everything that a person would need to do to carry out the activities that will be covered in the training.
• Describe how that performance can and should be tested.
• Develop test items to incorporate within the learning event and for final test of learning.

Summary Instructional Design Steps

1. Needs Analysis
2. Identify the Audience and their Background Knowledge
3. Define End Results
4. Document Steps to Achieve the End Result, Highlight Points Where Things Can Go Wrong
5. Develop Test to Verify Learning has Occurred
6. Draft Content Keeping in Mind the Performance that will Need to be Tested

Why Conduct a Training Needs Analysis?

This step is needed because it determines whether or not the problem can be solved with a training solution. All too often, when a problem impacts an organization, the Learning and Development organization will be tasked to correct the problem with training. Begin with an analysis of the problem rather then diving right into the development of the learning materials. This allows for identifying the root cause of the problem and determines whether or not training is a solution that can correct the problem.

Let’s look at an example.

• Sales revenue is entered into the tracking system when a product ships
• The revenue numbers for a new product are very low
• The training organization is asked to produce an instructional unit on how to sell the new product

The L&D team began by speaking to members of the sales team and learned that orders were being taken, but product wasn’t being shipped. In speaking with the shipping organization, the team learned that manufacturing wasn’t producing the new product. In speaking to the manufacturing organization, the training team learned that the manufacturing team was measured on how much of the older product was moved out of the warehouse. They were not goaled on producing the new product.

The training needs analysis uncovered a business issue that needed to be addressed. The sales organization was being measured on how many new products were delivered before the quarter closed. At the same time, the manufacturing organization was measured on how much of an existing product was shipped before the new product was produced. This is a problem that a training solution is unlikely to solve. This is a metrics problem, not a training problem. The training team provided their findings to management and the request for training was withdrawn.

Identifying the cause of the problem or opportunity before developing instruction provides the training developer with the information to determine if it is a situation that can be successfully addressed by training. If the analysis shows that the problem or opportunity can be addressed with training, further analysis can identify the areas to be covered in the learning event. The needs analysis is the process by which the training team learns about the work to be done, the end results, and the teams or individuals impacted.

Effective instructional materials are the result of a series of steps, which need to be taken before any content is developed. The learning needs analysis is the first of these steps.

Identify the Audience and their Background Knowledge

Ask the sponsor or learning event requester for a description of the target audience. Who do they want to take this training? What should they be expected to know before participating in the training. To the extent possible, include information in the course description that provides information on training, references or tools where the prerequisite learning can be obtained.

Defining the End Result

The next step is to define the end results of the training. What will the learner be able to do as a result of the training? The training development team will not always know exactly how the learning will be applied. But, knowing what to cover in the training allows the team to define what people need to be able to do. For example, if you are developing instruction on how to drive, you won’t know the type of vehicle the learner will drive. It could be stick shift, car, bus, or truck. But, the basics are the same. The rules of the road are mostly the same, and differences can be included that a specific group will need. A segment on stick shift can be developed, as can one for trucks, and so on. You know the end result is for the learner to be able to safely follow the rules of the road and drive the vehicle.

Your business partners can tell you most of what you’ll need to include. You will then have enough information to begin developing the learning objectives and test items.

Document Steps to Get to the End Result

What steps need to be taken to get to the end result? If someone is doing work that the training covers, shadow them and record what they do and why they do it. This will identify what needs to be included in the training. However, in most cases this won’t be available. If you are developing instruction on a new product or service, look at existing products and services. Ask the product developers what is new or different and why the changes have been made. This will provide some of what needs to be covered in the training. Also, to the extent possible, use or watch the product or service being used. Have your subject matter expert test all of the existing functionality. Document all of the results for inclusion in the training package.

By developing a series of statements about what the learner will be able to do; you define what needs to be included in the training. Each statement needs to contain an action verb and require an observable and demonstrable action. Use words like say or do instead of know or understand. Any non-observable verb statements – such as know or understand – should be placed on another list. This is the list of steps to get to the observable activity.

When the list of observable statements is completed, look at the list of knowledge statements and identify what a person will be able to do as a result of the knowledge or understanding. Take time to identify whether there are any new demonstrable activities that can be added to the “observable actions” list as a result of these knowledge statements.

Next, associate the non-observable skills from the second list – the “know and understand” statements – with the observable actions statements. What does the learner need to know or understand to perform the needed activity? Associate the skills to be learned and knowledge to be acquired with real world tasks. Each point covered in the learning event should map directly to one or more of the end results, the desired outcomes. These results are also known as the statements of the objectives of the training or the learning objectives.

The analysis should identify demonstrable activities that show learning has occurred. This is what needs to be tested. The instruction is a presentation of steps taken to get desired results, along with an explanation of how to perform each of those steps and examples which show how it is done.

It is very important to let the learner know why they need the information that is being presented. A learner might be told that red means stop and green means go. But if that information is never associated with driving or crossing a street, the learner may not know how or when to use that information.

Listing each observable behavior (each “able to do”) creates the blueprint for the development of the instruction. Essentially, this is a list to successfully achieve the desired end result. Each area which could potentially be a problem area needs to be identified. When the instruction is developed, carefully address each area in detail. This will help the learner gain the knowledge and skill to perform the desired end results.

Develop Test to Verify Learning has Occurred

List what needs to be tested and list how it will be tested. Develop test items. Test the effectiveness of the test items with subject matter experts and knowledgeable learners.

Draft Content

Instructional development is iterative, and also needs to be reviewed and tested. As you develop the instruction, it is best to stay in touch with people developing the new product or who are knowledgeable about the product or service. Find out what is new, changed or modified. Have the product developers review your materials and provide feedback. Also, identify a few people who need to be trained on the content. Find some who have the prerequisite knowledge and ask them to “take” the training as it is being developed. Have them identify any areas where they are lost or confused. This often indicates that content is missing or unclear.

As you develop the content, include examples and practice questions. This allows the learner to assimilate the material being presented. Developing short units for a learning objective or a small set of objectives hones in on any content that needs augmentation or revision.

More instruction or explanation may be needed, and it can be added. If there is too much detail or too much information is included, instruction takes longer to develop. It is difficult to determine what should be deleted. It is easier to identify what is missing and add it in then it is to figure out what needs to be removed.

Once all the end results or objectives are defined, the next step is to develop the questions. Questions test an individual’s ability to perform the needed activities. Questions test the objectives. In some cases, an organization may not want to include test items as part of the instructional package. However, writing the questions that test the required end results, even if they won’t be used, helps the instructional developer to ensure that the instruction that is developed contains the right content.

Summary

Following these steps will lead to transfer of learning. Following this type of development process allows extraneous or unnecessary content to be avoided. Like any science, there is no magic or trickery involved. It is a matter of defining what people need to be able to do, defining what they need to know to do it, and then adding “show and tell” and practice. These components may appear simple, but there is expertise involved in each step. Most importantly, it will produce learning events that work.