Another Way

I have found that when technical professionals are tasked with developing instruction, the hardest thing for them to do is learn how to write learning objectives. In this post I suggest a method for communicating what the learner will be able to do without writing those dreaded objectives. Even better, using this method will allow you to translate the end result of this exercise into objectives that are painless to develop.

Rather than beginning with writing statements of objectives, begin by writing questions that you want a person who took your training module to be able to answer. These questions should ask something about every area that you plan to cover in the module. Keep in mind as you do this, that we are now talking about covering small, discrete units of training in an instructional product.

 

https://www.shiftelearning.com/blog/numbers-dont-lie-why-bite-sized-learning-is-better-for-your-learners-and-you-too

 

“According to the Journal of Applied Psychology, learning in bite-sized pieces makes the transfer of learning from the classroom to the desk 17% more efficient. Here’s why microlearning is more efficient than traditional longer-duration courses:

When bite-sized learning content is easily and readily accessible, learners can take it at their own pace, wherever they are, and most importantly, when they are “ready.”

Because bite-sized courses are more focused, learners don’t have to clutter their memories with irrelevant information. This makes retention easier.

Learners have to digest only small chunks of information. This makes comprehension easier without spending too much effort.

Because microlearning content addresses only 1-2 learning objectives, courses, on an average, yield 4-5 learned takeaways.”
…….

“Cut to the chase. Do away with the history, the background information, and the theories. Deliver the “how-to” right away. Remember, your learners are looking for just-in-time solutions.”

 

If however, you are not developing a small unit then I suggest you partition out each area that you will be covering and develop questions for each of them. You may want to develop questions for just the first section and proceed as I recommend below before moving to the next segment of your learning product.

Develop a list of all the questions that you want somebody to be able to answer when they complete the training. In some cases your training might require a demonstration rather than an answer. For example if you are training someone in how to drive a car there are many questions that you will want to ask them to answer but you will also want them to demonstrate their ability to drive the car. In this case questions can be developed that say something like show what you will do when you get into the car, show what you will do when you want to turn left, show what you will do if someone darts out into traffic unexpectedly, etc.

The end result of developing these questions will provide you with an outline of what you should include in the learning materials.

You may find as you develop the learning materials that you think of additional questions. That is OK; add them to your list. Keep at this until you believe you have listed all the questions that someone will need to be able to answer and you are ready to begin to develop the instruction. It is at this point that you can stop and translate the questions into objective statements.

Examples:

• Describe how to develop learning objectives from a set of questions designed to cover the content of a learning module
• Explain what needs to be done to make a legal left turn

Hopefully this technique for developing objectives will help both you and your learners.

Developing Certification Exams

There are many factors to consider when implementing a certification exam program. The benefits of a well-developed and implemented program are many, ranging from well-prepared employees to increased profits due to a knowledgeable team. The tests themselves are most often a revenue generating process as many different populations can seek to be certified. But getting there requires multiple steps to ensure a quality exam.

It is critical to ensure that each step in the development process is well planned and executed. Below is an overview of steps that need to be taken to ensure a quality outcome. There are other steps that are followed by different development programs. However these are the major steps and processes.

• Begin by identifying the end results needed. What do the people who receive the certification need to be able to do.
• Once the end results have been identified, the steps needed to produce the end results have to be documented. For example, if the test covers driving different types of vehicles (perhaps cars, trucks, motorcycles, golf carts or others) include all the steps needed to drive each type of vehicle. This is known as a task analysis.
• Next a blueprint is developed. A team of experts most often develops the blueprint. The experts who develop the blueprint generally include practitioners and others who have expertise in the area. The blueprint describes each component of the task analysis. The blueprint identifies each area that will be tested by the exam. Most blueprints include the number of items for each objective that will be included on an exam. They also usually assign a weight to each topic area. Multiple examples of templates are available on line. Here is just one example.

https://www.nursingworld.org/~4acd24/globalassets/certification/certification-specialty-pages/resources/test-content-outlines/familynp-tco.pdf

• Once the blueprint is developed it needs to be validated by other subject matter experts (SMEs). Note that it is not expected that one person will have expertise in all areas covered by a full exam.
• At this point the item writing process begins. The very first step is to agree on the blueprint. Most often there are changes. Additions and some deletions can also happen, as topics may not be appropriate for the specific exam, may be covered on another exam, or might make the exam too long. Also there may be changes to the percentages and other edits before the team agrees on who will develop which items and the number of items each person will write.
• Handouts for the types of items to be included on the exam (for example multiple-choice, matching), how to write items, and what should and should not be included in an item are generally passed out and reviewed before participants begin writing the items. There are often handouts to explain how to input questions into the item development program used for item input.
• Here is an excellent set of guidelines for exam writing

http://www.cedma.org/blog/certification-exam-item-writing-best-practices

• It is important to let item writers develop just a few items (about three) and stop so that their items can be reviewed for feedback. Generally letting them write for no more than 30 minutes works well. It is also a good idea to review with the full team so that everyone hears the feedback. For a large group, the exam facilitator may want to just review one item written by each developer. Have them post their items in a way that allows for only the facilitator to see them. The exam facilitator should choose items to review so that different areas that need attention can be discussed.
• Once the first review is completed it is time to let the writers write more items. The exam facilitator needs to continue to review the developers’ items so that if they are making errors the facilitator can let them know what needs to be fixed. Usually a recommended rewrite helps. If the facilitator doesn’t know the subject area, they can use a gibberish word that will allow for the needed concept to be explained.
• After a period of time (usually two hours works) take a break and then begin to review the items. The facilitator can provide feedback on test development techniques and the developers need to provide feedback on the content of the questions. If there is agreement that the question is good, then it can be approved. If there isn’t agreement, the team can try to repair it. But in some cases it is better to delete the question, as time is often a factor in getting the number of questions that are needed and approved completed in the allotted timeframe.

Once the development workshop is completed the questions are sent out for review. Each question needs to be reviewed by multiple reviewers. Also each reviewer should only review a limited number of items to ensure the confidentiality of the exam.

That’s all for now. A future blog will cover additional information about certification exam development. Please let me know if you have comments or questions.

Telling Stories

Long before recorded history humans were telling stories to pass on important information. Research has consistently shown that presenting information in story format results in effective learning. Story telling can also be used in a business environment to convey different concepts and information.

https://www.nyu.edu/faculty/teaching-and-learning-resources/strategies-for-teaching-with-tech/storytelling-teching-and-learning/when-to-use-stories.html

“Storytelling can provide variety in the learning experience and enhance the ‘chalk and talk’ approach. They can be used in the very beginning of the sequence as a way to gain the attention of the learner. They can be used in the middle of the sequence when introducing a complex concept. They can be used at the end of a sequence as a way to summarize information.”

http://www.harvardbusiness.org/blog/what-makes-storytelling-so-effective-learning

“Good stories do more than create a sense of connection. They build familiarity and trust, and allow the listener to enter the story where they are, making them more open to learning. Good stories can contain multiple meanings so they’re surprisingly economical in conveying complex ideas in graspable ways.”

Telling a story of how to address a business problem is more impactful than listing the steps that need to be taken. By conveying what the problem was that needed to be addressed and explaining how the individual or group went about solving the problem usually gains the learners attention more fully than a set of “how to do” instructions.

It is also likely that telling a story after explaining a problem or opportunity will engage the learner in the solution. A topic can be introduced followed by a story and then the learners can be given an exercise where they will need to use the information provided in the story. Learning by doing is a proven learning technique. Combining both story telling and hands on practice both of which are proven technique for learning is likely to provide the results businesses need from the training they offer.

Even when the training involves new products or services a story can be incorporated into the training. New products and services are developed based on a perceived need or opportunity. Telling a story about the need a business has and explaining how the product can contribute to addressing that need can be an impactful story.

I recommend that if you are developing training on any topic that you think about how the subject of the training can be enhanced by a story.

How We Learn: Applying Methods for Learning Complex Games to Other Learning

Throughout our lives we learn many different things in many different ways. We may learn to choose a pair of shoes because they fit without hurting, or we may chose a pair based on how they look or based on where we will wear them. Most people won’t wear tennis shoes to a wedding and won’t wear dress shoes to a tennis court. People learn which foods they like through trial and error, or perhaps they try it based on a recommendation from a friend, or because it is being promoted as a new taste treat, or as something that is not only healthy but also tastes good.

All these learnings are usually much easier for us then learning an academic topic or a work related activity. We have looked for, and continue to look for, new and better ways of imparting knowledge and skills to learners. I wonder what the difference is in how we learn work or academic topics versus topics related to our avocations.

Over the years I have observed many different people learn how to play complicated board games. These games have many strategies and rules that need to be learned in order to play effectively. In thinking about how some successful gamers learn new games, I began to wonder if the methods they use might not be similar to effective learning strategies. In particular, they use iterative processes while learning and playing that also work in learning technical topics. I wonder why the game learning is usually successful but we are still looking for better ways to impart better learning outcomes in other learning environments. What am I missing?

Gamers read, listen to, or watch a presentation of a chunk of the rules. They then begin to play the game according to a portion of the rules they have just studied. They discuss what they have learned and how they applied their learning. They work to play the game as effectively as possible. They discuss all the nuances of the game, all of the complex rules, and all of the gotchas.

Looking at complex games, there are many options for how a player can interact with the situation and with other players. The game usually includes multiple rules, different outcomes; different paths to the end of the game, and of course multiple problems that can be encountered. It is very much like learning how to perform a new process or other aspect of a job role.

Most often new game learners will rely on the multi-chapter, multi-page, or audio or video guide or rulebook that comes with most of the more complicated board games. In other cases a well-versed player will teach the rules to inexperienced participants. Players often spend several hours-long sessions studying the game rules and testing parts of the game. They begin to play while often consulting the rules or notes they have taken while learning the directions. There is usually a great deal of discussion as they work their way through the game.

Even when one or more players have previously played the game, there are questions and discussions and reference to the game rules. Because this type of game is complicated, consulting the rules often results in all the players gaining more understanding and being better able to effectively play the game – even those who have previously played the game.

The game master or game instructions present the opening situation. Players are given the information about the game objectives and any other information needed to begin playing. As they begin to play, the players identify points of confusion and questions they have about the game rules and they frequently reference the rules. They continue this process until the game is completed. They then review all steps and assess changes needed for next game play session. They modify play as needed to better meet the rules. In some cases they modify the rules to meet the groups preference. They know there are no game police but rather rules that can be modified to meet their preferences.

Comparing this process to learning situations, I believe one key difference is that they use a process similar to formative evaluation of instruction to ensure the game materials meet their needs. Yes, their needs are fun where the needs of learners in business environments are to be better able to perform their job roles. I think we might try to use this game learning process in our evaluation of learning materials.

Formative evaluation techniques are rarely used in business environments for many valid reasons. The foremost is that learning products change frequently. Because the products and services being offered change constantly the training needs to also change at the same time.

I propose that if a modified formative evaluation process was used when the training is piloted, that we could produce better instructional materials. I suggest that, just like sitting down to learn a game, a team of evaluators should sit down with the training materials and follow the steps that gamers use. These techniques might produce better training events right out of the gate. I think it would mean adding a day or two to the pilot but in doing so we could end up with much better products. Let me know what you think.

New Year, New Blog

Happy New Year Everyone!

A while ago I was speaking to a women who taught English to recent arrivals from Spanish-speaking countries. She told me that one of the most important things for teaching them how to pronounce English words was to not show them the written word. She said she has had much more success by keeping them from pronouncing words as they would in Spanish when they encountered a new written word.

Remembering this conversation got me to thinking about how we train people. Would we get better results if we presented information in the abstract? Would we get better results without telling them where the new content connected to what they already know?

Yikes, I can’t believe I even wrote that paragraph. I have always been a big believer in telling the learner where things fit into what they already know. But, I have also been thinking about flipped learning and the Khan Academy practices. While this blog isn’t about applying these concepts, it is about looking for opportunities to think outside the box, which is what both the flipped learning and Khan concepts do. (https://en.wikipedia.org/wiki/Thinking_outside_the_box)

My thought is that, while we have had some good successes in improving learning, we are still looking for additional ways to improve how learning happens. In particular, I want to address learning in business environments.

Perhaps we can apply some strategies that wouldn’t be applicable to university settings. Perhaps in some business cases, it might be that people need to learn how to do specific things in a specific way. Given that we are starting with this premise then people only need to learn how to do the specific activities and how to correct any missteps. In this situation, you wouldn’t need to teach the context, the history, or any of the whys or wherefores. Yes, I’m out on a limb here, but please stay with me.

So for these specific situations, we could begin by creating a list of what learners need to be able to do. Then we could break each topic into all the steps that need to be taken. Next we divide each topic into what the learner needs to do step-by-step, and also what the learner needs to know to take the step. Then we identify where things could go wrong and how to apply fixes. Next, list just the minimum actions that the learner needs to take. Finally we can create an exercise that the learner will need to be able to do in order to demonstrate that they know how to do the activities.

One of the best ways to identify the steps that need to be taken is to have someone perform the steps while someone else takes notes. After each step, discuss what could go wrong and document it. Then integrate how to recover from the missteps as part of the instruction.

Once you are satisfied with the process, review the write-up and remove extraneous content. Does a new learner need to learn everything that has been documented? Also is there anything else that needs to be added to the steps?

Document the steps but don’t add any instruction. Next have someone who has the prerequisite background but doesn’t know the new activities test out the instruction. Sit with them and ask them to “think out loud” about what they understand about what needs to be done, as well as anything that they don’t understand or anything that confuses them.

I realize what I’m suggesting requires a bit of up front work but I hope that the results will provide better learning and performance. It is, at the very least, an experiment worth trying, I think.

Surveys: A Short And Simple Approach, Part One

I believe in using surveys. Surveys can provide valuable information with very little effort. Surveys come in all sizes. But surveys that are too long or too intrusive often won’t return the wanted or needed data. Most organizations and individuals don’t use surveys often enough to get meaningful data for decision-making. A quick and simple instrument can dramatically help to provide actionable information for decision-making.

Why might you want to survey a population? More importantly, what do you want to learn? Knowing what you want to know before you survey will lead to more effective results and more useful information. If you wait to see what you get and then try to figure out what you learned, you might not learn anything.

Perhaps you want to offer a workshop on a particular topic, but you aren’t sure that enough people would be interested. A survey administered to members of your target population might well help you decide whether or not to develop that workshop. Or perhaps you want to offer a new product or service in your establishment. Again, a survey might give you the information you need before you invest time and money in something that might or might not be of interest to potential customers.

Like anything else, surveying has proven methodologies associated with it. It is a science like any other science. There are procedures that work, but many of the most commonly used methods provide little to no useful information.

Many people are of the opinion that surveys are used to gain information regarding complex or critical decisions. That is true, of course. But there are other ways to use surveys for decision-making that take a minimum amount of effort to deliver results that are well worth it.

The first question to ask when developing any survey is, “What is the desired end result?” The survey results are a step towards getting to that actual end result.

Once the desired end results have been defined, most people go on to defining the questions that need to be asked. However, I suggest that the next step should be answering the question “What is in it for the survey taker?” You can ask questions that your survey population will answer, but it is important to ask questions that your population will want to answer. If you haven’t done it already, here is where you want to define the members of your target population(s).

Once you have identified the needed end results, what is in it for the participants, and who the participants are, you can then construct your survey items.

Come back next week for some tips and strategies for building your survey questions!

The Changing Times

Usually when I select a topic for a new blog I start by doing some research. I think about the topic and what I want to share. Then I look for current postings about the topic, what the most common thinking is, and what new or interesting information I can impart.

For this blog I wanted to look at the roles of corporate course developers and trainers. It wasn’t so long ago that people who wanted to be hired into a technical role often found that only training roles were available. They generally came in, did their time in the role, and then moved on within the corporation.

I was very surprised to learn that the requirements for current jobs had changed significantly. In what I think of as only the recent past, the main requirement was a technical background and a willingness to learn on-the-job how to develop and/or deliver training.

Today the role of instructional designer now has many additional responsibilities. And, of course, along with the additional responsibilities there are additional areas of knowledge required to carry out the role responsibilities.

The instructional designer is now a sought after role. People want to be in the role rather than wanting to move from it into a more prestigious role.

https://elearningindustry.com/6-figure-instructional-designers
WANTED: Six-Figure Instructional Designers
By Vicki Kunkel
May 26, 2015

“Once considered the red-haired stepchildren of corporations, instructional designers are finally going from no-respect Rodney Dangerfields to super cool content heroes.”

Kunkel also had two important observations to make.

The first was that the role now commands a six-figure salary, but she also said:

“One caveat: nearly all of the director and VP level positions required both design and development skills; not one or the other.”

This last comment was of great interest to me as, other than when I was a student, I have worked as a course designer or as a consultant to course developers but never both. The role of the course developer has expanded to include more areas of expertise and is generally now referred to as an instructional designer.

I am wondering where the role of the individual who uses a structured process to design and develop instruction has gone. In the past this role was known as an instructional designer. This role, while still performed in an academic setting, doesn’t seem to have a place in corporate training departments. As this is a role that I have performed in several different corporate settings I believe it is a role that has gotten lost and that corporate training departments might do well to consider bringing it back.

What Do Instructional Designers Do?

What Instructional Designers Do: Is this a career for you?
by Connie Malamed

“Instructional design involves the process of identifying the skills, knowledge, information and attitude gaps of a targeted audience and creating, selecting or suggesting learning experiences that close this gap, based on instructional theory and best practices from the field.

Ideally, workplace learning improves employee productivity and value and enhances self-directed learning.”

I believe that the change in the role responsibilities for the corporate person who is called an instructional designer are excellent but I would also like to see corporate training departments considering the value of adding someone who does the up front analysis of what is needed based on learning theory. As we move more and more away from instructor led training I believe this role will become even more important. I believe using these techniques will allow for better-targeted learning experiences with better learner retention.

If we look at other professions we can generally see that people who perform those roles do a creditable job with delivering the needed end result. Accountants, engineers, business executives and many others for the most part deliver a more successful end result than training professionals. Almost all business training organizations request feedback on the quality of the course and when an instructor is involved the company also looks for feedback about the delivery. This is because there are, in my opinion, far too many complaints about the quality of training offerings. If a learning professional does the up front analysis and identifies the end results for a learning event, it will go a long way towards offering a better quality product.

Mini MOOCs: Right Sizing Online Training

MOOCs (Massive Open Online Courses) are a part of today’s training toolbox for some universities and businesses. The concept of this type of learning event is to provide instruction to a massive worldwide audience. While the use of MOOCs to provide needed learning is a good concept, to date MOOCs have had only a mixed reception. Most MOOCs offer instruction through several types of delivery methods including readings, recorded lectures, videos, gaming and many other delivery techniques.

Many of the people who sign-up for a MOOC don’t complete all of the content offered. This may be because of boredom. Or perhaps they only needed to learn about some of the topics covered in the course and weren’t interested in the rest of the content. Or it might be that most MOOCs don’t provide effective methods for interacting with instructors and other learners. Furthermore, satisfactory methods for testing and feedback are still a work in progress.

Feedback and scoring of tests usually comes from individual MOOC participants grading each other’s work. Even when a participant receives scores from many other students most learners don’t find this satisfactory feedback. There is also the problem of worldwide audiences not sharing a common language. Whatever the reason for not completing the learning event the stats show that many people who start a MOOC do not complete it.

“MOOCs are not single, long-haul flights, but rather they are more like train journeys where some people want to get to the end of the line but most people get on and off along the way.”
https://www.class-central.com/report/moocs-course-completion-wrong-measure/
MOOCs: Course Completion is the Wrong Measure of Course Success
by Donald Clark . Published on April 11, 2016

At this point in time MOOCs are showing limited success. The use of MOOCs is still not where some of the developers had hoped it would be, but they are still being developed and used. There are still people and organizations that believe MOOCs offer a delivery method, which is needed in our ever-growing globalized learning environments.

“In the future, MOOCs have the potential to completely transform education. However, as of right now, don’t expect to see universities shutting down as a result, as some experts have begun projecting. Despite the recent rapid rise in MOOCs, this format continues to be an evolving model, and one that isn’t quite established yet.”

What’s Wrong with MOOCs and Why Aren’t They Working?

The tech edvocate
WHAT’S WRONG WITH MOOCS AND WHY AREN’T THEY WORKING?
BY MATTHEW LYNCH JUNE 12, 2016

So while MOOCs continue to evolve and offer learning events to meet many different needs I believe we have an opportunity to look at how the “MOOC concept” can be applied in other ways. One way might be to develop very short units of instruction or in other words Mini Open Online Courses (mini MOOCs). I believe if we use the MOOC processes for units of instruction that cover a single – or very few – topics, we may find that the mini MOOC is an effective delivery method.

Multiple mini MOOCs could be linked together to provide a learning experience that can be tailored for each individual. These mini MOOCs can be thought of like a large lunch buffet. The mini MOOC concept would allow learners to pick and choose what they want or need to meet their individual learning needs. The mini MOOCs with very few topics would be like a college curriculum and be applicable to multiple degree or certification programs.

Short modules addressing just a few topics have been shown to be very successful. I propose that we look at whether this instructional methodology can be applied to the MOOC concept. Perhaps this can best be described as an offshoot of the MOOC concept. Most MOOCs present multiple lessons on an overarching subject. They also include several different types of learning activities. There are self-paced, group, team etc. types of learning experiences contained in most MOOCs. It is likely that the developers want the learners to have the option to experience a wide variety of learning methods and also allowing learners to choose learning techniques that work for them. These mini MOOCs can follow the MOOC methodology of making each mini MOOC available 24x7x365.

“… from our ongoing research on adult learners that while they do still prioritize degrees, there is growing interest in shorter-form online programs.”
http://www.eduventures.com/2017/05/seismography-101-understanding-shockwaves-purdue-kaplan-deal/
Seismography 101: Understanding the Shockwaves from the Purdue – Kaplan Deal
By Howard Lurie, Principal Analyst. Published May 2017

Much like instructional guides developed for instructors, guides for the learner can be developed which present different ways to group lessons together to gain the needed learning. Each mini MOOC would present just a few topics along with practice and feedback. Once a learner completes a mini MOOC it would be important for the learner to apply their learning to their work or find other ways of practicing what they learned. It might be that the mini MOOC developers create exercises or games for the learners to use once they complete the mini MOOC. This will allow learners to incorporate the new learning into their knowledge base. Testing also would need to be delivered and scored effectively. I have some recommendations for testing that can be applied to MOOCs and mini MOOCs as well as more traditional learning events, which I will present in future blogs.

Learning Design: Do it in Chunks

When learning programs are designed for the K-12 environment, the designers need to use a broad brush; they need to provide the learners with life skills in the topics they are addressing. University programs need to deliver instruction with a variety of examples. Again, the learners may need to apply the concepts and skills to a variety of different applications.

A corporate environment is different, and I believe the design and development of training needs to reflect how the learning materials are applied in a business situation.

In corporate situations most people attend or consume training to learn about what they will need to do on the job. Developers of corporate training events must identify the end results the learning event prepares people to deliver. In other words they need to identify what people will need to do on the job. Based on what the learners will need to do, learning events should be developed which tell the learner what they need to do, how they will do it and provide multiple opportunities for practice and feedback. They also need to tell the learner where they can go wrong and how to prevent or correct mistakes.

Delivering elearning via slides is counterproductive. Why use slides when the learning materials are being viewed on-line by the learner? Words, graphics, video etc. that assist learning are all excellent ways of delivering content that provides the learner with the knowledge they will need to carry out their job role responsibilities. Just be careful to ensure that the tools used provide learning rather than something nice to see but devoid of instructional relevance. The instructional materials should also include a table of contents and an index, both of which allow the learner to click and go right to the content they need.

All the instructional material should be chunked, delivering small pieces of content, examples, cautions for mistakes and practice and feedback, and a summary of what was covered. In general, each chunk of instructional content should be able to be consumed by the learner in a 5 – 7 minute period prior to an interactive exercise. Pointers to where the information was covered and also additional references should also be included with the exercises. The exercises should ask the learner to do something that they will need to do on the job. They should be asked to show their ability and to apply what they have learned. Memorization questions, unless memorization is needed on-the-job, should not be asked. A question which asks where information can be found is, however, appropriate.

There is much more that can and will be said in future posts on topics related to corporate learning events. Please comment and also let me know what else you would liked covered in these brief posts.