Reviewing a Curriculum Company's Work Samples

Thinking about hiring a learning development company to help you develop custom curriculum? Then it's a good idea to review other work they’ve done before you sign an agreement.

Our recent series on this blog described the five steps to take before hiring a curriculum development firm for your next training project. But let's examine the necessary step of more closely reviewing their work samples. The following 10 tips will describe how to do this effectively. For the most part, this list applies equally well to the development of both online and classroom programs (although it's weighted slightly more to online training).

10 items to consider

1. Define your motives. Why are you reviewing work samples? What’s important to you and your program? What would rule a vendor in or out based on the samples you’re going to view? If you can’t define your motives, you’re just “window shopping” and likely won’t advance your selection decision in a meaningful way.

2. Ask the company, “What was your role in developing this program?” Did they do it all or just a piece of it? Once you know that, you can focus on the work they did and not spend time evaluating work someone else did on the project.

3. Evaluate the appearance in context. The program needs to look pleasing and professional to students, but it also needs to be instructionally effective. Don’t let the look—good or bad—keep you from evaluating the instructional design. Also keep in mind that many times the appearance is a function of the client’s branding or their previous learning programs, rather than the creativity of the vendor. When you’re finding out what the vendor was responsible for (see the previous item), make sure to include appearance.

4. Probe the instructional design. You are an experienced student as well as a learning professional. Review the program from both perspectives. As a student, is the material clear? Does it respect your time? Are you learning from it? As a learning professional, is the instructional design intentional? Does it incorporate appropriate and effective instructional techniques?

5. Consider the learner engagement. This is a tricky one for online programs. It would be great if every program had the budget to use all the tools and techniques available to engage and interact with the student. But that’s not always the case. Evaluate the engagement for what it is. If it’s very simple Level 1 interactivity (based on the U.S. Department of Defense handbook “Development of Interactive Multimedia Instruction”), assume that’s what the client could afford and that the decision to restrict interactivity is appropriate for the content and audience. Then evaluate whether it was done well, is appropriate, and supports learning, retention and application. Similarly, if the engagement techniques are higher level and obviously reflective of a big-budget project, was the investment used wisely? Does it achieve engagement? Is it more entertaining than educational?

6. Understand the development tools. This item depends on whether you have the expertise to make the call. Assuming you do, by finding out what tools were used and how they were used you learn a lot about the expertise of the developer. Did they use the wrong tools (like a pair of pliers to drive a nail)? Or do they understand and have access to the right tools for the job? Evaluating this will give you the opportunity to learn whether they are “tool neutral”—that is, they have access to virtually any development tool and select one based on the instructional design or requirements of the client—or whether they only use one program and make every program conform to their tool and limited expertise.

7. Read the descriptive materials. Is the sample accompanied by a paragraph description or case study that details the background of the project as well as learning tools and techniques used? If so, analyze the descriptive materials to help you gain additional context of the sample. If there isn’t, ask the company if they can provide background material.

8. Understand that this isn’t your program. Good custom learning programs are unique. If you see one sample and determine that it's not what you want your program to be like, you’re missing the point. First, evaluate more than one program from a vendor so you can develop a sense for whether they have a style (which is good sometimes, but not always) or whether they’re a chameleon and can make the program fit your style rather than theirs (almost always a positive). But unless you see multiple programs and they all look the same, don’t assume that’s how your program would look if you hired this vendor.

9. Understand that communication, responsiveness, positive attitude, expert advice, project management, overall partnership skills, and even price are also critically important. Work samples are valuable, but they’re only one factor. They’re usually more helpful as a qualifier than as the deciding factor. Use them to help you determine which firms have the expertise to develop your program. But then make your final selection based on which vendor you want to live with through the course of the project.

10. Understand that custom curriculum is proprietary. It’s typically owned by the client, not the developer, and sometimes contains confidential and/or sensitive information. As such, appreciate that a quality vendor has some restrictions, probably can’t show you everything they’ve done, may have to restrict how and for how long you can see a work sample, and so on.

That does it. What have we missed relative to reviewing work samples? What experiences can you share where work samples were helpful (or not helpful) in hiring a vendor for custom curriculum development? Drop us a line and tell us about it!