Nothing to Smile Sheet About

I’m admittedly bad at titles, so I’ll summarize what you’re about to read:

Those smile sheets most of us use to “evaluate” an educational event are nothing to smile about.

Refresher: “Smile sheets” are those questionnaires handed out (or worse, left on tables or chairs) for attendees to fill out at the end of an educational session. They usually ask for a numerical rating of the session and/or presenter and are generally limited to less than a page. They’re supposed to help us determine whether the session and its presenter(s) delivered on the promise the attendees felt was given.

Smile sheets don’t work.

• They’re overused
• They’re poorly written
• They’re inaccurate

Overused: We’re busy people, no doubt about that. To be more efficient, we take every possible shortcut, including making a generic smile sheet and using it for all of our events. What’s worse, when smile sheets are the only means of soliciting feedback and results about our programs, we are asking them to do something they’re not designed to do: assess final learning and application of that learning to the workplace.

Poorly Written: Because they’re so generic, they don’t have the specificity needed to generate helpful responses. Asking an attendee if the presenter was knowledgable about the topic or if they will be able to apply what they learned back on the job is useless. Forming useless questions in perfect ways won’t help, either.

Inaccurate: Generic, poorly written smile sheets won’t provide accurate feedback. And if you ask the smile sheet to do more than it can, you’ll believe you’re getting guidance from your members about future programs that they’re really not providing. Smile sheets are not accurate for determining whether learning occurred. It won’t even tell you whether the session had value for those attending.

Why not? Because that’s not what a smile sheet should be designed to do.

What should a smile sheet do? A smile sheet can be very useful for gathering feedback on:

• Location: Was the hotel/conference center conducive to learning? Was the city a desirable place to be for this topic, theme, or session? Why or why not? Answers to location questions can help you determine if you leveraged the local environment sufficiently for your event. If people are going to travel to attend your conference or other events, make it worthwhile. Then find out if it was.
• Content: Asking the attendees to list three things they learned that they will be able to apply back in the “real world” (whether the “real world” is at work, at home, or in other actvities) will help you determine whether any key information was new to the general group, whether any of the content “stuck,” and if what was presented was done in a way that made people believe they could do something with what session covered.
• Format: Were they engaged the entire time? If not, why did they disconnect? Were any questions they had answered?
• F&B: If your sessions include providing refreshments of any sort, ask about them. Was there enough variety? Were special dietary needs met?
• General: If you’ll be using the same location in the future, you should ask about the room and service: was it comfortable? Were they able to stay focused? Were there distractions (for example, did hotel staff disrupt the session when they came in to clear the refreshments?)?

The bottom line is that people will have an immediate reaction to their environment (“That was great! The food was awesome!”), a general idea about what they’ll find useful (“I loved that idea about using Twitter to send my cookbook customers tweets about what I’m ordering at the restaurant for dinner tonight.”), and whether they found the session interesting or not (“That guy seemed to know his stuff, but sitting for 90 minutes and hearing him talk just couldn’t hold my attention, no matter what an expert he is.”).

To find out what your members end up applying back in the real world, conduct separate, follow-up surveys or interviews.

Basing your decisions about content on smile sheets will lead you down the wrong path. And basing your ROI (return on investment) will take you down even worse roads.