Copyright © 1998, Nanette Miner
Permission is granted to reproduce or excerpt this article if proper credit is given in this format:
Reprinted/excerpted with permission. The Training Doctor. Author: Nanette Miner. www.trainingdr.com

Evaluations Ain't What They Used To Be

In theory, the idea behind anonymous evaluations is a good one.  It allows participants to give feedback without fear of argument or retribution, and the trainer should be given critical but useful advice to improve his or her performance -whether that be content or delivery oriented feedback.  In reality however, participants frequently abuse the benefit of anonymity to express pent up aggression and anger toward the organization or the trainer.

For example, one participant in a sexual harassment training workshop stated in the comments section of the evaluation form, ‘this is a blatant attempt by (company  name) to cover its ass, rather than provide real training.’  Other trainers report receiving attacking comments about their style or physical appearance.  Alan Weiss, Ph.D., President of Summit Consulting Group based in East Greenwich, Rhode Island, believes that at least 10-20% of our audiences have a personality disorder which impacts the type of feedback they provide.  In other words, anonymous evaluations are the perfect forum for the passive-aggressives in our audiences.

Precisely because anonymous evaluations allow for the expression of hostility without retribution, participants seem to increasingly use the opportunity for trainer-bashing rather than concentrating on the purpose of the training session.  One training consultant, who asked to remain anonymous, relayed the story of providing an all day training event for a large women’s retail organization.  After submitting a proposal, meeting with the client to finalize the agenda, spending a work-week designing and customizing the training, producing the materials and finally providing the eight hours of training, the participant’s feedback was fixated on the trainer’s style of dress.  The trainer reports that she was wearing a suit and high heels.  The feedback received from the client after the event expressed that the participants felt, because the company had a business-casual culture, that it was inappropriate for the trainer to dress “better” than they, and that her high heels were out of fashion and she should have been wearing  chunky heels.

The client tried to explain away the feedback by saying that in a retail environment, this type of comment is to be expected - that retail professionals are overly concerned with appearance and image.  The trainer felt, that while the retail viewpoint may be true, the participants were there to gain new knowledge, skills, and abilities, and the fixation on aspects that were extraneous to the training left her baffled.  In addition, the trainer wondered why the client even bothered to transmit the information under the guise of “feedback.”

“What benefit is there to tell me, three weeks later, that my shoes were out of fashion?  As a consultant, in establishing the consultant-client relationship,  I frequently ask questions that have little to do with the content, but a lot to do with the participants ability to learn, such as ‘what’s the reading level of your participants?’. The lesson learned from this experience is that I now need to ask the client, ‘what do you want me to wear on the day of the event?’ ”

Lanette Macaruso, a training consultant based in New Britain, Connecticut, says that she sees a real danger in allowing anonymous feedback for two reasons.  The first, which we’ve already alluded to, is the fact that the participant is able to provide negative, accusational and potentially harmful information without having to take responsibility for his or her comments.  A second detriment in allowing anonymous feedback is that it does not allow the trainer to follow up and ask for further explanation for those comments that could truly be an enhancement if the comments were more fully flushed out.

To return to the story of the trainer who provides sexual harassment training, the trainer’s outlook and methodology for the topic is to let employees know what is and what is not considered to be harassing behavior, and to empower individuals to take responsibility for their own comfort in the workplace, as well as to take responsibility for the professionalism of the workplace in general.  During a session for a client, the trainer received a comment that said ‘this course should be more about prevention and less geared toward how to build a case.’  The trainer was dismayed to read the feedback because he didn’t believe that a litigious focus was in any way transmitted during the session.  He would have appreciated the ability to go back to the individual who wrote the note to ask what he had said that led them to believe that he was guiding them in building a case.  His concern is that a future participant might come away with the same impression if he expresses his thoughts in the same way again.  He says, “Had I been able to contact the writer after the session, I would be able to modify my approach or my comments and prevent future misunderstandings.”

In working with his clients, Dr. Weiss has adopted a strict policy: he does not allow his clients to use level one evaluations, period.  “They measure the wrong things, from the wrong people, at the wrong time.  Many are filled with ridiculous, vituperous feedback.  Anyone who reads that stuff deserves what they get.”

Breaking the Cycle

Dennis Cashman, an Organizational Consultant for CIGNA, feels that we sometimes place the power of giving feedback into the hands of people that are not really capable of giving it.  He feels the fault for this lies with the trainer, or the organization, for two reasons.  The first reason is that the questions usually asked on level one evaluations are too vague. Questions that are unfocused, such as ‘was the workbook useful?’, can summon useless feedback such as the comment he received after a time management seminar for a prior employer.  The participant wrote, ‘the paper quality is not as good as the quality of the (outside vendor’s) workbook.’ At first Cashman scoffed at such feedback, but after some thought, said to himself, “Why not?  How would a participant with no training background know what type of feedback I was looking for?  If what I wanted to know was whether or not there was enough whitespace for notes, why didn’t I ask that specifically?”

Cashman’s second complaint regarding evaluations is that he feels they frequently lack a real purpose.  He suggests that a trainer or client who is truly interested in using the feedback gathered, first start by analyzing what the feedback will be used for.  Will it be used to improve the content?  Will the evaluation be used to modify the trainer’s style? Will the feedback help the organization to judge the level of participant skill or learning?

Most evaluation forms are too general and fragmented.  Typically ten questions are asked: a few ask about the trainer’s knowledge and performance, a few ask about the material delivered and a few ask what the participant learned or what s/he might put into practice.       One way to reduce irrelevant feedback is to match the type of feedback requested to one specific objective and to ask more pointed questions specifically worded to generate useful feedback, as in the whitespace question mentioned above.

To Sign or Not to Sign?

Another way to make evaluations more useful is to not allow for anonymous feedback.  Lenny Laskowski, founder of LJL Seminars of Newington, Connecticut, stopped using anonymous feedback three years ago.  He states that providing space at the bottom of the form for the participant’s name, address and phone number has helped his business to grow in three ways. First, he is able to garner positive quotes and attribute them in his marketing materials.  Secondly, he is able to follow up on poor evaluations to ask the participant to expound upon his/her comments.  And third, he is able to target future seminars to those people that gave positive feedback. Laskowski believes that when a participant provides their name, they are more likely to give critical thought to the comments being written.  As Macaruso indicated, forcing the participant to take responsibility for their comments reduces the potential for comments that lash out at the trainer or the organization.

Conversely, Joanne Schlosser, a quality improvement consultant based in Scottsdale, Arizona, encourages anonymous feedback.  “When I am a participant I appreciate the opportunity to give constructive criticism.  I think anonymity gives people the ability to be honest about what they liked and did not like.  I do ask for names on my feedback forms, but I’m very clear that it is an optional field,” she says.

The trainer that requested anonymity stated that the best feedback she ever received was from an individual that took responsibility to a new height.  “I was providing a training seminar for an insurance client in which I had participants from two distinct business lines - personal insurance and commercial insurance.  At the conclusion of the seminar, a woman approached me to tell me that she had given the seminar a low rating but she wanted to further explain her comments.  She told me that she enjoyed the seminar format and liked my presentation style, but that the content really missed the mark in speaking to commercial insurance representatives.  She spent about twenty minutes with me and really helped me to refine the content of the seminar.  I am very grateful to her.  Had I just received her feedback form which said ‘not relevant to my needs’ I would not have known where to begin or why.”

While asking for a signature is easy enough, actually getting one may be another story.  Laskowski says he rarely has trouble getting participants to give their name and address.  He acknowledges that he is in a unique position however, as he is external to an organization and the public attends his workshops on a voluntary basis.

Susan Keen, Ed.D., Director of Corporate Education for a Tucson-based international corporation finds that less than 50% of participants will provide their name - even when it is clearly stated, both in writing and by the instructor, that a signature indicates attendance only and is necessary for ISO9000 documentation.  Through informal information gathering, Keen has determined that participants are reticent to give their name for one of three reasons:

1. Lack of trust.  “What do I mean by that?  Employees have heard talk that doesn’t match walk for so long that they trust no one, including each other.  Thus, when a trainer - even a manager as trainer - says that evaluation information will be kept confidential, no one believes it.”
2. Some employees perceive training as a ‘perk,’ and are afraid that if they sign a negative evaluation, they will never be allowed to go to training again.
3. Fear of reprisal from either the instructor, the instructor’s department or their own management.

Now What?

Where do we go from here?  Unfortunately the answer isn’t black and white.  There are valid, logical, useful reasons for asking for, or even requiring, one’s signature on an evaluation form.  And it can be equally argued that there are valid, logical, useful reasons for not asking for identifying information.  The best approach is to begin by asking yourself, ‘what do I expect to do with the information I gather through this feedback form?’  Beginning with the end in mind will help you to design an appropriate evaluation instrument that truly serves your purpose.