침묵->경청->기억->실행->남을 가르치기

By Scott Berkun, September 2004

Good feedback is rare. It can take a long time to find people who know how to provide useful criticism, instead of simply telling you all the things they think are “wrong” with you or whatever you've made. A good critic spends as much energy describing what something is, as well as what it isn’t. Good criticism serves one purpose: to give the creator of the work more perspective and help them make their next set of choices. Bad criticism uses the opportunity provided by someone else’s work to make the critic feel smart, superior or better about themselves: things that have nothing to do with helping the recipient of the critique (Or in the case of movie reviews, the reader of the critique). Given the difficultly of creative work, it would seem that giving and receiving useful feedback should be an important part of what designers, writers, programmers and others are taught to do. This essay attempts to serve that purpose.

Assumptions bad critics make

There are four fundamental assumptions bad critics make:

1. There is one universal and objective measure of how good and bad anything is.
2. That the critic is in sole possession of the skill for making these measurements.
3. Anyone that doesn’t possess this skill (including the creator of the work) is an idiot and should be ridiculed.
4. That valid criticisms can and should always be resolved.

Let’s work with these one at a time. First, the idea of objective measures runs against everything we know about the history of man made things. To objectively measure how good and bad anything is would require not only that the universe is objective, but that the people in it are objective. There is no film, book, software, website, or album that is universally liked by everyone (including those who have the word critic in their job title). Some people may be more informed or knowledgeable than others, but this doesn’t make their opinions objective.

More important perhaps is the idea of measurement. To measure how good or bad something is requires knowledge about the intent of what the thing is trying to do. If you show me a frying pan that you’ve made, and I criticize it for not playing MP3 files, there’s a mismatch of intention in what we’re trying to measure and evaluate. Unless the intention of the work is clear to everyone “I want to make omelets”, good criticism is impossible. There are an infinite number of intentions and goals in the universe, and if two people can’t agree on what the creators intentions are, real communication is impossible. It might be fair to say that the intentions of a work should be transparent in the work itself: A toaster oven should look vaguely like something that can receive slices of bread. But in the case where the intentions aren’t clear, critics have a choice: they can trust the creator and invest more energy trying to sort out what the intentions are, or they can assume the worst about those intentions and begin criticizing what they don’t understand.

Second, believing that one person has sole possession of good perspective is a contradiction in terms. Good perspective by definition means the recognition of how many alternatively valuable perspectives there are on any matter. Two smart knowledgeable people might both love the new super-thin I-mac design or the recently U.S. released film Hero, but for entirely different and non-overlapping reasons. Good criticism generally comes with some degree of humility and respect for the possibility of other equally valid points of view. The better the critic, the more holistic their sense of how their own perspectives and tastes fit into the diverse pool of informed opinion of others.

Third, respect and ridicule don’t mix well. To offer good criticism must be an act of respect: an act of communication with the intention of helping the other person do better work, or understand their work better. If you are shaping sentences and remarks to be snide, snarky, or sarcastic, the intention of being helpful is unlikely to be served (Unless you know the recipient of the criticisms well enough to be comfortable razzing or joking with them about their work). It’s entirely possible to offer criticism, commentary and advice without any negative energy attached: it’s just so rare that we see it done properly that most of us don’t realize it’s possible, much less more effective.

Lastly, finding a valid criticism doesn't mean that it can be fixed or is worth fixing. In many situations responding to one kind of criticism will just make the design or the work vulnerable to another kind of criticism. A film or essay that is dark and brooding could be made lighter and funnier, but then another critic could say "it wasn't dark and brooding enough". And in some cases, fixing a particular problem will cause other problems that are worse. Until the creator explores the alternatives presented by feedback, it's impossible to know whether responding to a piece of criticism is possible, much less desirable.

Collectively, this means that criticizing and giving feedback should be a thoughtful activity. If you're flippant, arrogant, dismissive, curt or annoyed while giving feedback, you're probably making one of the four assumptions above and not giving very good criticism.

How to give critical feedback

The verb criticize, once a neutral word somewhere between praise and censure, is now mainly used in a negative sense. To say “He criticized me for being so friendly” generally means something different and less positive than “He made me think about the possible effects of being so friendly”.

crit•i•cal (adj.)
1. Inclined to judge severely and find fault.
2. Characterized by careful, exact evaluation and judgment: a critical reading.

Now I’m not saying that finding fault isn’t useful. On the contrary, it’s very important. It’s just that of equal importance in understanding the value of a design, algorithm, script, or film is to know what isn’t broken, or god forbid, what’s actually done brilliantly. What you want to do when you are offering criticism is to live up to the second definition listed above: Careful evaluation and judgment. To do this you need to do the following:

• Before you speak, know the goals: What problem is the work trying to solve? What are the goals? If you don’t know the intention of the work it’s very difficult to offer careful evaluation and judgment. Remember the frying pan? If I don’t know what the creator is trying to achieve, how can I possibly offer any commentary that’s of value? Now it should be the creators job to inform me of what they’re trying to do, or tell me that they think it should be self-evident in the work, but if they don’t there’s not much harm in me asking “What are you trying to accomplish here?”, and it will save everyone much time and grief. If the problem is at the level of intention, discussion will ensue at that level instead of trying (and failing) to sort out intentions at the level of specific design choices.
• Good and bad, is not the same as what you like or don’t like. You must shatter the idea that anything you like is good, and anything you don’t is bad. If you can’t separate your personal preferences from more abstract analysis of a kind of work, then you will rarely provide much useful feedback: criticism is not about you. It’s about the work you are viewing and the person that made it. Your personal preferences only get in the way of providing the work (and its maker or possible consumers) with useful information. Learn to see the good and respectable attributes in work you do not like: they are there if you let yourself see them. For example: a good film review should evaluate the film's merits somewhat independently from the reviewers personal tastes. It should be possible to read a review about a film the reviewer didn’t like, but be inclined to see it anyway based on the observations he made about it’s content, style, and form.
• Talk as much about what it is, as what it isn’t. While it can be more efficient to focus on problems and what’s broken, rather than what’s good and working, if the creator can’t see both, there’s not much hope of their next choices being good ones. Make sure you spend as much energy helping them to see and keep the strong parts of what they’ve done as you help them see the weaker and more questionable parts.
• Try the PNP sandwich (positive negative positive): I don’t like this idea much, but I think it can be a good one (see what I did there?) for dealing with people sensitive or new to receiving criticism. The idea is simple: find a way to alternate your feedback. Find something positive, then find something negative, then find another positive thing. It’s an easy way to develop trust and help people become comfortable with hearing other people’s opinion. I don’t like it because it has a touchy-feely vibe and it can lead to pretension and insincerity. However I have seen it work as a way to get strangers to warm up to each other, and eventually grow out of this little pattern of behavior.

Receiving critical feedback

It’s much harder to receive criticism than to give it. By the time most people make it through college there have been so many bad experiences with receiving feedback, especially on creative work, that they tend to avoid it or ignore it. Nothing can be worse: feedback is essential to developing ideas, and if the project involves a team in any way, the dialog and communication that falls out of feedback is essential. Anyone that makes anything must find ways to not only obtain feedback, but to master the skills of milking it for all it’s worth.

• Shut up. Just shut up and listen. Creators often fall into the trap of speaking for their work, trying to use words to defend things that should be in the design. This is a form of denial: The work has to speak for itself. Even if only for a few minutes, let the prototype or draft be its own thing, and stand on its own. If you respond right away to (or perhaps interrupt) every point made in a critique, you can’t possibly be thinking about what’s being said to you. Thinking takes time. Try to talk as little as possible, and let the time be used for critique, not for defense. If you don’t trust the people critiquing you to be fair, that’s a problem best solved by defining sound ground rules (See below), or by investing more in finding better critique partners.
• Ask clarifying questions. Again, avoid filling the conversation with defensive chatter. Instead respond to questions by trying to sort out any ambiguities or points you don’t agree with by getting whoever is critiquing to restate their point. “When you say the style in my design is sloppy, do you mean that the lines aren’t sharp, or that the composition isn’t balanced quite right? Can you show me exactly what you mean?” By asking clarifying questions you allow yourself time to decide if you agree with the criticism or not by working with the other person to understand their point/question better. It makes the critique into a dialog, which is what it should be, and not a courtroom trial.
• Refer back to the goals. If you’re not getting what you want from the critique, provide some goals for the work that you’re trying to achieve. If you’re working on a project this should be easy: the goals for a given design should derive from the project goals. Ask whoever is giving you feedback to do so in terms of those goals or your derivations of them. Then whenever the conversation goes astray, you can refer back to the goals and set things in a useful direction again.
• Ask for what changes you can make that will satisfy the criticism. The goal of criticism is not to learn every nuance about a design’s weaknesses: its to know enough about a design so that the designer can make it better. If you agree with a criticism, but don’t see a path to improvement, ask for one. Turn the question back around on the person who made the comment. “Good point. So do you see anything I can do to improve on that?” Often they won’t have anything to say: critiquing is not the same as creating. But by asking the question you do move the conversation forward into thinking about future action, instead of staying stuck in criticism mode.

Ground Rules

• Take control of your feedback process. Feedback is not something that happens to you: it should be something you make happen. If you wait for feedback to come to you, it tends to be less positive and supportive than if you seek it out. If you walk into someone’s office and say “hey, can I have 5 minutes of your time to look at something?” you are taking control. You put yourself in the driver’s seat of the process, and can frame and shape the criticism you get however you want. But if you wait and wait and wait until deadlines approach, you have less and less control over how feedback will be given to you. It will have more edge to it and will tend to serve others more than serve you.
• Pick your partners. Who do you get the best feedback from? It’s probably not the person who loves everything you do. If you don’t think you get good feedback from anyone, part of the problem might be you haven’t taken control of the process. Be more specific about what kinds of criticism you need, and go to people and ask for it. If you find a good source, cherish it, and reward them for it. Much of what a good mentor does is provide good, consistent, honest feedback. If you can get this from a peer or a manager find ways to cultivate and reward it. Look for people outside of your company or organization that might be willing to form a peer review group: meet once a week/month over coffee and show each other your work.
• Strive to hear it all, informally and early. The sooner you hear a question or criticism of something you’ve created, the greater your ability to do something about it before it’s finished. If there is any kind of formal review or feedback process (e.g. a spec review or group critique) make it your job to find out what opinions there are of what you’re doing well before it happens. This can be as simple as going to door to door and showing sketches, and asking for a few quick comments. Give yourself the opportunity to benefit early from other perspectives and think things through. But do know how much feedback you can handle: you don’t want your work driven by other people’s opinions, but you do want to give yourself the opportunity to benefit from them.

References

Boeing Win on A-10 Contract Would Have $135 Million Impact on Georgia
Thursday January 25, 12:00 pm ET

"The Boeing A-10 program would manufacture the center wing panel structural assembly at our Macon facility if we win the A-10 contract. That work includes parts kitting, drilling, assembly and painting," said Curt Nothstine, director for Aircraft Sustainment and Modernization for Boeing's Maintenance, Modification and Upgrades business. "By placing the work in Macon, we can leverage the outstanding performance of the local workforce that we have seen on the C-17 program. We would expect to add new jobs by 2011."

The A-10 wing replacement program calls for the replacement wing sets to be delivered in parts and kitted for easy installation. The Air Force is expected to evaluate competitors on past performance, capability, total evaluated price and program schedule, before awarding the contract in Spring 2007.

"Our plan is to provide the Air Force with the skilled expertise and engineering know-how that will address the needs of the A-10 program with a low-risk, low-cost solution," continued Nothstine. "Using advanced lean manufacturing technologies, we will be able to provide a solution and assembly kit that will allow the A-10s to fly at least 20 more years. The outstanding past performance on the C-17 manufacturing work at the Macon facility provides Boeing with a strong base from which to compete for new work."

The contract is valued at up to $1.5 billion over two five-year periods. The company plans to announce Georgia-based suppliers within the month and anticipates similar financial gains in the states of Utah and Missouri.

The A-10 Thunderbolt II, first introduced in 1976, is a twin-engine jet aircraft designed for close air support of ground forces. The simple, effective and survivable single-seat aircraft can be used against all ground targets, including tanks and other armored vehicles.

A unit of The Boeing Company, Boeing Integrated Defense Systems is one of the world's largest space and defense businesses. Headquartered in St. Louis, Boeing Integrated Defense Systems is a $30.8 billion business. It provides network-centric system solutions to its global military, government and commercial customers. It is a leading provider of intelligence, surveillance and reconnaissance systems; the world's largest military aircraft manufacturer; the world's largest satellite manufacturer; a foremost developer of advanced concepts and technologies; a leading provider of space-based communications; the primary systems integrator for U.S. missile defense; NASA's largest contractor; and a global leader in sustainment solutions and launch services.

Boeing has unexpectedly ditched plans to fit leading-edge wireless in-flight entertainment (IFE) technology to the 787, but insists the move to the more conventional hard-wired replacement system will not impact either schedule or cost.

News of the change, which the company says was only firmly decided on in the second week of January, comes just two days after an industry analyst issued a disputed report saying some 787 customers had been told their aircraft deliveries could slide.

Boeing denies the Wachovia Capital Markets report and insists the schedule remains firm. Boeing says “there are no delivery delays in 2008 and we are still scheduled to meet entry-into-service in May 2008”.

The first flight remains on track for the end of August 2007.

Boeing’s 787 systems director Mike Sinnett says the “hard decision” to reject wireless IFE was made “because a couple of things converged on us”.

Boeing could not get 100% international confirmation from countries around the world to allocate frequencies in the IFE system’s 5GHz operating bandwidth. The frequency issue, he adds, was due to several countries requiring the bandwidth close to the IFE frequency for various air traffic, weather radar and military requirements. “We got 99% complete, but there are a couple of places in the world where those frequencies are already allocated to other uses.”

Concerns were also raised about the ability of the wireless chipset technology to use the same frequencies for multiple uses, and for it to keep pace with the expected growth in volume of seat-back content.

The IFE wires will now run to each seat through the adjacent seat track using similar housings developed for the power supply. “In the end we did not add a whole lot of weight. In fact, when we look at the antenna modules we had for each seat group, we end up with more than 45kg (100lb) of weight saved,” says Sinnett.