|
|
 |
[ Current Topics | Complete List of All Active Topics | RSS feed | Search ] |
|
|
|||||
|
PowerPoint Does Rocket Science--and Better Techniques for Technical Reports
-- Edward Tufte, September 6, 2005 |
|
How to make engineers write concisely with sentences? By combining journalism with the technical report format. In a newspaper article, the paragraphs are ordered by importance, so that the reader can stop reading the article at whatever point they lose interest, knowing that the part they have read was more important than the part left unread. State your message in one sentence. That is your title. Write one paragraph justifying the message. That is your abstract. Circle each phrase in the abstract that needs clarification or more context. Write a paragraph or two for each such phrase. That is the body of your report. Identify each sentence in the body that needs clarification and write a paragraph or two in the appendix. Include your contact information for readers who require further detail. -- William A. Wood (email), September 8, 2005 |
|
This doesn't exactly fit `rocket science' but I was not sure where else to put it. A better title might be "Power Point does lifesaving--NOT." If you want to see low levels of useful data per slide, let alone irrelevance to the task at hand [presumably saving lives threatened by Katrina], you could hardly beat those referred to below. "On lunch at work, but still would prefer no identification if referenced. Thought you might like to experience what our elite fire and hazmat volunteers are going through. This is insane. "It appears that all people under FEMA for over two weeks must take "awareness and prevention of sexual harassment," "equal rights officer orientation," and "valuing diversity" training programs. The programs total 3-4 hours. "The mandatory training matrix is here: http://training.fema.gov/EMIWeb/downloads/Mandatory05Matrix.doc "The letter adopting the matrix is here: http://training.fema.gov/EMIWeb/downloads/Mandatory%20TrainingII.doc "Powerpoints for the trainings: http://www.training.fema.gov/emiweb/downloads/DF506/DF%20506%20Sexual%20Harassment%20Visuals.PPT http://www.training.fema.gov/emiweb/downloads/DF416/DF%20416%20Diversity.ppt -- John Liljegren (email), September 8, 2005 |
|
I wasn't surprised to see my remark about Nature and Science contested but I was surprised to see who did the contesting, because what I see as the faults of these journals are exactly the sort of faults that you often criticize in your books. They both publish a lot of work that is fashionable and apparently exciting, but they don't insist on including the supporting information that allows readers to know exactly how the experiments were done -- half the time they wouldn't even allow authors to include this information because they would say it made the article too long. What happens in practice, therefore, is that high-profile authors will publish a claim-staking exercise in Nature or Science and then, if you are lucky, follow it up later in a journal of lower prestige with a "full paper" that includes the essential details omitted the first time round. Let me quote (in suitably anonymized form) an e-mail that I received last week from a distinguished colleague in the US: Thank you for bringing the (journal) paper by (authors) to my attention. I have not been keeping up on the literature (relevant comments below) and was not aware of it but absolutely agree with it. The paper by (other authors) (Nature, 2003) is pure bullshit, and the editors and reviewers responsible for letting it be published in Nature should hang their heads. (Name) and coworkers have hit the major problems (a totally incorrect assay procedure, highly suspect immunoblotting results) on the head in the third paragraph of his discussion.... This is just one isolated example, of course, but one out of many that one could cite. -- Athel Cornish-Bowden (email), September 11, 2005 |
|
Scientific articles as a model for technical reports Moving from the PP slide-format to the Nature-style concise report would be an enormous improvement for any applied technical organization. Fretting about the differences between Nature and the Proceedings of the National Academy of Sciences is not relevant to improving NASA workaday technical reporting. Nature and Science publish about 15,000 authors and co-authors a year, which means, given their high rejection rate, they disappoint perhaps 50,000 aspiring contributors each year. That is a lot of enemies to make. Nature and Science rejections probably annoy and bruise more scientists than all other scientific journals combined. Very few scientific publications have high rejection rates, in large because publication is financed by page charges (not unlike a vanity press), paid for by the author's research grants or institution. Just about everyone who has attempted to publish has their own personal collection of injustices to retail. These horror stories describe biased, incompetent, envious referees and idiotic editorial decisions--at least at every journal with a rejection rate greater than 0%. The anger and the whiny sense of entitlement occasionally exhibited by rejected authors can become rather intense (even experienced on this little board with a contribution acceptance rate of about 30%). Publication horror stories and associated gossip are rampant in the social sciences and humanities, where rejection rates for the top journals routinely exceed 90%. Talking to journal editors, not just the multitude of rejected authors, will fill out this picture. Over the years I've served on a dozen editorial boards of research journals and have gotten a pretty good idea of selection processes. This wisest thing I heard was from the editor of journal with a 1 in 20 acceptance rate: it is easy to identify and reject the 90% of the submissions below the line; but for the top 10% (only half will be published), it's a lottery (depending on the quirks of the referees). For NSF proposal reviewing (at least in the social sciences), there were usually just a few star proposals, which then led reviewers to ask how deep in the pool of routine dustbowl empiricism do we wish to dip? For journals and for grants, overall I was impressed with the care and integrity of the selection processes; I think most of us involved, other than the true believers, were seeking to find something, anything, that was good, novel, and true. For marginal submissions, those on the edge of accept/reject, non-meritorious factors tend play a more important role in the decision (as it also does in faculty hiring in my experience). The performance of a journal must be measured in aggregate and not merely by the embittered anecdotes of the rejected; that is why citations per published article and circulation numbers are relevant. Measured by citations per article, Nature and Science are close to the top, sometimes at the top. And they are by far the most widely circulated scientific journals. A measure of overall system performance is whether every minimally competent article gets published somewhere, if not in the most-cited and widely circulated journals. That is surely the case, since the median number of citations resulting from a published scientific article is zero.
-- Edward Tufte, September 11, 2005 |
|
Will Microsoft improve PP? The record for incremental reform in the cognitive style of PowerPoint is not promising. In the many release versions of PP, the intellectual level has not been raised. New releases have drifted toward ingrown self-parody, featuring ever more elaborated PP Phluff and presenter therapy. These changes have made the new version different from the previous version, but not smarter. There are no incentives for meaningful change in a monopoly product with an 86% gross profit margin, only incentives to make it different, somehow, from the previous release. PP competes only with itself. -- Edward Tufte, September 20, 2005 |
|
Unfortunately, NASA is not re-evaluating the use of PowerPoint, instead, the "MINIMUM INTEROPERABILITY SOFTWARE SUITE" requires all federal employees and contractors employed at NASA centers to have a current version of MS Office installed on their desktop computer. Back of the envelope cost*: $13,200,000 every time a new version comes out. Appropriately, the justification for these standards is contained in a PP presentation. For masochists: http://desktop-standards.nasa.gov/ * 60,000 on-center employees X $220/upgrade = $13,200,000 -- Robert Simmon (email), September 21, 2005 |
|
NOAA hurricane reports better than PP
Here's a counterpoint to PowerPoint: the current NOAA National Hurricane Center
forecast discussion for Hurricane Rita. http://www.nhc.noaa.gov/text/refresh/MIATCDAT3+shtml/ 211447.shtml HURRICANE RITA DISCUSSION NUMBER 16 NWS TPC/NATIONAL HURRICANE CENTER MIAMI FL 11 AM EDT WED SEP 21 2005 THE RECONNAISSANCE PLANE WILL NOT BE IN THE AREA OF RITA UNTIL LATER THIS MORNING. HOWEVER...SATELLITE IMAGES INDICATE THAT THE CLOUD PATTERN IS TYPICAL OF AN INTENSE HURRICANE WITH A CLEAR EYE SURROUNDED BY VERY DEEP CONVECTION. INITIAL INTENSITY IS ADJUSTED UPWARD TO 120 KNOTS AT THIS TIME. HOWEVER...OBJECTIVE T-NUMBERS FROM BOTH TAFB AND THE UNIVERSITY OF WISCONSIN CIMSS ARE PEAKING NEAR 7.0 ON THE DVORAK SCALE...SUGGESTING WINDS OF NEAR 140 KNOTS. I WILL WAIT FOR THE PLANE TO REACH RITA TO INCREASE THE WINDS FURTHER...IF NECESSARY. THE ENVIRONMENT IS CONDUCIVE FOR STRENGTHENING AND RITA...AS KATRINA DID...WILL BE CROSSING THE LOOP CURRENT OR AN AREA OF HIGH HEAT CONTENT WITHIN THE NEXT 12 HOURS OR SO. THIS WOULD AID THE INTENSIFICATION PROCESS. THEREAFTER...THE INTENSITY WILL BE CONTROLLED BY CHANGES IN THE EYEWALL WHICH ARE DIFFICULT TO PREDICT. THE HEAT CONTENT IN THE WESTERN GULF OF MEXICO IS NOT AS FAVORABLE AS IN THE AREA OF THE LOOP CURRENT SO SLIGHT WEAKENING IS ANTICIPATED....BUT RITA IS EXPECTED TO MAKE LANDFALL AS A MAJOR HURRICANE...AT LEAST CATEGORY THREE. THERE HAS BEEN NO CHANGE IN THE STEERING PATTERN. RITA IS MOVING WESTWARD AT 11 KNOTS SOUTH OF A STRONG HIGH. AS THE HIGH MOVES EASTWARD...RITA WILL GRADUALLY BEGIN TO MOVE TOWARD THE WEST- NORTHWEST AND NORTHWEST BASICALLY TOWARD THE TEXAS COAST. THE OFFICIAL FORECAST IS VERY CLOSE TO THE MODEL CONSENSUS AND HAS NOT CHANGED FROM THE PREVIOUS FORECAST. BOTH THE GFS AND THE GFDL SUGGEST THAT THE WIND FIELD WILL EXPAND. THEREFORE THE FORECAST WIND RADII HAVE BEEN ADJUSTED ACCORDINLY. ON THIS TRACK AND DUE TO THE LARGE WIND FIELD ASSOCIATED WITH RITA...A HURRICANE WATCH WILL LIKELY BE ISSUED LATER THIS AFTERNOON OR TONIGHT. FORECASTER AVILA FORECAST POSITIONS AND MAX WINDS INITIAL 21/1500Z 24.3N 85.9W 120 KT 12HR VT 22/0000Z 24.5N 87.9W 135 KT 24HR VT 22/1200Z 25.0N 90.0W 130 KT 36HR VT 23/0000Z 25.7N 92.0W 125 KT 48HR VT 23/1200Z 26.6N 94.0W 120 KT 72HR VT 24/1200Z 29.0N 96.5W 100 KT...INLAND 96HR VT 25/1200Z 32.5N 97.5W 40 KT...INLAND 120HR VT 26/1200Z 35.5N 97.0W 25 KT...INLAND -- Robert Simmon (email), September 21, 2005 |
|
The Rita forecast is certainly a big improvement on a typical PowerPoint presentation, but it's not beyond criticism.
However, I agree that it conveys reasoning and uncertainty in an honest manner, and that it is clearly intended for readers who think about what they are reading. -- Athel Cornish-Bowden (email), September 21, 2005 |
|
My guess is the NWS all-caps style 1) is left over from teletype days and 2) remains because there's quite a bit of overlap between NOAA and the Navy's oceanographic community and the Navy still uses a lot of all-caps Courier. At this point I think they keep all-caps out of nostalgia. As a New Orleans evacuee I have come to exclusively rely on NOAA reports. Interestingly I'm now following Rita closely because Tulane's medical school was supposed to reform in Houston this Saturday. That's now postponed until 1 October. I'm also staying in College Station, which is expected to get some of the wind and rain. -- Niels Olson (email), September 21, 2005 |
|
The NWS caps style is also re-inforced by the fact almost all military orders (pick any service) are created in this teletype format. There is a argument that is eliminates a degree of ambiguity, at the expense of readibility. In other words, a capital letter contains the same meaning as a lower case letter. There is no implied difference of meaning based on capitalization. But historically, it is based on telegraph and teletype styles. The format and ordering of the paragraphs can contain significance. In military communications there are generally headers and footers to tell you where you are in the message, what information to expect next, and that you have reached the end of the section or message. Assuming that you have the rosetta stone to decode often obtuse headers and footers. By the way, this subject is very topical to me because we are frequently driven to produce reports in powerpoint rather that word. I usually object to call a powerpoint presentation a report; prefering to call it a brief or presentation. It is a very common to produce a powerpoint brief with notes pages and print the combination as a report. Even that combination leaves a lot to be desired in presenting complex issues, because all you have really done is add a text area about the same size as the powerpoint slide to amplify the slide. This, of course, means that the slide is probably not understandable to someone who is not already intimate with the subject material. But even in this case the notes pages very frequently suffer from the same choppiness. I work in an organization where we are frequently more motivated to produce emerging findings quickly rather than conducting thorough analysis to produce quality findings. This means not only is the powerpoint brief terse and choppy, but often the results are misleading or wrong. Not to mention contentious, because it is nearly impossible to vet the findings and achieve concensus in a short period (such as less than a week in the Columbia case). I am almost sure that this situation existed in the Columbia case. And, as in my organization, you can almost assure that none of the results presented are accredited at the time they were first presented. -- Clyde Smithson (email), September 30, 2005 |
|
After thinking about this over the weekend the thought struck me that the NWS style messages are more a function of data packing than anything to do with readability. Since these messages are transmitted across an electronic network that has a fixed data rate it is more important to reduce the character set so as to provide greater message bandwidth. It is probably the case that the engineers have employed a bit packing technique to maximize the amount of data carried in a word. Typically 8 or 4 bytes, 64 or 32 bits, but 2 or 1 bytes are encountered as word length of data transmitted. In a bit packing system if a value is binary (say on or off) only 1 bit is needed to represent the data, and so on based on the number of states. So why waste more data than needed. To go back to the telegraph example, Morse code has 39 defined characters (26 letters, 10 numerals, and 3 special characters) so would require 6 bits (2 to the 6th bits = 64) to represent in a digital message. This means that in a 4 byte word (32 bits) that you could represent 5 Morse code characters and you waste 2 bits of information, a 6.25% waste. Standard ASCII, which does contain lower case and more special characters, has 128 different characters (2 to the 7th bits). A 4 byte word could contain 4 Standard ASCII characters but wastes 4 bits of information, 12.5% waste. Extended ASCII has 256 characters (2 to the 8th bits) so a 4 byte word contains exactly 4 characters. Engineers sometimes deal with the lost bits by using them to contain other information such as headers, or sometimes split characters across multiple words. This, of course, requires more software at either end of the message to encode and decode the information. But from an economy of scale ASCII takes twice the bandwidth of Morse code and Extended ASCII takes four times the bandwidth. This does matter even today with high speed internet because much of the government and military infrastructure runs through older systems with lower data rates. Additionally, as both institutions become more network-centric, our desire to put more data through the network approaches or exceeds the capacity of the faster networks to carry all this data. This extra data flowing across the networks is not the equivalent of extra knowledge. The NWS example would provide no more information to the reader if mixed upper and lower case were used, but would consume 2 or 4 times the "data" space if ASCII or Extended ASCII were used. For some history (http://en.wikipedia.org/wiki/ASCII#History) -- Clyde Smithson (email), October 3, 2005 |
|
Return to Flight Task Group biographies Here are the biographies of those on the Return to Flight Task Group who saw the NASA engineering by PowerPoint and denounced it in their final report (quoted extensively in the last 2 pages of my essay). As I wrote in the essay above, "Both the Columbia Accident Accident Investigation Board (2003) and the Return to Flight Task Group (2005) were filled with smart experienced people with spectacular credentials. These review boards examined what is probably the best evidence available on PP for technical work: hundreds of PP decks from a high-IQ government agency thorough practiced in PP. Both review boards concluded that (1) PP is an inappropriate tool for engineering reports, presentations, documentation; and (2) the technical report is superior to PP. Matched up against alternative tools, PowerPoint loses." Here are the biographies of the NASA PowerPoint critics:
-- Edward Tufte, October 18, 2005 |
|
For an excellent technical report on a complex engineering matter, see the NASA report on the foam loss from the external tank during the recent launch of the Discover (STS-114). The report is written in sentences and paragraphs, not bullet-point grunts and slides. The report presents many extraordinary images of the tank post-launch. The difficult analytical issue is the lack of comparisons of the tank conditions from the previous 112 launches. Here is the link to the report http://www.nasa.gov/mission_pages/shuttle/main/ Then go to the link "full report, 2.1 MbPDF". -- Edward Tufte, October 23, 2005 |
|
What a stunningly beautiful piece of work, thank you. This note needn't be considered for the thread. Just a couple of clean-up ideas, in addition to "are be...," it may be good to look at: -- 5 grafs from the very bottom, "In the final report, 7 of Task Group Members... " -- First graf below the detailed slide analysis, last line: "This is a lot of insecure format... ." -- The "Marketing Strategy - Good4U" PP slide seems orphaned. Thank you so much for the close work of your team. DJ -- David Johnson (email), November 17, 2005 |
|
Many of the contributions to this thread are helpful -- especially those that point out flawed PowerPoint design. Has anyone found an example of skilled information design in a PowerPoint presentation? I feel that we can only learn so much from examples of what NOT to do... -- Scott L. Mitchell (email), December 9, 2005 |
|
PP is a competent Projector Operating System for full screen images and videos, replacing the little forward-back button in old-fashioned projector systems. PowerPoint is neither the best nor the worst Projector Operating System. It faces strong competition from the projector itself with its own forward-back controls. A Projector Operating System, however, should not impose Microsoft's cognitive style on our presentations. PP has some low-end design tools helpful in constructing PowerPoint parodies. PP might also help show a few talking points an informal meetings, but why not instead print out an agenda on a piece of paper? PowerPoint may now and then benefit the bottom 10% of all presenters. PP forces the really inept to have points, some points, any points.
-- Edward Tufte, December 9, 2005 |
|
One of my professors, Dr Sandor Vigh, embeds what he calls zoomimage graphics in his PP presentations. I can't say I've seen anyone else do this and I haven't used Quicktime myself enough to know the buttonology how-to, but he basically puts a very high resolution image into a Quicktime frame that occupies the entire PowerPoint slide. So it's a slide projector. But he can zoom and pan! So he scans in Nettergrams, histology slides, whatever, at, say, 3000x2000 and then explores the details of the image while the projected image is always above the resolution threshold of the projector. This is vastly superior to 'mere' projection, despite the LCD projector. And look, it displays on the web! Use the mouse to pan and shift/ctrl to zoom. Zoom in on the bones of the intermediate phalanges of the four fingers. Compare the fine lattice work of trabecular bone in the ends to the dense hollow tubes of cortical bone in the shafts. -- Niels Olson (email), December 9, 2005 |
|
To follow up on Niels Olson's comment, the precise technology being used is Quicktime VR. Quicktime VR is typically used to stitch together 3-D panoramas or a 3-D view of a solid object - like Niels, I haven't used to to display two-dimensional images in this manner before. Dr. Vingh's use of it in his slides is an interesting end-run around the resolution limits of a normal computer presentation. -- Zach Heaton (email), December 15, 2005 |
|
Elizabeth Lane Lawley, a professor visiting Microsoft, comments on "the culture of the deck": http://mamamusings.net/archives/2005/11/19/the_culture_of_the_deck.php Her experience at Microsoft is comparable to that of the NASA Return to Flight Task Group with regard to the persistent disutility of using PP decks to replace technical reports. -- Edward Tufte, December 29, 2005 |
|
PowerPoint Does New Orleans All but one of the committee 'final reports' for Mayor Nagin's Bring New Orleans Back Commission are out. Most are in PowerPoint. The report on levee recommendations is the lone exception. NOTE ADDED, JULY 1, 2006 BY NIELS OLSON: Does plain text or PowerPoint tell the story better? Compare Howard Reich's piece in today's Chicago Tribune, Crisis of culture in New Orleans, to the official Culture Committee report of Mayor Nagin's Bring New Orleans Back Commission. -- Niels Olson (email), January 20, 2006 |
|
On the subject of writing, a favorite professor of mine said: "What's in the head goes on the page." The "final reports" to which Niels Olson links are astonishingly awful, and ought to be required reading for the those who would defend the general utility of Power Point. That these are "final reports" -- not merely tools to supplement the oral presentation of conclusions from actual written reports -- suggests that an antiliterate (I was going to write subliterate, but the word is insufficiently strong) approach now dominates public policy discussion in the United States. What a shame. -- Alexey Merz (email), January 20, 2006 |
|
Cognitive Style of PowerPoint 2nd edition now published Just published is the 2nd edition of The Cognitive Style of PowerPoint. It is now 32 pages long; the original essay was 24 pages. The new edition contains the material on PowerPoint and rocket science that opens this thread, a long discussion of the causes of presentations (sorting out variation among users, content, and presentation methods), and an essay on lists. This new edition is also a chapter in Beautiful Evidence.
For more information and to order click here. -- Edward Tufte, April 25, 2006 |
|
Thank you thank you thank you! I am an environmental scientist, and my company does environmental impact assessment 1-10 day training workshops -- WITHOUT Powerpoint! We have a programmer who prepares an occasional animated presentation in Flash (which is also helpful because it is only images, so we can offer it in any country, in any language) -- so I will have a projector in the room, and you should see everyone's face fall when they first enter and see the projector, and watch them light up when I tell them it is only there to scare them! Our trainings use case studies, flip charts, an occasional video or a cartoon on an overhead during the break, and music...NO Powerpoint or even Apple's lovely and way easier and more beautiful implementation of a similar system, called Keynote. I try and try to explain to prospective clients why we don't use it -- but sometimes we are actually required to use it by, gasp, their TRAINING departments....anyway, this is a long- winded way of saying that the PP disease is very hard to cure but there are many of us who simply refuse to be infected...and thanks for providing some good ammunition to use with our clients. -- Leslie Wildesen, Ph.D. (email), June 23, 2006 |
|
It's not just reports that suffer. I've recently attended a "technical" training course in which we were presented with an enormous number of PowerPoint slides, most of which were merely read aloud. Not only was this course mind-numbing to teach — I can only pity the instructor — it's also an active waste of time. At the end of the course we were given a book containing one slide per page, along with a small amount of notes below. I've struggled to find the worst of these slides, and I've recreated one of the candidates here (90k PNG). I'm assuming direct scans would be forbidden, and I've neglected to include the "content" provider's logo. I am still unable to find any shred of meaning in this slide, or many of the others. -- Dan Avis (email), June 29, 2006 |
|
DISCOVERY LAUNCH Here is a link to William Harwood's excellent account of shuttle risks in the upcoming flight, scheduled for this Saturday. This link provides context for my comments that follow. http://www.spaceflightnow.com/shuttle/sts121/060629preview/part1.html ____________________________ About 18 months ago in Houston I reviewed the shuttle Probability Risk Assessment (PRA) material for NASA. PRA works with a list of possible threats, estimates their probablilities and expected losses, and then seeks to assist decision-making for shuttle risk-reduction. After the PRA group presented their results, I had two major suggestions: (1) They should prepare a detailed summary matrix (on, of course, 11" by 17" paper), ordering the risks and providing, in a comments column, relevant background for each estimate. Let that intense matrix, backed up by similar more-detailed 11" by 17" arrays of risk estimates, be the main presentation device and analytical tool for making decisions. This was designed to replace their chippy and twiddly PP slides, which made a hash of their good technical work and made it difficult to assess the overall risk context. (2) The PRA assessments did not take into account a major risk factor in both the Challenger and Columbia accidents: on-ground intellectual failures in engineering analysis. In the case of the Challenger, the analytic process on the day before the accident was seriously deficient, in the sense that--in hindsight to be sure--the Challenger would not have been launched on that very cold day (which compromised the O-rings and caused the accident) if smarter engineering analysis and better decision-making had taken place. In the case of the Columbia, better analysis and decision-making during the flight might have yielded rescue efforts to try to save the crew, which was endangered by damage to the Columbia suffered at launch. I suggested to the PRA group that on-ground analytic problems contributed to something like 1.3 of the 2.0 accidents in the 113 flights. But there was no risk assessment of such in the PRA; that is, about 65% of the directly observed empirical risk in the 113 flights was not accounted for by the PRA model. The shuttle itself was considerably less risky than what was happening on the ground in decision-making about the shuttle. At the meeting, I also handed out Richard Feynman's famous discussion of shuttle risks, which Feynman prepared as a part of the Challenger investigation in 1987. ____________________________ The analysis for the upcoming launch of the Discovery in July 2006, as the link above indicates, was an intense evaluation of risks and trade-offs. On the basis of reading some of the public documentation (and no direct knowledge) for the upcoming flight in the last few weeks, I think that NASA has made a reasonable and well-informed decision for the upcoming flight. It was also a contested decision. I would vote for the launch. The on-ground factors that contributed to 1.3 shuttle losses appear to be mitigated by the thorough analysis for this flight. The current risk number is a cloudy 1 in 100, which is risky but has been acceptable in the past. The cloudy contributions to risk are the recent changes in the foam, which turns Discovery into something of an experiment. In the Discovery discussions, a telling distinction was made between "programmatic risks" and "crew risks." The programmatic risk is very high right now no matter what happens. Having flown once in 3+ years, the shuttle program might well collapse if unable to fly soon (within a year or so), or if there is another accident even if the crew escaped unharmed. This rescue scenario is itself troublesome, since the rescue launch must quickly take into account what caused the need for the Discovery crew-rescue in the first place. -- Edward Tufte, June 29, 2006 |
|
DISCOVERY LAUNCH Another detailed and excellent account by William Harwood on the eve of the flight: http://www.spaceflightnow.com/shuttle/sts121/060630griffin/ There is a Feynman-like clarity to the Discovery analysis done by Michael Griffin, NASA's director. Now it just has to be confirmed empirically! ET, Saturday, 6.53am -- Edward Tufte, June 30, 2006 |
|
POWERPOINT FOR DISCOVERY FLIGHT READINESS REVIEW: THE FOAM SLIDES, OR "MAKE THAT CALL NOW, THAT'S 1-800-. . . . " http://images.spaceref.com/news/2006/2006.06.16.ET_FRR.pdf This pdf file should be up in a separate window to read in parallel with the comments below: These slides summarize the results of the enormous amount of resources (probably >$1 billion, some estimates are much higher) devoted to the external tank foam problem. The slides do not display a sense of engineering intelligence or discipline. In the main report, there is a persistent habit of dequantification and a general absence of units of measurement. The back-up slides are more quantified and at a higher intellectual level. Several of the slides look like they were produced by a designer lacking in scientific training.
The key overview slide (page 3) is a very good idea but a presentation mess. The good idea is to have an intense and fairly detailed summary early in the presentation. But PP's lightweight resolution and lousy design tools compromise the summary slide. Students of PP design might, however, appreciate the 5 sets of orange drop-shadows, 4 wavy-purple color fields, 3 unintentionally 3D blue time-lines, 2 overactive grids, and floating-off-in-space bullets in the highlight box (with an arbitrary change from dots to dashes midstream in the box). All this stuff on one over-produced but importantslide. In real science, every photograph has a scale of measurement built right in to the photograph. This low-resolution display method makes it impossible to do so. (Even the shuttle close-out photos, just about the most documentary type of photographs one can imagine, have no scales of measurement and no rulers in the pictures.) The bullet lists tend somewhat to be base-touching grunts, which show effects without causes, actions without actors, verbs without subjects, and nouns without predicates. The branding with 3 logos on every slide (the title slide has 4 logos) is unprofessional, pitchy, turfy. Are we doing engineering analysis or marketing here? Some 20% of the space of every slide (already a a very low resolution display method) is devoted to branding and to the boxed-in awkward and repetitive slide titles. It is as if each and every slide has to remind the viewer what the presentation is about. So the top 20% of every slide is something to skip, perhaps putting some viewers in the mode of skipping and sliding through the rest of the slide. It is as if the top of every slide announces "nothing important here, you've seen it all many times before." In several slides, the visually most active materials are the cross-hatched exploding 3D arrows linking the external tank to the magnified areas. Why are the arrows pointing anyway? It's just a simple linking line. The idea here of close contextualized imaging of the problem areas is a very good one, but the badly-drawn giant blue arrows are silly, and result in making the dequantified images of the foam problem areas too small. The typography is poor, with odd hierarchies (underlined bold italic in parentheses at one point). Is "O2" the proper way for NASA contractors and NASA to write the oxygen molecule (even wikipedia uses a subscript)? Does the slide designer know how to write a subscript in PP? The overlapping statistical graphics on page BU-2 are presented as decoration, not evidence. The report is 33 slides long; yet about 10 slide-equivalents are essentially content-free (compulsive repetitive branding, twiddly hierarchical organization, empty space, assorted title pages, and so on). This PP fluffed-up material here and quite a bit more could easily be placed in a technical report on 4 pages of an 11" by 17" piece of paper (folded in half), an exercise left to the student. The tone and style of the presentation seem alienated from professional engineering. It looks like the slides were prepared by a PP designer, assisted now and then by an engineer. Or maybe it is just the PP pitch style diluting the content. At an FRR?! I hope the actual engineering for the shuttle is a lot better than the evidence for the engineering shown in this presentation. How much does a problematic presentational style signal poor engineering? Is it just PP or a PP designer weakening the quality of evidence? Or are there deeper intellectual failures? The dequantification, the failure to follow professional engineering conventions, the infomercial tone are worrisome. There is no sign of engineering discipline here, except in the back-up slides. Thus the effect of the presentation is to suggest that there just might be some problems with foam engineering and analytical quality. A danger of problematic presentational styles, such as NASA PP, is not only that they enable sloppiness but also that they can place the truth in disrepute. It is also a shame that all that expensive engineering work winds up being represented in this manner at a Flight Readiness Review.
-- Edward Tufte, July 1, 2006 |
|
Discovery foam problems occur on launch pad
This morning, Monday July 3, news of a small foam crack came out. On the evening of July 2, after 2 tankings and drainings for launch attempts scrubbed because of the weather, inspectors found a 5" crack in the foam on the Discovery external tank on the launch pad. Here's an excerpt from the KSC Ice/ Debris Team: "The inboard strut for the L02 Feedline Bracket assembly at XT-1129 was found to be cracked. The damage is approx 5-6 inches long and appears to originate near the where the strut connects to the feedline and extends toward the ET. The TPS crack is approx 1/4 inch wide with an offset of approx 1/4 inch. An IPR was initiated for this item. Inboard views of the remaining visible brackets did not reveal any similar damage. Outboard views of the feedline brackets revealed areas of TPS debris in the gap between the feedline and the bracket - this condition was noticed at XT-1129, 1377, and 1623. No obvious indications of crushed foam or debris was detected at the XT-1871 and 1978 brackets." There's a picture of the foam crack and the full inspection report here. Note that the inspection report is written in sentences and not in the cryptic grunts of PowerPoint. There is a research design problem or a control group problem here: are we seeing cracked foam or inspections of cracked foam? Perhaps every launch of the 114 has had some foam debris shedding, and we're only seeing small pieces and cracks now because the intensity of inspections has increased since the Columbia. Or maybe not. -- Edward Tufte, July 3, 2006 |
|
From 9.00-9.30 pm Monday, William Gerstenmaier, NASA's head of Space Operations, gave an informative and smart news conference on the foam issue. He provided a summary of the evidence and answered questions from the space press. There's no problem as a result of the foam liberation incident. Weather permitting, the Discovery will launch tomorrow. -- Edward Tufte, July 3, 2006 |
|
Response to PowerPoint Does Rocket Science (and the upcoming Discovery flight) Did the news conference present the PP slides, or did they a different medium to convey the details? -- Allan T. Grohe Jr. (email), July 5, 2006 |
|
They usually give a brief talk and then answer questions in a straightforward and intelligent manner, accompanied by occasional physical props, such as the broken-off piece of foam or model of the external tank. They did not use PP in the 8 to 10 press conferences I've viewed. You can see the press conferences and the launch by going to nasa tv at http://www.nasa.gov/multimedia/nasatv/ Apparently the PP Flight Readiness Review for the foam (reviewed above) was something of a leak; the other FRRs at the meeting are not going to be made available. Keith Cowing, who runs NASA Watch, sent me an email saying that I might be interested in the foam FRR that he had posted at his website. You can see more on this at I think the press conferences are excellent, assisted by a well-informed space press. After the flight, the head of NASA Michael Griffin was asked at the press conference if he felt "vindicated" by his decision to launch. He said not at all, if anything, it was vindication for the scientific method--that is, looking at the evidence and the numbers at hand. What a wonderful thing for the Director of NASA to say. This contrasts to the PP cognitive style, which often seems to encourage presenters to pitch rather than present evidence.
-- Edward Tufte, July 5, 2006 |
|
The FRR-foam summary slide is now shown about 5 contributions up (in my review of that presentation). And, also added, immediately above, a press conference photo of William Gerstenmaier showing the foam chip. -- ET, July 6, 2006 |
|
Below, a link to a good account of the Discovery inspections by John Schwartz of the New York Times on the problem of distinguishing useful evidence from additional evidence, a problem that also occurs with newly developed exquisitely sensitive measurements (for example, PSA tests and the monitoring of contaminants of drinking water). John Schwartz, New Scrutiny for Every Speck on the Shuttle, New York Times, 11 July 2006. These issues can lead to quite subtle consequences, as my Yale colleague Alvan R. Feinstein suggested in many studies, including this one in the Archives of Internal Medicine: ". . . many breast cancers found by mammography screening have excellent prognosis not just because of early detection, but also because many of the cancers are relatively benign, requiring minimal therapy." Sandra Y. Moody-Ayers, MD; Carolyn K. Wells, MPH; Alvan R. Feinstein, MD, MS, "Benign" Tumors and "Early Detection" in Mammography-Screened Patients of a Natural Cohort With Breast Cancer, Arch Intern Med. 2000;160:1109-1115. (Thanks to Niels Olson for the NYT permalink above.) -- Edward Tufte, July 10, 2006 |
|
Here's a clear technical report and press release, using a 4-page format (similar to A3 or 11" by 17", folded in half). If the report were printed as a 4-pager folded-in-half, then the June 2005/April 2006 images would fall somewhat closer together, which would facilitate comparison (although both images can be seen vertically adjacent simultaneously on, for example, a 30" monitor). From the Institute for Science and International Security (ISIS): http://www.isis-online.org/publications/southasia/newkhushab.pdf Here are links to the Guardian and the Washington Post accounts of the ISIS report: http://www.guardian.co.uk/pakistan/Story/0,,1828058,00.html http://www.washingtonpost.com/wp-dyn/content/article/2006/07/24/AR2006072400995.html Several other ISIS reports are distinguished by their sourcing, detail, use of satellite photographs, and estimates of uncertainty. See, for example, Chinese Military Plutonium and Highly Enriched Uranium Inventories, by David Albright and Corey Hinderstein: http://www.isis-online.org/global_stocks/end2003/chinese_military_inventories.pdf -- Edward Tufte, July 24, 2006 |
|
Interesting presentations with full-screen dynamic graphics by Hans Rosling: http://www.ted.com/tedtalks/tedtalksplayer.cfm?key=hans_rosling&flashEnabled=1 (These links were provided by Kindly Contributor Cesar Martin.) -- Edward Tufte, July 25, 2006 |
|
From Nature, 13 July 2006, still more on PP, this from Martin Kemp, an Oxford art historian:
-- Edward Tufte, July 26, 2006 |
|
Here is a well-designed technical report: http://evo.bio.psu.edu/printclock It is about 7 pages long. Note the excellent illustrations, integration of text and images, documentation, careful citations, and different types of evidence. Note also the use of sentences and paragraphs and flowing text, not the grunts of hierarchical bullet points on slides. This is a very high standard for a technical report, but why not start at that level?
-- Edward Tufte, August 1, 2006 |
|
In your workshops, you describe how to replace PowerPoint presentations with 11 x 17 sized reports, and provide many good arguments for why "engineering by PowerPoint" doesn't work very well. This is a website that might interest you and your audience. It describes a process for creating A3 sized technical reports, and using them to make better decisions. http://www.coe.montana.edu/IE/faculty/sobek/A3/index.htm Dr. Durward Sobek of Montana State University spent six months in Japan as a grad student, interviewing and observing Toyota engineers to uncover the reasons why Toyota was able to develop cars much more quickly than other auto makers and also maintain high standards for reliability. Toyota uses these A3 reports extensively in their engineering processes. They believe that the discipline required to accurately capture a problem on a single sheet forces the author to express the issue with both clarity and conciseness. They emphasize using visual models to express ideas rather than a lot of text, and value the ability to have all of the pertinent information within a single field of vision. The engineers are also required to bring their supporting documentation, so that the team can dive into the details when necessary. Since then, Dr. Sobek has taught many engineers how to use A3 reports to make better technical decisions. I can tell you from my personal experience with this technique that it is amazingly powerful. By using one of these reports, we solved a technical problem within a single meeting that we had literally wrestled with for years through engineering by PowerPoint. By forcing us to make our knowledge about the problem visible in a systematic way, the tool helped us come to a deeper understanding that led to the solution. Best regards, Katherine Radeka -- Katherine Radeka (email), August 2, 2006 |
|
The "A3 Process" described above begins with a good idea and then dilutes it into a Business Methodology Fad. BMFs are characterized by a germ of a good idea, but also by over-reaching, over-simplifying, excessive focus on a single idea, pitchy and enthusiastic over-simplified examples, and pretentious names ("The Toyota Method," "The Long Tail,", "The Genghis Khan Guide to Mastering the Universe," "The Takahari Guide to Infinite Profits," and so on). In the Beautiful Evidence chapter on corrupt techniques in evidence presentations, the section on over-reaching concludes with this: "When a precise, narrowly focused technical idea becomes metaphor and sprawls globally, its credibility must be earned afresh locally by means of specific evidence demonstrating the relevance and explanatory power of the idea in its new application." (p. 151) The A3 method, which at its heart is a good idea, requires some down-in-the-trenches detailed and complex examples. And it should avoid bullet lists in describing the method. -- Edward Tufte, August 11, 2006 |
|
A remarkable account of "Death by PowerPoint," as the phrase takes on new meaning: http://armsandinfluence.typepad.com/armsandinfluence/2006/08/death_by_powerp.html -- Edward Tufte, August 11, 2006 |
|
From Thomas Ricks' book Fiasco: "[Army Lt. General David] McKiernan had another, smaller but nagging issue: He couldn't get Franks to issue clear orders that stated explicitly what he wanted done, how he wanted to do it, and why. Rather, Franks passed along PowerPoint briefing slides that he had shown to Rumsfeld: "It's quite frustrating the way this works, but the way we do things nowadays is combatant commanders brief their products in PowerPoint up in Washington to OSD and Secretary of Defense...In lieu of an order, or a frag [fragmentary order], or plan, you get a bunch of PowerPoint slides...[T]hat is frustrating, because nobody wants to plan against PowerPoint slides." That reliance on slides rather than formal written orders seemed to some military professionals to capture the essence of Rumsfeld's amateurish approach to war planning. "Here may be the clearest manifestation of OSD's contempt for the accumulated wisdom of the military profession and of the assumption among forward thinkers that technology-- above all information technology--has rendered obsolete the conventions traditionall governing the preparation and conduct of war," commented retired Army Col. Andrew Bacevich, a former commander of an armored cavalry regiment. "To imagine that PowerPoint slides can substitute for such means is really the height of recklessness." It was like telling an automobile mechanic to use a manufacturer's glossy sales brochure to figure out how to repair an engine." This raises some of the same issues discussed in the report by members of the NASA Return to Flight Task Force in the contribution at the top of this thread. -- Edward Tufte, August 11, 2006 |
|
I have just found and bookmarked this brilliant set of material - good work and great dialog by all. I've had my share of problems with PP and it's predicessors. Managing marketing groups in Silicon Valley, I always tried to get product managers to keep it simple. My mantra was "no more than three bullets" and "if it takes more than three, you need another slide". This approach was a recognition that presentation software was going to be used no matter what. I would say that my success rate was less than 10%. The prevailing culture often obsessed with creating slide templates that allowed as many bullets on a page as possible. One earlier post illustrates a similar "can't see the wood for the trees" communications problem. Clyde Smithson does a creditable job of describing the efficiency of coding systems (Oct 2005) but misses that larger point. He takes a one-dimensional approach to value in communications - bandwidth is expensive - and advocates everything possible to reduce the useage of this expensive resource. This kind of thinking comes from the early days of communciations and microwave engineering. In a society dominated by Moore's Law, how valuable is that saved bandwidth compared to using a more verbose coding system like UTF that alows us to communicate in most languages on the planet? Clearly, the message is about communication, less about cost. This kind of thinking has us guiding 100 million dollar aircraft around the sky using restricted-voice bandwidth technology that dates back to early phone systems and carbon radio microphones. Anyone who has used a full-spectrum voice system like Skype knows that fewer mistakes and mis-hears occur when all of the audio information is present. Should the lives of hundreds of passengers be placed at higher risk because the radio designer and regulatory agency saved 20 dollars on a radio costing a few hundred? More power to your biting analysis and critical assessment of communication. -- Brian MacLeod (email), August 20, 2006 |
|
Having just completed ET's course yesterday, I was curious to see if PP could handle mathematical expressions (operators, superscripts, subscripts...). It can. Maybe not that easily, but with a little work, the pivotal Boeing slide could have looked much better. This means that those who are required to use PP should take the extra time to make their slides the best they can. Of course, we would all be better off with an 11 x 17 paper document with real analysis... -- Andy Orr (email), August 25, 2006 |
|
A follow up to the quote from Thomas Ricks' book _Fiasco_ and an indication that things can change. Again, from Thomas Ricks' _Fiasco_: "Col. McMaster also challenged the U.S. military, all but banning the use of Power Point briefings by his officers. The Army loves these bulleted briefings, but McMaster had come to believe that the ubiquitous software inhibits clarity of thinking, expression, and planning." pg 421 -- Dave Froberg (email), August 29, 2006 |
|
Engineering software problems at NASA: http://www.hq.nasa.gov/office/oig/hq/audits/reports/FY06/S06012.pdf http://images.spaceref.com/news/2006/NAC.Science.Cmte.report.05.18.2006.pdf -- Edward Tufte, September 3, 2006 |
|
The Harvard School of Public Health Instructional Computing Facility may be capable of writing witless PP designer stylesheets, however, I suspect that much of the problem is generated from above by corporate "identity management" requirements. Consider this advice on how to generate materials which inform their audiences about "world class" research. http://www.leeds.ac.uk/identitymanagement/styles/research.htm -- John McMillan (email), September 6, 2006 |
|
The War on Prose 9/11 Five Years Later: Successes and Challenges: http://www.whitehouse.gov/nsc/waronterror/2006/index.html That's not writing, it's PowerPoint (apologies to Truman Capote). -- Alexey Merz (email), September 7, 2006 |
|
I've worked at several large software companies ($1B+). At both companies I worked to help improve business processes, planning, reporting, and decision making. The unfortunate truth in my experience is that the executive team and their business managers didn't want more than the pablimized bullets. I sat in one meeting where a senior executive chastised product management for "not writing business plans." But then, he'd never asked for one ... so why would anyone ever have written one ... and frankly if someone had written one, I'm not sure he would have read it. It's easy to blame the tool ... but I think .ppt is just a symptom of something else. There are two sides: authentic presentation and authentic listening. We need both. I appreciate all the work Tufte has done on authentic presentation ... who's working on authentic listening? -- Dennis Allen (email), September 12, 2006 |
|
One more piece: Lousy PowerPoint presentations: The fault of PP users? -- Edward Tufte, September 15, 2006 |
|
A well designed single page technical report from Science:
-- Edward Tufte, September 25, 2006 |
|
POWERPOINT GOES TO IRAQ! Here is an excerpt from today's (September 29, 2006) New York Times article on Bob Woodward's latest book "State of Denial": "The book describes an exchange in early 2003 between Lt. Gen. Jay Garner, the retired officer Mr. Bush appointed to administer postwar Iraq, and President Bush and others in the White House situation room. It describes senior war planners as having been thoroughly uninterested in the details of the postwar mission. "After General Garner finished his PowerPoint presentation -- which included his plan to use up to 300,000 troops of the Iraqi Army to help secure postwar Iraq, the book says -- there were no questions from anyone in the situation room, and the president gave him a rousing sendoff." Now, would you not love to see that presentation? -- Bruce Post (email), September 29, 2006 |
|
Better Techniques for Technical Reports in New Orleans The latest New Orleans population estimate, produced by the hard-working folks at the Louisiana Public Health Institute. Estimates for nearby parishes are available at LPHI's Louisiana Health & Population Survey website. -- Niels Olson (email), October 8, 2006 |
|
You may wish to take a look at what Microsoft considers good presentation design: http://www.microsoft.com/atwork/getworkdone/presentations.mspx Those who make the tool seem quite out of touch with the needs of those who use it. I've read far better advice even on sites that uncritically accept PowerPoint as a valuable tool. The slides Microsoft presents as models, I would present as examples of what to avoid. In particular, take a look at the 'better' slide at: http://www.microsoft.com/library/media/1033/atwork/images/getworkdone/55686_375x325_basicbetter2_.jpg Sigh. Dominic Brown -- Dominic Brown (email), October 11, 2006 |
|
Dr. Tufte, I attended your recent one-day course in Chicago. You have advised the 11 x 17 "folded in half" suggestion to help with spatial adjacency (great for handouts when presenting). I'm seriously looking at a variety of printers that support printing to larger sizes including 11 x 17 (also known as "Tabloid" size) and its very close relative in the ISO standard world known as A3 which others and yourself have further commented on aforementioned on this web page. The prices of nice printers with oversized support seems to be gradually dropping (such as the HP 2800dtn). Its one thing to start shopping for a printer that supports oversized paper and duplex printing, yet another thing to figure out what tool options exist especially those that support open (non-proprietary) formats which lends well to document collaboration and preservation. My background is not in design or graphic arts, but I'm determined after attending your course to become a little more renaissance-like and develop some decent design skills (my technical writing skills are already very good according to my peers so my goal is to get the design part down nicely and then let the content flow - no bullet points coming out of my keyboard). That being said, I'm a big proponent of open source tools because I do not like having a proprieter being in control of my destiny. I'm also a big proponent of open formats such as the OpenDocument format. For group collaboration and sharing of knowledge, I truly believe that the formats have to be open and free of any proprietor's control. In my opinion, Microsoft serves up the double whammy for 1.) introducing the PowerPoint cognitive style, and 2.) holding billions of people hostage for decades due to the closed and proprietary Office formats. What applications can you recommend that people use such that the open source / open document format criteria is met, which support decent layout and design of A3 (preferably A3 since its an ISO standard and not a U.S. - influenced alleged standard) or 11 x 17? Clearly there are tools Adobe Illustrator but an open source equivalent would be nice (and one that would work cross platform including Mac OS X, Windows (which isn't going to disappear overnight) and I would not rule out Linux Desktop (closely monitoring the burgeoning success of Ubuntu Linux as a Desktop alternative). Document format-wise, it would seem to me that the W3C proposed standard SVG would be ideally suited for what I have described. SVG would also be great for Sparklines but the browsers need to catch up and make SVG standard (and I'm not sure that Microsoft, once again, is committed to SVG despite Microsoft taking a seat on the W3C's committee). Any further thoughts? -- Eddie V. (email), October 20, 2006 |
|
An easy way to reproduce the 11 by 17 or A3 report (resulting in a 4-page report when folded in half) is to print out 4 pages of 8.5 by 11 paper and then to copy the pages on a copier that runs 11 by 17 paper (with pages 2 and 3 on one side, and then in a second pass through the copier (pages 1 and 4 on the other side). Then fold. We do this all the time. This is especially good for mass production because rather than using the usual printer linked to the computer a high-speed copier goes to work. If you are making more than a few hundred copies of your technical report, then going to a copy shop will prove economical and fast. They can really bang the copies out. The cost for printing and paper for each copy is about .12 euros or 15 cents. If you have color images, things get considerably more expensive. A good use for slideware is to project full-screen color images during your talk. Since copiers often now take direct digital input, your report can be directly linked to the copier without the intervening 8.5-by-11 page waltz. There are surely straightforward ways to do the page imposition necessary for the 4-page format in word processors or in any page layout program (but you'll still have to turn the paper over for the backside printing in most printers). Can some Kindly Contributors help with this question, suggesting easy and inexpensive methods for producing the 4-page/1-piece-of-paper technical report? The idea is not to get caught up in an elaborate computer problem when what's really important is to devote all your preparation time to perfecting your content. -- Edward Tufte, October 20, 2006 |
|
Maybe a bit off topic, but a simple way of folding a single sheet of paper with content into an eight page booklet. -- Andrew Nicholls (email), October 20, 2006 |
|
Excellent example of a public technical report Here's an excellent public technical report sent to all the customers of the South Central Connecticut Regional Water Authority. Note the straightforward text, the names of those responsible, and the competent data tables.
-- Edward Tufte, October 24, 2006 |
|
Coaches Use Laminated Game Outlines for Any Situation Double-sided, 11 by 14 inch technical report--or project management chart--here. -- Edward Tufte, October 26, 2006 |
|
THIS MATERIAL IS FROM OUR THREAD PP AND MILITARY INTELLIGENCE, WHICH HAS ADDITIONAL CONTRIBUTIONS ON THE SLIDE BELOW. Central Command Charts Sharp Movement of the Civil Conflict in Iraq Toward Chaos By MICHAEL R. GORDON, The New York Times Published: November 1, 2006 See the NYT news story here
Here are some preliminary comments on the slide "Iraq: I&W; of Civil Conflict." It appears that "I&W;" means "Indications and Warnings." Replacing the acronym in the slide title does pep up meaning to outsiders: "Iraq: Indications and Warnings of Civil Conflict," but maybe it wouldn't fit on the slide. Only this single slide was leaked (by the military? by DoD?), and so maybe some of the analytical problems are better handled on accompanying slides. Maybe. Doing competent political analysis, epidemiology, nation-building, and war planning (all of which they're trying to do) in a chaotic situation is impossible, and not much good social science and epidemiology can be expected in chaos and from a military entangled in Iraq. In real-time chaotic situations, the data-collection is going to be sloppy because people have more important things to do. (Recall, for example, the gross errors in counts of 9/11 deaths, as the count went from 6,000 to 2,800 in a few weeks.) And what's taking place is in profoundly different cultures and in different languages from those of the non-local military in Iraq. But sloppy data does not justify analytical sloppiness in reporting. In fact, sloppy data requires greater analytical precision of thought. The slide reports performance data--a list of phrases, with each phrase accompanied by a measure of performance. This is what the tables in the sports section, mutual fund page, and weather page of newspapers do very well. Those designs are much better for reporting performance data than the slide format here. In sports and stock market tables, each phrase is accompanied by multiple measures of performance, often over varying time-periods. All that won't fit on the slide; this suggests that we should use better reporting method than PP, instead of abbreviating the evidence to fit the slide. As the millions of readers of sports tables each day demonstrate, people can easily manage large tables of information. Thus those being briefed in the military should ask: Why are our presentations operating at 2% of the data richness of routine tables found in the sports section? Let the viewers read and explore through a range of material; different eyes will search for different things in the evidence. The metaphor should be the cognitive style of the sports section (or weather or financial newspaper pages) not the cognitive style of PowerPoint. There is no cloud of uncertainty or error history associated with the editorializing color. At times, such color codings suggest an excess of certainty. The Iraq slide above provides some relevant but thin and overly short-run time-comparisons: 2 arrows on the left showing "change since last week," and the "Index of Civil Conflict (Assessed)", which sort of compares "Pre- Samarra" with "Last week" and "Current". And there's a potent time-comparison in words: ". . . violence at all-time high, spreading geographically." To get more time comparisons on the 14 "Reads" and "Additional Indicators," 14 sparkline time-line histories for the last year (week by week, if available) would be useful as a overall but detailed summary. This would reduce the snapshot tone of the 14 reads and indicators. In our thread Sparklines: theory and practice, there are (at the top of the thread) data tables with sparklines that report daily and longterm financial data; one such table shows 14,000 numbers, many of them accurate to only 2 digits (not much for financial data) under the philosophy of "Try to be approximately right rather than exactly wrong." The short-run weekly jitters and non-reports need to be smoothed out to see (and compare with)the long-run trends. Weekly data cooperate with the notorious recency bias, whereby way too much weight is given to the most recent piece of data, just because it is recent. These weekly reports should be in the context of longer run information to reduce the chances that analysis will be dancing around only with today's news. The list style, surely one damn thing after another here, is merely descriptive and thus preliminary to policy analysis. That analysis might have been done on the other slides or maybe this report is merely meant as a scorecard. If it is a scorecard, it is grossly impoverished compared to sports, weather, and financial tables. The current fashion (it, too, shall pass) in government is the stoplight style (green, yellow, red), which tends to dequantify data. With categories of this sort, there's always a concern with how the breaks among categories are chosen and with the meanings of the categories. It will often be better to provide some evidence or numbers, and then a separate editorial-judgmental color about the number. The slide contains odd uses of the color-words: for example, a green dot indicating "routine" next to the exciting phrase "unorganized spontaneous mass civil conflict". Shouldn't "routine unorganized spontaneous mass civil conflict" be red-critical? After Hiroshima, would Nagasaki get the routine green dot for nothing different than what happened three days earlier? It looks like weekly wiggles get too much attention, and longrun levels of seriousness too little attention on this slide, as chaos becomes routine week by week and bit by bit. Monthly rather then the sketchy weekly reports might be better for policy analysis. Or at least provide a monthly aggregations over a period of many months (even the entire war) in a scorecard along with the weekly incidents. The leaking of the slide makes a point about the differences between the government's secret analysis and the public reports by the Administration, a common theme of the insider books on Iraq policy-making (most recently Colin Powell's book). At some time, "reality must take precedence over public relations," as Richard Feynman remarked about the shuttle Challenger accident. A good many comments by our contributors are on-point but are not taken into account here. Note the measurements, definitions, and comparisons to standards in the customer scorecard in the "Report of the South Central Connecticut Regional Water Authority" (above). Finally, the over-riding metaphor of all this--"the war on terror"--is a big conceptual problem. Once it's a war, then it almost necessarily invokes large-scale military action and searching for a locale (some place, any place, Iraq) for large-scale military action. But terrorists are more like the Mafia or gangs than they are like armies. Perhaps a better metaphor is that terrorism is largely a law-enforcement problem (requiring focused and clandestine local action, informants, endless detective work, detailed knowledge of the local languages and cultures). Among the grand truths about human behavior, surely the principles of "the unintended consequences of purposive social action" and "it's more complicated than that" are among the top five. Sometimes unintended consequences are largely virtuous or benign (as in market allocation of goods and services if externalities are mitigated) and sometimes the unintended consequences are appalling. That's because it's more complicated than that.
-- Edward Tufte, November 2, 2006 |
|
It occurred to me partway thru the discussions of NASA presentations that it was not always that way at NASA. In the 60's, NASA reps regularly visited elementary schools across the nation. They were good speakers, and they brought many physical props (rocket models, liquid oxygen, various active electronics). What they did NOT bring was any sort of slides or overhead projector. Their shows were wildly popular. -- Carl Witthoft (email), November 14, 2006 |
|
At the completion of my undergraduate studies (1999), I had to present a thesis I wrote as a requirement for latin honors. The thesis dealt with a software API I had written, and I wanted to demonstrate my work by running an executable program on a laptop connected to a projector. Simple as that may sound, I had no success in communicating the nature of my requirements to the college secretary. The discussion when something like this... Beau: Can you provide a computer and projector so that I could run a computer program to demonstrate my work to the audience? Secretary: You'll have to talk with OTR if you want to do a PowerPoint. Beau: I don't want to use PowerPoint, I just want to briefly run a program I wrote myself for the audience. Secretary (bewildered): You've got to talk with someone from OTR if you want to do a PowerPoint. Shelly is doing a PowerPoint and she called up OTR weeks ago to schedule the equipment. You have to check out a laptop with a copy of PowerPoint and a projector. You can't just ask for these things three days before the presentation... there's a lot of setup that has to be done! Beau: Well, if you could just make sure there is a rudimentary PC-compatible computer at the head of the table I can run my program. The room is small so I guess I don't really need the projector. Secretary: You'll have to call OTR if you want to do a PowerPoint. I ended up using plastic transparencies with screen shots. I was never on the PowerPoint bandwagon, but from that point on I developed an acute disdain for the product. I am proud to say that I have never presented a PowerPoint presentation. And I have done a great many presentations... I have presented profesionally to groups of managers, presented data at technical conferences, served as MC for large awards banquets, etcetera. On some occasions I have used HTML to present information. It lends itself remarkably well to this task, especially with the browser in full-screen mode. As far as the simple bullet / agenda type data that PowerPoint handles well is concerned, I just pass out hard copies. It is amazing how few people do that. If I want to show a bunch of pictures in sequence, I'll either use a slide projector of I'll just open the pictures up in sequence, e.g. from Windows Explorer. If the content of my presentation won't stand on its own merit despite the fact that I (unprofessionally) had Explorer open in front of the audience for a few seconds, then there's a larger problem that cannot be solved by PowerPoint. To me, PowerPoint is yet another example of a phenomenon that has become epidemic to the software industry: the answer to a question no one asked. -- Beau Wilkinson (email), November 15, 2006 |
|
I've taken your suggestions to heart in the presentations I've given recently, and I thank you for the good advice. My questions is: how can this advice be applied to a multi-day training course? I'm about to design a three-day technical course, and am determined to not assault the students with days of PP junk. My plan right now is do divide the course into 3-hour sections, each with a single-page handout. Slides would be mostly graphics. But I'm concerned how students will react to the lack of a text book (yes, usually just a set of slide printouts). I'm also trying to design the course so that others can teach it, but the lack of text slides makes this more problematic. I'm guessing I'll need a teaching guide that contains a prose walkthrough of each section. In any case, I'd just like to get some opinions about how to best present a multi-day technical course. -- Patrick Paulin (email), November 30, 2006 |
|
The above advice about the 4-page technical report applies to multi-day courses as well. You probably should average about one 4-page technical report per hour or two; that report should contain graphics, tables, text, and whatever it takes to explain something. Use a slide projector only for full-screen color images and videos. Use the handout technical reports for everything else. In short, your presentation program is Word not PowerPoint. Use slideware only as a projector operating system for full color images and videos. It is useful to hand out all the day's technical reports at the beginning. If your audience goes through them before you get to the material, you're already a success and you should be thankful--for you have an alert active audience using their own cognitive style to look at your stuff. -- Edward Tufte, December 1, 2006 |
|
An opportunity missed Here is an unfortunately typical recommendation: that data graphics intended as feed-grain for talks presented to nonscientists be stripped of (among other things) error bars. The better option is, of course, showing raw data along with the means (or other markers of central tendency). This would not needlessly complicate the presentation, and it would allow even the untutored to evaluate consistency or variability in the data - presumably, key parameters in a study of human behavioral responses, the example shown. If the study is a good one, the raw data will underscore that the conclusions presented sit atop a substantial observational foundation. As Dr. Tufte and many others have pointed out, any audience that can understand a typical sports or business page is going to be underwhelmed by the data-paucity of a typical data-summary bar graph. A dozen dinky little earthtone bars? You fellas got grant support for that? In short, graphs of raw data clarify how observations are accrued and interpreted: the very mechanics of science. Scientific process - every bit as much as the conclusions - should be the central goal of communicating science to a lay audience. Otherwise, the growing fears and suspicions that science is merely an empty belief system are reinforced. -- Alexey J. Merz (email), February 7, 2007 |
|
Agreed. The goal of the summary statistics and consequential graphic must be to show the distribution of the responses. Measures of central tendency rarely tell the whole story (excepting of course when discussing binary data). What are needed are summaries that tell us the whole story; if the mean doesn't do the trick, then we need more. The data (through the order statistics) are always jointly sufficient statistics, regardless of the underlying distribution. That's why showing all the data works. Consider this as an example displaying the perils of using only rudimentary summary measures. In general, error bars that were removed in the above blog are a poor solution to showing the distribution. At least for continuous data, boxes and whiskers, or the like, must be greatly preferred; the ink on the page ought to map us to the data. 'Dynamite plots', showing means as thick bars and standard deviations (or standard errors?) as error bar whiskers, violate this principle: at the bottom with have ink that represents no data; then moving up, we have ink that represents data; above the bar, we have blank space that represents data; then way up high, we have blank space that represents no data. No wonder people are confused. In the end, we are quite rarely interested in the measure of central tendency, regardless of how easy it is to compute. What matters is the distribution; show the atoms. Rafe -- rafe donahue (email), February 8, 2007 |
|
I've read (or, at least, skimmed) the range of cogent comments and responses on the topic of over-utilization of PowerPoint and am surprised to find little (if any) mention of the factor that constitutes my own violent objection to its prevalent use, that being that increased reliance on PP and other visual aids in education, training, project and crisis management fundamentally undermines the active involvement and cognitive engagement of the target audience/students/team members by converting the desemination of information to a passive presentation. I'm by no means anti-tech and absolutely could not do my job (major infrastructure electrical engineering and project managment) without the transforming efficiency of CADD, CPM scheduling, electronic communications and spreadsheets. However, I consider it my good fortune to have been educated, through college, in the pre-personal computer (and PowerPoint) era of blackboards and handwritten notes. Is anyone aware of any formal studies on the pedagogical effects of wholesale conversion of knowledge transmission from the "active" lecture/blackboard and individual note-taking model to the "passive" distribution of pre-written and pre-organized information? To illustrate my, perhaps, unclear point: imagine if the NASA briefing on the renegade foam chunk had included, not a series of well-intentioned but arguably biased and confusing PP "slides" but, instead, a presenter with a handful of notes reading aloud the issues of concern and outlining them on a black or whiteboard. I guarantee that the convened participants would have been actively: (1. taking more copious notes (hence, paying closer attention) (2. asking questions and requesting clarifications (3. organizing and formulating their OWN heirarchies of concern (4. forcing, by their active participation and feedback, the presenter to adjust and expand his blackboard presentation and illustration of the issues to incorporate the group feedback. I would further argue that an individual's depth of comprehension and longterm functional recall of any information is greatly enhanced by the simple act of physically transcribing it. When people are provided, as passive recipients, with pre-organized information and know they will have access to hard or electronic copy of same, they do not intellectually absorb nor process it at the highest level nor are they as motivated to focus their attention. Anyone can test this on themselves (I have.) Next time your organization mandates non-critical HR or IT informational "training" , have them run sessions with and without PP or other graphic aids and handouts and require the attendees at the "without" sessions to take notes. Collect the notes at the end and give both groups the same quiz on the topic. Care to guess the results? The outcome can be even more profound in crisis management meetings (again, I've tested this). PARTICULARLY in a crisis situation the last thing you need for dynamic and creative problem-solving is information distilled/arranged/codified/biased by one or a few people (which is what use of PP inescapably tends to do) and presented in a format that encourages intellectual passivity. -- Kerry Parslow (email), February 9, 2007 |
|
Iran influence in Iraq - the 11 Feb briefing http://www.talkingpointsmemo.com/docs/iran-in-iraq/?resultpage=1&; -- David Person (email), February 12, 2007 |
|
As a high school student, PP is by far the most common way my peers give presentations. While some take the slides as an outline for a more meaningful conversation with the class, others read the bullets verbatim. I guess part of the problem with PP being used professionally is that it's inculcated as the most efficient at a very young age. This is one problem. A more troubling problem is that my physics teacher teaches the class with PP slides. In some ways I feel cheated, and I can tell the difference because I've been fortunate to take the same Physics class at University. The University professor taught with a chalkboard, and, as cheesy as it sounds, really brought the subject to life and made it interesting. Even though I understand there is a difference in quality for high school teachers and college professors, I feel that I was taught Physics by the professor, while I am self-learning the subject that I am taking in high school. It's even more disconcerting to find that some college professors (I've seen an MIT physics class with PP notes) are beginning to use PP slides to teach. In a rigorous mathematical course like Physics, I feel that it is a mistake. But, I'm only a student, what can I do? -- High school student, February 24, 2007 |
|
How is PP like a software house? I have just finished reading the PP essay, which I thoroughly enjoyed and agreed with, but one statement in it bothered me. In mentioning the history of PP, you make the claim that the structure of PP presentations mirrors that of a software house; I fail to see the reasoning here. If you're referring to the hierarchical division of a software house into departments and so forth, than how does this differ from the structure of, say, a university? On the other hand, the operation of a software house (not to mention software itself) is a very complex thing, and surely not a model for dumbed-down presentations. -- Erez Volk (email), March 18, 2007 |
|
For what I wrote about PP's metaphor, see our
thread Metaphors for Presentations: Conway's Law Meets
PowerPoint. For more on design and bureaucracy, see my Visual Explantions, pages 146-149. -- Edward Tufte, March 18, 2007 |
|
Lying is a process -- John Jones (email), April 3, 2007 |
|
Related to my post on Dr Vigh's zoomimage use of Quicktime VR of 9 December 2005, the Raskin Center has a similar demo. Of note is that the Raskin Center's demo is based on the small web format (swf), so more people can use it. Quicktime is not natively supported on Linux and other open-source distributions. The Raskin SWF also implements the obvious by putting many thoughts on one page. -- Niels Olson (email), May 24, 2007 |
|
Other PP threads NOTE: Above is one of our major threads on PowerPoint. The substantial PP threads on this board are: PowerPoint does rocket science--and better
techniques for technical reports Cancer survival rates:
tables, graphs, and PP Plagiarism detection in PowerPoint presentations
PowerPoint and military
intelligence Metaphors for presentations: Conway's Law meets
PowerPoint. Apple's Keynote vs. Microsoft's PowerPoint Lousy PowerPoint presentations:
The fault of PP users? A detailed analysis of PP is my 32-page booklet
The Cognitive Style of
PowerPoint: Pitching Out Corrupts Within, -- ET, May 29, 2007 |
|
Teaching : blackboards, tablets, and powerpoint I read something recently which I thought you might like: http://www.upscale.utoronto.ca/PVB/Harrison/BlackboardPptTablet/BlackboardPptTablet. pdf (linked from http://www.upscale.utoronto.ca/GeneralInterest/Pedagogy.html ) This is a study of how to teach physics to a large class, either by using multiple blackboards (the traditional way, but hard to enlarge for a huge room) or powerpoint (a disaster, of course) or by writing on a tablet PC. They understand clearly what I haven't seen said elsewhere: first, that having the last few minutes' material still visible is crucial, you're always a little behind the speaker and need to refer back. Flipping back in powerpoint doesn't work, having a printout doesn't work well, having the original still visible is wonderful. They achieve this with the tablet by having multiple screens and using them in turn. Second, that making the lecturer write in real-time provides a useful limitation on how much detail can be included, forcing him (or her) to focus on what's really important, and also communicates all sorts of extra clues about how he's thinking about the problem. For instance, one often writes an equation in a several steps, starting with what's important and then working backwards to fill in minor numerical factors and such. This is invisible in the final result (unless elaborately faked in powerpoint) but unavoidable writing it by hand.
-- Michael (email), June 13, 2007 |
|
The first half of this TED talk is very reminiscent of Dr Vigh's zoomimage and the Raskin Center demo above; then Blaise Aguera y Arcas takes it to the next level by reconstructing and navigating the Notre Dame cathedral using Flickr imagery.
-- Niels Olson (email), June 24, 2007 |
|
PowerPoint Does Tunnel Design I saw today that the NTSB announced their suspected cause of the Big Dig tunnel collapse. I teach writing for engineers and I wanted to see if the reports of their investigation would be useful for my classes. Though they do have a synopsis written in sentences and paragraphs all of their presentations are in PowerPoint: http://www.ntsb.gov/Events/2007/BostonTunnel-MA/presentation.htm Another example for my students on how not to present evidence. Will these government agencies ever learn? Bill Wolff-- Bill Wolff (email), July 10, 2007 |
|
I work at an Insurance company and I can't tell you how much I enjoyed the essay and the comments -- A cavalcade of ...what? I'm not sure what to call it. In my blog, Rhetoricia, I wrote about our PPT practices in Biting the Bullet, but it's not very serious -- and there is much to say. I'll be mining these links for some time. Thanks to Anonymous, who directed me here. -- Abby Shaw (email), July 25, 2007 |
|
The Greater New Orleans Community Data Center existed before the storm, but they have really done demographers and statisticians proud since the storm, coordinating data collection efforts and providing analytic consultation to government, academic, NGO, and commercial analysis agencies. The Brookings Institute has teamed with GNOCDC to produce an annual index of progress in the city. Here is their just-released second annual report (there's two page addendum with some late-arriving Post Office data). This is day-and-night compared to the mayor's PowerPoints above. Full disclosure: Alison Plyer of GNOCDC helped some of my fellow medical students and I in the months after the storm, getting us the only numbers that existed at the time to help locate a new student-run clinic in the city. That data came from Dr Bob Post at the Louisiana Public Health Institute, cited above. -- Niels Olson (email), August 24, 2007 |
|
Dear ET, Not sure that this should go here but I think its pertinent. I have been musing on the "Cognitive Style" aspect of your pamphlet about Powerpoint. I have a hypothesis that the quality of thinking in a Powerpoint type slideshow is very much dependent on the manner in which the slideshow was composed. I have a hunch that if an author sets out to write a piece of text for publication, with all the high level cognitive, verbal, linguistic skills etc required to do this well (e.g. Kellogg, R.T. Professional writing. In A. Ericsson et al, Cambridge Handbook on Expertise and Expert Performance. New York: Cambridge University Press), and only after finishing this writing task transfers their thought into a "Powerpoint" presentation the quality of thinking shown on the slides and in the lecture/presentation will be higher than if an author sits down at a computer and directly types into Powerpoint slides. Best wishes Matt R -- Matt R (email), October 4, 2007 |
|
Update to link on October 23, 2005 post External Tank Tiger Team's Interim Report, Fact Sheet http://www.nasa.gov/pdf/136219main_FS_ET_Tiger_Team_Report.pdf ET Tiger Team Report - Part I : Interim Rev C 07 Oct 05 80 pages http://www.boeing.com/defense-space/space/returntoflight/reports/136149main_ET_tiger_team_report.pdf -- Wiley Holcombe (email), October 27, 2007 |
|
|
