Aaron Marcus is the founder and President of Aaron Marcus and Associates, Inc. 
(AM+A). A graduate in physics from Princeton University and in graphic design from Yale University, in 1967 he became the world’s first graphic designer to be involved fulltime in computer graphics. In the 1970s he programmed a prototype desktop publishing page layout application for the Picturephone ™ at AT&T Bell Labs, became the first designer to program virtual reality art-spaces while a faculty member at Princeton University, and directed an international team of visual communicators as a Research Fellow at the East-West Center in Honolulu.
Download PDF Transcript of Podcast
Note: This is a transcription of a podcast. It has not gone through a professional editing process and may contain grammatical errors or incorrect formatting.
Related Podcast: A Journey Through User Experience Design
Transcription of the Podcast
Joe Dager: Welcome everyone! This is Joe Dager the host of the Business901 podcast. With me today is Aaron Marcus, founder and President of Aaron Marcus and Associates, Inc. (AM+A), and a graduate in Physics from Princeton and in Graphic Design from Yale, in 1967. He became the world’s first graphic designer to be involved fulltime in computer graphics. His lineage is long and impressive. Aaron I would like to welcome you, and can you tell me what the world was like for a graphic designer? How did you get your first job? How did that all start? I’d love to hear that story of a graphic designer starting out.
Aaron Marcus: Well thanks Joe. I’m really pleased to have this time with you today. And you’ve asked me a very interesting question. I think one of the ways to answer this is to ask “how did I as a graphic designer first get involved with computer graphics and interactive design and so on?” I was trained at Yale University’s graduate school in Corporate Graphic Design intended to introduce professionals into the corporate world of Manhattan publishing corporate design in general. Because of my own special background I taught myself computer programming because I had the skills in Mathematics and logic from my undergraduate career in Physics. It took me a while, in fact, to come to understand the terminology and concepts and the way of life of designers, of graphic designers in particular. I remember realizing that I had no idea what anybody was talking about in my first six months of graduate school. And then I finally did come to understand the terminology and enjoyed it so much that I didn’t sleep for three years until I graduated. It was very exciting. After I learned about computer programming I immediately started to try to do some computer graphics, and I remember so vividly going for a job interview in 1967.
In those days, AT&T’s Bell Telephone Laboratories would send out recruiters and one of my colleagues in school suggested that I go for an interview. Well let me advise you. I did exactly the sort of thing that you should never do at an interview. I walked in, and I said to the two guys who were there, “I know you don’t want to hire me. I can’t imagine what I could do for you. But I will tell you that I have this background in Mathematics and Science, and now I’m in design school.” And they smiled at each other, looked back at me and said, “Actually we’re looking for someone exactly like you.” From that I became a summer intern consultant, let’s say, designer and began to play with this most advanced computer equipment in the world. I didn’t realize at the time that Bell Labs was one of the premier research laboratories in the entire human civilization. I was exposed at that time to giant mainframes, interactive computer graphics, the first video screens connected to computers. In the subsequent year or two as I acted as a consultant to them I programmed a desktop publishing system for the picture film which was about five to ten years ahead of commercial systems. It remained a kind of research toy at AT&T Bell Labs, but it certainly introduced to me a lot of the fundamentals of how to think about designing displays and interactivity for a group of people, in this case graphic designers as it happened.
I knew my audience. After I left graduate school, I became a faculty member at Princeton University School of Architecture and Urban Planning for about nine years.
I did a lot of teaching and academic work. I published and did research in computer graphics and graphic design in general. I finally wound up at Lawrence Berkeley Laboratories research lab; government supported, at the University of California where I helped to write one of the first user interface design manuals and became familiar with a lot of the current equipment of that time which is now we’re talking 1979. I eventually got up the courage to go for a research grant from DARPA, which has funded a lot of the research in computer graphics technology including the internet, and started my own design firm. So in some ways except for very brief stints, my first major employer was myself as I started one of the first graphic design firms in the world to be based entirely in computers.
I purposely moved to California to be near Silicon Valley to be able to apply my skills. And it became clear to me in about 1979 that I could use my background in graphic design for the benefit of computer technology people to improve the display of information. It became particularly clear to me in a project that I did at the East-West Center in Honolulu where I had to lead a team to visualize global energy interdependence. And that really oriented me to the challenges of creating forms, charts, diagrams, and maps to convey complex structures and processes to people. And that meant I could take advantage of the skills that I had and be able to do that. So that’s a capsule of my first work in computer graphics and graphic design and how I started my own firm.
Joe: Well it doesn’t seem to me that you live on the cutting edge, you live years ahead of the cutting edge a little bit. Don’t you?
Aaron: Well I did in effect live five to ten years ahead of the bleeding edge. That was my role in life at the time.
Joe: Is it still that much ahead of commercialization?
Aaron: Well it’s interesting you know, what happened a lot in the 1980s and ‘90s is that our corporate clients outsourced R&D to all the small startups. We used to be hired by R&D groups at NCR, at Apple, at HP, at Intel, etc. and gradually a lot of our work shifted to more commercial put-it-out-on-the-street software. And by the time in year 2000 when we did a major project for Samsung, we were already working for Samsung to create new product ideas that would be introduced into North America within two to three years on their smartphones. And at that time Samsung, in the year 2000, was almost unknown in terms of mobile technology. So we had to shift to, in some cases, much more practical shorter term development. I will say that we worked a lot for people working ten to fifteen years ahead of commercial development. Then it became one to three years. Nowadays people are trying to get things out in six weeks or eight weeks for better or worse.
Joe: How new is design thinking and innovation because those seem to be the buzz words right now?
Aaron: Well these are buzzwords. They are popular. There’s no doubt about it. But in my own humble opinion the kind of introduction to design thinking that I had as a graduate student is not too much different from what people are being introduced in the business world today. It’s just being given the latest version of clothes to look stylish and to help a lot of consultants and design firms market their services to people who may feel a little insecure about design or about being an “artist” or not, and not understanding a lot of the iterative processes, the various stages of development of products or services. I’ve seen it – I’ve got to admit that’s it’s an effective marketing technique for services of design consultants and design firms.
Joe: Well I kind of smile a little bit because I’ve never seen so many innovation consultants and startup consultants. I didn’t know that there were that many people that actually started up companies before.
Aaron: The other term that’s used and overused is innovation. I mean people want to not only innovate but disruptively innovate. And it’s kind of interesting to compare a lot of what I’ve read and seen with two recent projects of ours for Siemens Corporation. We spent a good part of the year trying to visualize their disruptive innovation process which they’ve been perfecting for years within one particular advanced R&D group. We studied video tapes; we studied many documents. We tried to encapsulate that process that takes place over specific activities and specific segments of time to involve technology companies, technology explorers within Siemens, and the business units that need to absorb new technologies into the processes, into their product and service offerings. So we’ve ourselves actually done a very detailed study of this innovation process, and it’s quite complex, it’s quite specific.
Siemens doesn’t want to talk a lot about that outside of Siemens. We did do another project called The Innovation Machine in which my associates and I did a study of innovation theory and tried to present a series of conceptual functions for a mobile device that would assist people within an organization or a corporation to be more creative and to be more innovative.
In fact everyone in general is somewhat innovative and creative but very often people within a large organization may not actually know how to turn their latest brilliant idea into something more to combine that with other people’s critiques and assistance to get it into a kind of internal marketplace for new ideas and to be in the process of assembling teams of people who might be able to implement something. So what our project – and we published a paper about this in the proceedings of the Design, User Experience and Usability Conference in 2013 – we explained this conceptual design which is also part of a series of projects we’ve developed called the machine projects, like The Green Machine to persuade people to save energy, or The Story Machine to persuade families to tell more stories.
In the case of The Innovation Machine we were trying to persuade people to be more effective and efficient in their innovative activities and their ideas and to turn a quick notion or an initial concept into something more concrete to put it into an internal marketplace where people could decide to contribute time to it and so on, to see it all the way through to completion. So yes I do agree that we see more and more people labelling themselves Chief Corporate Innovator, but I think those are somewhat hyperbolic in some cases, and again a technique for marketing services to business clients.
Joe: Tell me a little bit more about the machine project. Can you explain what they are and how they work?
Aaron: I was very much moved by attending a conference that BJ Fogg at Stanford happened to be running in 2008 or 2009 called Persuasive Technologies I believe. I was introduced to a number of projects for desktop computers and mobile phones etc. as platforms to help change behavior of people to get them to lose weight or to stop smoking or to take their medicines when they’re supposed to. I became quite intrigued with this idea and felt that mobile devices – phones primarily at that time – would be an excellent way to help change people’s behavior. Reading through Fogg’s theories and Robert Cialdini’s theories on the science of persuasion as well as Abraham Maslow’s theories on basic human needs, we put together a concept for helping to change people’s energy conservation behavior.
We tried to do some competitive analysis of current products at the time which were mostly oriented to the PC – not to mobiles phones – and envisioned an architecture mental model for the functions and the kind of data that it would have that would bring about this change of behavior. Now one of the key things about all of our machine concepts – and we’ve done now ten of them – is that they all embody five fundamental components I’ll call them. One of them is a dashboard to understand where you are in a journey, in this case to save energy or to tell more stories or to be more innovative. So a dashboard is one thing. The second thing is a roadmap or a world view that tells you or reminds you about where you came from and where you’re going.
The dashboard tells you “how am I doing on the road.” The third aspect is focused social networks. Not all 500 or 5,000 of your Facebook or LinkedIn friends, but who are the five to ten people – family members, personal friends – who really care about your journey and can contribute support or specific information. The fourth component we call just-in-time knowledge. It’s again not the results of all possible internet searches on a particular topic but what is really useful to me to know right now where I am on my journey, whatever the subject matter is. And the fifth component and very, very important is incentives. That means all the rewards, awards, games, nostalgia shocks, workshops, whatever it takes to keep people motivated and interested.
For each of the different subject matters, we have developed these five components and embodied them in specific screen designs which we’ve published. These are conceptual designs. They are not commercial products. We try to interest clients in the subject matter and see if they would like us to work with them to bring one of these products into being. This worked well with SAP Enterprise Software Company that saw our Green Machine project, brought us in-house, and we worked with them for a quarter of the year to create further more advanced prototypes oriented to enterprise users, not to home energy savers.
We consulted with them on the incorporation of persuasion design and sustainability into enterprise software. So from our point of view that worked pretty well. We have been trying to since then find other clients for some of the other topics.
For example, we did a Health Machine to try to change people’s behavior about exercise and nutrition to avoid obesity and type 2 diabetes. We did a Money Machine that helped retiring baby boomers save and spend money appropriately for the rest of their lives. We did the Story Machine which I mentioned to increase storytelling from grandparents to parents to children. We did a Travel Machine which helped turn routine leisure and business tourists into cultured tourists to better understand the people they’re actually with. We’ve done The Leaning Machine to improve online education. We’ve done The Driving Machine to improve driver safety and also sustainability. We recently worked on The Happiness Machine which is making people happier through mobile devices. We’ve just finished, probably the final one for a while, called The Marriage Machine which is really intended for all kinds of committed couples to improve the relationships between people.
In each of these different subject matters, we investigate the relevant subject matter theory; we combine it with information design and visualization and also persuasion design to put together very specific screens, example screens that show quite specifically how this would all function. Now I’ve got to admit they’re fakes. They’re not functioning prototypes. But they show any team of people what could be done and suggest the benefits of doing something like this. Those are the ten machines, and I’m in the process of putting together a book that summarizes all of our work over the past five years on these machine projects.
Joe: These aren’t light projects or easy projects at all, are they? This is very intense.
Aaron: No actually each one of them takes a minimum of three months. So for any prospective client or partner in developing them further that person or group is getting about three months of free R&D as a starting point because they would have to be redone very specifically for the final set of personas and use scenarios, the target market, and the specific content, the specific functions that would finally be embodied in a commercial product or service. We recognize that. So the fact is we give away this information. As I say, we’ve published case studies that are available to everyone.
Joe: So the Story Creator app – now that’s something that do you make a story out of yourself? How does that work?
Aaron: Well it’s intended to provide workshops in good storytelling that are built into the device because a lot of people don’t necessarily know how to ask questions in an interview and to illicit good stories from family and friends. There are ways to keep track of stories and for people to add to them as much as one can on some story oriented applications and activities. But you know NPR has done storytelling for years now all around the United States. This is publicly available in archives. You can go to certain places, maybe centers, regional centers, or urban centers and people tell stories. They’re archived. They select some of them for broadcasting on public radio. Some of these stories of the Story Machine might really be for a family and be quite private. They might be very moving, touching, disruptive in some cases and some of them might have a time lock meaning “Well I’ll tell the story, but you can’t reveal it until after I’m dead.”
The fact is that a lot of storytelling within families has diminished as small families have split off and try to raise their children separate from grandparents. Coupling them together with Skype or whatever communication application is very helpful to be sure. But even then these communication media don’t necessarily foster storytelling. They do enable conversations to take place but not necessarily storytelling. There are typologies of stories. There are techniques to encourage good stories. We tried to research that subject matter and build in a few helpful bits of assistance to encourage people to improve their storytelling abilities and to more regularly engage in storytelling behavior.
Joe: That brings me to something I want to ask you. Do you think technology has really assisted us or held us back? Maybe I think it subverted us a little bit.
Aaron: Well you’ve asked a fundamental and powerful question – has technology helped us or hurt us? And as we have been trying to answer that through the ages, we find that technology has many benefits, and we also have to be aware of some of the side effects or unintended consequences. A friend and colleague of mine Ed Tenner published a book called something like Unexpected Consequences. It has a very catchy name, and I can’t remember it right now. But almost inevitably, people creating various kinds of improvements or technologies or assistance of some kind completely overlooked some of the side issues and as a consequence sometimes some very undesirable things occur. You can just as well say “Is it good that we got printing?” Johannes Gutenberg was condemned by the church, by kings, by all kinds of people who said, “Not only are you robbing all the scribes of their livelihood but you’re creating these cheap books that can get into the hands of people,” and there’s no telling what might happen if people are informed.
Certainly that was why the earliest royal presses were setup because it was controlled by the rulers and was ruled by a very dangerous technology. You may recall that at one time in the Soviet Union every Xerox machine had to be known about by the government. It was illegal I guess to have your own copy machine or to be using it without someone supervising. Of course that failed to be controlled when some is that I believe opinions of people reproduced and circulated began to be available.
If you take us and fast forward into the present we have again disruptive technologies of social connection, of wearables, of smartphones, smart t-shirts, smart toothbrushes. Where does this all lead? It leads to our needing to continually ask questions, “What is this for? What are the implications of it? Who made it? How is this to be used? Who determines who’s going to use it? And do we notice any unintended especially negative side effects to this product or service?” And people are certainly being able to be more and more aware of such things. One of the challenges that we face now created by the technology is the emergence of again a buzzword called Big Data. We’ve actually had Big Data for a long time and of course what constitutes big changes over the centuries. We now have just bigger data than ever before.
My concern and the concern of some people is that people will drown in all this data and not be able to really make use of it. Both the specialists, the experts, the paid consultants, and the ruling powers as well as the people in general. And one of the missing ingredients in the discussion of Big Data is Big Data visualization or Big Data sonification, any way to be able to understand “What does this mean? What should I see here? What should I be concerned with? What should I worry about or be happy about and rejoice about?” One of the slogans of our firm – “We help people make smarter decisions faster.” That’s what we do.
We need help in making smart decisions more than ever before. If our educational system is floundering we can’t have a good democracy. If we’re overcome with bad information presentations, I mean poorly constructed; people can be misled and be misguided by intention or by accident. We really need to be cognizant of and vigilant about just how much information are people being given and how we can summarize that and provide action advise to turn more data and more information actually into knowledge that we can use to make decisions about our lives. That is I think one of the larger challenges of our time.
Joe: Big Data you’re saying really it’s been around a while. Now we’re just taking it to the point that we’re understanding it better and how to use it better? Or we’re trying to get there still?
Aaron: I’d say that we have bigger data than ever before. It’s just absolutely astonishing how much we can store on a little flash drive now. When I started my firm we were very, very excited because we bought a gigantic hard drive. It cost $5,000. We were so proud of it. Do you know how much data was stored on that hard drive? It was more than we imagined we could ever use. The amount was 20MB, about double whatever you send in the maximum email messages today constantly. At that time in 1982 30 plus years ago, we’ve had Big Data. We just continually are adjusting our expectations of how big.
Joe: You’re doing something and expanding a little bit. What are you doing in China?
Aaron: Well for several years now I’ve been working with an organization the DeTao Academy. And specifically I’ve been trying to help them start teaching and activities related to user experience design. There was a study done in 2002 – that’s about ten years ago – that showed that most of the user experience designers in China had two years or less of professional education and experience. And that wasn’t enough for what China was facing in terms of software development and hardware development of course, as well. It seemed likely that being able to train people to help work on projects that would create better products and services for China as well as for export to the world, would be beneficial for people. I first visited China in 1975, but I really didn’t start visiting as I have in the past decade until about 2002 at various conferences.
In 2009, I established a relationship with the DeTao Academy to try to create workshops like the one I ran at the end of 2013 called Mobile User Experience Design. And we had about 40 people from all over China participate and actually we were working on I think it was the fifth version of our Happiness Machine. All the participants seemed quite eager and interested in that because I was trying to give it a specifically Chinese focus. That is participants could design screens and think about the needs of people in China.
Specific use scenarios came up that was quite different from whatever would be thought about in Europe or North America, and that was just perfect. I actually gave this conference in the previous month at the Hong Kong Polytechnic University School of Design to another set of students working on Happiness Machine 4.0. And not to get off China but I had given it earlier at the Institute of Design in Chicago. That was Happiness Machine 3.0. There are more and more courses now in user interface design, user experience design, etc. There’s not enough and not enough good quality I think, but it is improving. The largest tutorial I ever gave was at SIGGRAPH Asia in Hong Kong and that was to 200 people I think in 2012. But there’s a real demand for this kind of input to help make professionals who can design better products and services.
There is a movement in China, as there is in other countries, not only to just see China as an outsourcing place for manufacture but to, in fact, have the design be accomplished in China. That is designed in China, not just made in China. And I believe that we’ll be seeing more and more of that. China has done a remarkable thing in creating a middle class of 400 million people within a small number of years relatively speaking. And that’s larger than the EU and larger certainly than the entire United States. This is a group of people who are now much more sensitive about quality and products. They are happy to consider domestic brands. They know about international brands. And I’m sure it’s only going to be a matter of time – and we see this already with the rise of companies like Huawei or now Alibaba and others trying to get New York Stock Exchange listings – to see these companies reach out to the rest of the world and to produce products and services, to compete on the world market.
Joe: Could you tell me what is upcoming for you and maybe some of the projects you’re working on.
Aaron: I will be giving a tutorial about mobile persuasion design at the Computer Graphics and Media Design Conference in Chicago on July 8-10. I’ve mentioned my conference called Design, User Experience and Usability. That’s going to be held in the island of Crete in June of 2014. And the next year’s conference will be in Los Angeles.
I expect quite a large showing of US and North American people. It’s also the case that I’ll be in Poland giving three tutorials at I think it’s the Association for Human Factors and Ergonomics Conference that’s being held in Krakow. I expect to be back in Shanghai to give further tutorials to the DeTao Academy in the middle and end of this year. So there will be opportunities to hear me talk about our machine projects, persuasion design. And also I haven’t mentioned another project which is of interest to me, and that is “What can we learn from Science Fiction movies and television in terms of worlds of the future and worlds of the present?” I have a lecture and a workshop called The Past 100 Years of the Future, which looks at what science fictions movies and TV have decided to show us about advanced communication products and services, what they got right and what they got wrong.
For example in the very, very important movie “2001” Arthur C. Clark worked with the famous film director – Stanley Kramer [Kubrick], if I’m remembering correctly – and envisioned the future of space stations and so on. But they had completely neglected the concept of mobile phones. There wasn’t a single one to be found. So certain parts are right, and certain parts are completely wrong.
There are things to learn from the science fiction movies about trying to predict the future, how to introduce new concepts to mass audiences, viewing the science fiction movies as a kind of user experience testing situations or ethnographic research situations. Besides being very entertaining to look at these movies and videos, I think there are important lessons to be learned for professionals. So I published a free eBook on that topic which is available for downloading on our website. I will be publishing some more books either free or at little cost to get them out for people. I’m working on this mobile persuasion book which will be published through Springer Publishers. In terms of the projects, we’ve been working for a cancer data provider, most recently for mobile phone design to help clinical laboratories and doctors better understand the available clinical trials and research related to advanced genomic testing.
We hope to get involved with process control, depictions of the usability studies of the screens that help to run entire transit systems for cities. And it’s really hard to predict who will next knock on our door. Generally the projects that we like are ones that are challenging, complex. At first we don’t have any idea what to do and then gradually the fog clears and then we see a clear path to developing specific solutions that meet not only of course the clients’ needs, but many stakeholder needs and of course first and foremost the end user.
Joe: I would like thank you very much for taking the time with me today. What’s the best way for someone to learn more about you and to contact you?
Aaron: Well we have a Facebook page, a LinkedIn page, a website under amanda.com or Aaron Marcus and Associates. And people can certainly email me directly to [email protected], and I’ll do my best to answer each and every inquiry.
Joe: The podcast will be available on the Business901 iTunes store and Business901 website. So thank you very much Aaron. I appreciate it.
Aaron: You’re welcome Joe. I appreciate the opportunity to speak with you.
Lean Service Design Trilogy: Bring this Workshop on-site!