“The designer creates the environment of others. Until now the designer’s vocation has been both conceptual and concrete. The workshop has been a place to perceive and build models destined to be copied form as production. The question now, however, is how this vocation has been modified, for better or for worse, by the inexorable development of the immaterial culture. The design activity it self is changing because the designer’s tools are becoming immaterial, as are the lives of those to whom the products are marketed. Design by holographic models may still be far in the future, but the concrete activity of design even now partakes substantially of immaterial techniques, or usage of artificial representations, images, and diagrams composed by image-generating machines.”?-Abraham A. Moles

When designing novel graphical user interface controls, interaction designers are challenged by the “immaterial” materiality of the digital domain; they lack tools that effectively support a reflecting conversation with the material of software as they attempt to conceive, refine, and communicate their ideas. To investigate this situation, I and my colleagues at Carnegie Mellon conducted two participatory design workshops. We framed the design process consisting of three phases, conceiving, refining, and communication. In the first workshop, focused on conceiving, we observed that designers want to invent controls by exploring gestures, context, and examples. In the second workshop, on refining and communicating, designers proposed tools that could refine movement, document context through usage scenarios, and support the use of examples. In this workshop they struggled to effectively communicate their ideas for developers because their ideas had not been fully explored. In reflecting on this struggle, I began to see an opportunity for the output of a design tool to be a boundary object that would allow for an ongoing conversation between the design and the material of software, in which the developer acts as a mediator for software.

One of the challenges of interaction design is the lack of tools supporting a flowing conversation between the designer and the immaterial material, contrary to the richness of material dialogues in traditional industrial design

This research is part of a project called Exploratory Programming for Interactive Behaviors (EPIB) that aims to develop natural programming tools for people who don’t have programming background. By interactive behaviors, I mean what an application does as opposed to how it looks. Today, programming is necessary for designing interactive behaviors, and such behaviors are often developed by using scripting tools like Flash or Javascript, or else designers collaborate with programmers who implement the designs in conventional languages like Java or C#. This process poses two major issues, which are: 1. When using scripting tools, the techniques available to designers are too difficult to use and do not adequately support the fundamental needs of designers to explore diverse design ideas freely. 2. When collaborating with programmers, previous surveys show that designers often have communication problems for delivering their idea of some interactive behavior for implementation. Visit the website of a relevant workshop on immateriality that I and my colleague Miso Kim organized during the 7th Design and Emotion conference in Chicago. Check below the conference presentation that I gave during the CHI 2010 in Atlanta.