In this text  I will present how we developed different  processes for collectively producing a series explorative soundscapes and mechanical artefacts using specific constraints influenced by theories from art and architecture. I will show how we worked with a design methodology that brought together an editor and the final expression of the artwork into one surface of interaction and execution using a virtual cityscape as an iterative ground for sound and music explorations, and give some examples of our different prototypes and iterations. I will also discuss how we tweaked/iterated with the parameters of the framework, the sounds and the final visual expression to match our artistic intention, and finally to bring some noise into Abadyl. Also infuencing the overall framework

Keywords: Art, design, interface, interaction, loop, music,3d, cityplanning

INTRODUCTION

Over the years working with different projects I have learned about and experimented with how to transfer methods from creative writing and worldmaking into my own work practice in the fields of art and design. In Postscript to the Name of the Rose, Umberto Eco writes on the generative logic he has adopted, a logic which both limits and expands creativity. The fundamental parameters of this logic guide what can and cannot be included in a fictional but plausible universe. In my own  work and together with others I have chosen to focus on the generative itself of this logic; that is, it is not about parameters resulting in a consistent, watertight universe, but rather what can be generated from a large number of predetermined real and fictitious parameters. These parameters resonate through a mix of creative writing, physical artefacts, virtual objects, and modified game engines in a constant loop that results in novels, film, interaction designs, and art installations. By letting them evolve in these different settings they will separately host detailed and comprehensive perspectives, which incorporate surprising visual and technical proposals. “At the heart of creativity lie constraints: the very opposite of unpredictability. Constraints and unpredictability, familiarity and surprise, are somehow combined in original thinking.” Boden (1997)

Bring the noise

In March 2009 I started a series of collaborations called hit strip music, Timescapes and Abadyl of tunes with musicians and sound artists were we  tried to explore the soundscape of Abadyl and bring some noise to the city, and also to provide some ideas of how the locations in Abadyl sounds like.

Hit strip Music

In Hit strip music we follow a musical composer on the day when he decides to stop play and write music. How he turns to his paper shredder to get rid of all he scores and notes, but suddenly rediscover them in a way he never imagine. In this scenario the co-creators explore how short musical fragments connected to physical paper strips (that are tagged with rf-id tags) can be used to explore music making by play and combine musical fragments in real-time in different ways, and use them for composing new music.

1. F1 DJ rfid based music generator prototype  2009

Timescape

In this project we use a physical object based on three independently rotating discs with different graphical elements that picks up external information on the internet and local network connected to different activities, statistics, news etc that will influence and change the expression of the object and at the same time be able to trig different media in an unpredictable way – mystical machine that both collect, express an actuate.

2. Time scapes Adobe flash based digital prototypes

Abadyl of tunes

The quest in this project and that is going to be presented more thoroughly in this article as follows,  is to look upon how to create an awareness for the invited co-creators of the underlying programming structure that makes the interaction and execution of the music possible and at the same time make it visible interesting to play, interact, watch and listening to, and at the same time, where interface and

These different works illustrates in a good way how projects are set up and how we conduct them to extract content and assimilate it back to Abadyl.  To be able to conduct these different projects we use Fieldasy and three different perspectives on production we gained developed in the project a Journey to Abadyl.

3. Artefact extraction and assimilation using Fieldasy (Johansson, 2003)

Fieldasy was a method we developed in 2003-2004, first presented as an exhibition in Malmo, Sweden at the Gallery Skanes Konst, and later as a research paper for the Pixel Raiders conference (Johansson and Linde 2004). In Fieldasy we tried to unify different methods into one creative process that attempts to understand and redefine our world in a situation where information is lacking (Figure 4). This lack of information is used as a resource, for example by providing ambiguous fragments as a starting point, removing constraints on the imagination. It was designed with respect to staging a conflict that has a mind triggering influence on the co-creator with a set of problems that only can be captured in a given material. Fieldasy is a process for engaging multiple perspectives in the creation of a world, and the mapping of its virtual space, by extracting artefacts and stories through the developed scenarios, partitur and game-boards.

To finalize a work of art in the field of digital media is not easily achieved, when often the different professions work separately and in different knowledge slots and black boxes, maintaining the linear and assembly line thinking from the past. We instead wanted to have a process were the actual development have to  be conducted simultaneously, with constant iterations in-between them.  In 2008 I started to developed a framework for working with digital and dynamic media and interaction in such a way with my colleagues in PRAMnet, in a project called “a Journey to Abadyl”. Here we  created three new perspectives that put traditional roles of production and research  in the area of digital media  aside to better bridge and mix the different professions in  new ways. The three perspectives are as follows:

• Expression – is everything that has a n actuation and presence in the play/performance/installation. Here all the traditional media producers, put forward examples that can be tested and tried out in different ways. It is about calibrating and creating expressions in relation to the other exploration clusters, involving; actors, groups, audience, film, sound,  stage design, images etc.

• Play mechanics – is about matching the story and the audience/players development through all forms of interaction, actuation and sensory input, designing the underlying structure that support the possible outcomes of all scenes in the play. Here scriptwriting, dramaturgy, scenography and interaction design propose and test different set-ups. This will be an layout of time and space to establish a dialogue with the other two exploration clusters involving; spacial and dramatic models, dramatic tension, sensors, actuators, spacial functions, interaction methods, interaction sequences, deliverance, believability

• Dynamic behavior – is about matching every aspect of programming with time, space interaction and narrative progression. This will be an computer model and the program core which opens up and establish relevant in and outputs to the other two exploration clusters involving; processes, computer models, game play- architecture, state machines, sensor calibration, computer-vision, narrative progression, discrete mathematics

The overall goal with this set-up or approach was to look at how we can build a narrative and interactive framework, and create a testing and production environment for the different projects we are working on. We also saw a need to develop a format for creating and sharing knowledge in-between different professions and stakeholders in this process – from tests through rehearsals and prototypes into the finished productions, so the participants could work closer together exchanging knowledge and experiences between a broad range of discipline, to support the   quest for a surprise somewhere in between generative logic and artistic intentions.

Co-Creation

So the question is not only how  you go about exploring a complex digital space but also how that work is set up and how the actual production is conducted. We have for over ten years used the framework of Abadyl, and the fieldasy method to stage different events and spaces in the form of written scenarios. The scenarios are handed over to the invited temporary citizens and co-creators. They can then act in relation to the scenario,  in and by themselves chosen environment. That in the end helped them produce the artefacts. Our scenarios try to slowly create a discreet dynamic tension and/or displacement between person, objects, time, places, workprocess and events that are not usually – if ever – associated into new and surprising conjunctions. By using scenarios we are able to provide detailed and specific data, which the co-creator can use as background material for their action. Scenarios relation to the overall project is loosely defined as to allow the creation of artworks, that though enriching the database, still are autonomous from the mother project in the sense that they can be exhibited by themselves. They also act as generators while they generate new and unforeseen processes which extend into new and likewise unforeseen contexts.

Abadyl of tunes

In March 2011 I started a collaboration with the media designer Jim Hall.I was introduced to his work Isle of tune by one of my former students who saw his work and said that it had some similarities with The city of Abadyl and how we over the years used racing tracks to compose voice and sounds for different investigations and exhibitions. The idea was that we would make a Abadyl modification of the original Isle of tunes and produce sixteen soundscapes, one for each city part and a soundtrack for the city of Abadyl and release it as a multiple art piece over two years, and at the same time explore Abadyl sound and music wise. The proposed modification included different crossroad configurations such as random, clockwise and anti clockwise, to make variations of how a specific tune could be played and how the music developed over time.

Loops

When we are trying to follow the artistic intention or concept, we want and try to build our different prototypes to produce qualities not known beforehand. But to achieve this the prototype has to reach a certain state of a complexity, a complexity max Hoberg (2006) so we can begin to explore it thoroughly. Therefore a production environment such as the Abadyl framework, with its, constraints, obstacles, limitations, stories, is rewarding for conducting practice based research in the area of art and new media. To many art projects are just illustrations of technology, and to many design projects unreflected uses convention and qualities from art. We do art well, and computing more dubiously; we do computing well but the art is questionable. Bardzell (2009)

The underlying shapes of racing tracks or loops in the city has been used by us and other in many different projects in time based media. In our loop-based scenarios we try to employ parallel and repetitive elements, we can establish and create an open narrative field were creative input from the user can add new elements to the experience. The loops are static and dynamic and can be varied in length and tempo and be interlinked and/or crossed by other loops. The loop also, in its basic version can be seen as a tool for reflection, a moment to revere the constant streams of attractions and desires created and recreated in every digitally performed action and story. Since new media itself has matured, the process is no longer depended on the predecessors more traditional and linear methods of authoring, instead every part of the process is constantly changing the way we author, script and express a multithreaded open work.

When working with 3d graphics and sound we choose from a partly new, partly re-appropriated, palette of narrative tools as the loop. Were the timebased linearity can be substituted with other means of dramatic tension, in a mix of controlled, generative and random execution functions, accessed through the interface and at the same time creating the visual expression of the artwork itself. By closely integrate the interface, the visual expression and the programmable structure we wanted to facilitate an open tool for our collaboration.

Compositions

For Abadyl of tunes we used two basic concepts for the design of our editor on a more philosophical level. Firstly we used something that already been explored in the Abadyl project in earlier works. Coming from Scandinavia with a firm tradition of participative design, we wanted to contrast our design work in the relation to that by using some of Sol Lewitts (1969) Sentences on conceptual art.

Sentence no: 29

The process is mechanical and should not be tampered with. It should run its course.

Based on Sol Lewitts sentence we designed a fixed and nearly mechanical setup where only two options were available for the co-creators – adding sounds to the database and ordering of the interactive graphical elements, houses, tress, objects, roads and cars. Working with explicit and visual recognizable constraints triggers creativity. The “mapping” of a conceptual space involves the representation, whether at conscious or unconscious levels, of its structural features. Much as a real map helps a traveler to find – and to modify – his route, so mental maps enable us to explore and transform our conceptual spaces in imaginative ways. Boden (1997) At first it looked simple and trivial to the co-creators but they soon learned how complex a constraints system can be were its simple programming principles were revealed for the user. The paradox is that constraint enables creativity. By constraining the generative system into an appropriate conceptual space, a writer gains a conceptual structure that can be systematically explored and transformed. A conceptual space can be explored by testing the sounds by the existing constraints. Sharples (1996) In this way of constructing a city space for musical execution, the co-creators soon learned how in different ways make use of the space for exploring and designing their sound scape.

4. Formula one story engines in collaboration with Jon Ovander 2003

Secondly we went back to one of the basic ideas we worked with when Abadyls architectural space was designed. Through Boden (1994) Creativity cannot only and simply be reduced to novel combinations of old ideas. For example, In Architecture and Disjunction, Bernhard Tschumi [ outlines a framework for multiple combinations and substitutions that exist simultaneously, including the following three operations:

Cross-programming;  the use of a space not as intended

Transprogramming;  the combination of disparate programs

Dis-programming; disparate programs that contaminate each other.

These operations possibilities of combinations and permutation of existing programs as well new programs. In the same book he also stated that there are four ways of working with architecture.

1. Design a masterly construction, an inspired architectural gesture(a composition)
2. Take what exists, fill in the gaps, complete the text, scribble in the margins (a complement)
3. Deconstruct what exists by critically analyzing the historical layers that preceded it, even adding other layers derived from elsewhere, from other cities, other parks (a palimpsest)
4. Search for an intermediary – an abstract system to mediate between the site (as well as all given constraints) and some other concept, beyond city or program (a mediation)

For this project we choose to work with the first one since the sounds and graphical elements in its final form have all the characteristics of a composition, a score, a sound player and a visual expression capture in one single unit.

5. Abadyl of tunes map 18 musician Johan Salo Scenario

To be able to host complex and specific projects like this one in Abadyl. We provide scenarios of the city planning and architectural theories we use and misuse during the design of both the different city parts and the architecture of the city. In these scenarios we try to show how the city was constructed and to introduce both the city itself and the specific city part chosen for the invited co-creators. Our scenarios tries to bring field studies and fantasy together, to slowly create a discreet dynamic tension and/or displacement between persons, things, times, places, and events that are not usually – if ever – associated into new and surprising conjunctions. By using scenarios we are able to provide detailed and specific data, which the co-creator can use as background material for their action, choosing their sounds and music clips. Hopefully then, the co-creator themselves imports qualities into the world, which do not and cannot stem from the City of Abadyl itself.

1. Interface, interaction and iteration.

6. Interface  detail  for Abadyl of tunes

The elements we use in the isle of tune modification consists of ;

• Forty four one second long audio tracks.  Fve provided from recordings from the actual locations were the fragments of the city part was originail captured  Thirty nine one second long audio tracks provided by the invited co-creators

• Twelve graphical 3d rendered elements to represent the city parts architecture

• Three vehicles

Figure 7.  3d houses modeled in Softimagel  for Abadyl of tunes

The structure for this setup we use the original isle of tune 2d grid that matches the different city-parts that constitute how the co-creator can layout the graphical elements, with connected sounds. The graphical elements are divided into six groups were four of them contains the sound database and the other two is used for layout the roads and the cars.

The importance in this project is to let the co-creators have full access to both the media database as well as the layout of the sounds and player objects on the grid. This establish a practice for the invited co-creator that make use of the geometrical or geographical space where interface and visual expression are inseparable units. The idea here is to reveal the certain levels of the programming structure of both the visual and sound-scape construction for the co-creators so they can easily understand its constraints.

The art and act of then letting the cars drive around in this constructed space, the cars become the pickup on a record player. Where the three cars follow the thick and thin of the urban tune, they shape, play and share the soundscape of Abadyl.

Worldmaking

Quite late in this project we recognized Nelson Goodman´s Ways of worldmaking (and recognized how we over the years in a similar way worked with his proposed activities of worldmaking that is:

Composition and Decomposition Weighting Ordering Deletion and Supplementation Deformation

Layout, ordering and editing the roads includes periodicity and proximity; and the standard editing and ordering of the tunes in the database as well as adjusting the rhythm and tempo of the different  tunes that are played by the cars.

Taking apart and putting together, analyzing the components at the same time composing wholes, by connecting features into complexes, by adding, breaking up, and restructuring. The importance  here is to let the co-creators have full access to both the media database as well as the layout of the sounds and player objects on the grid. Establish a  practice for the invited co-creator that make use of  the geometrical or geographical space where interface and visual expression are inseparable units in their making of their world. The idea here is to reveal the certain levels of the scripting structure of both the visual and sound-scape construction itself.

The art and act of letting the cars drive around in the constructed space is to the urban system what the pickup is to the record player. Where the three cars  follow the thick and thin of an urban tune. And by letting the cars drive through  the streets of  the  city one shapes, plays and share the sound scape of  Abadyl.

Conclusion

The three parts in the projects we have conducted so far has all proven that this limited approach challenges the co-creators presumptions about how to create sound scapes, play music and construct city-spaces, even though all of the different parts are already known beforehand.

This way of letting the co-ccreators redefine a world in a situation where information is lacking, has again proven both for us who are working with the actual framework and writing the scenarios but also for the co-creators themselves, were they both de and reconstruct their music making capabilities through limitations and constraints. So by providing ambiguous fragments (content/constraints) as a starting point we managed to challenges the co-creators assumptions in order to produce soundscapes and dwell the city of Abadyl at the same time. Where the question for us is how to create an awareness the underlying programming structure that makes the interaction and execution of the music possible and at the same time make it visible intresting to look and listening upon. By the interface revealing some of the scripting structures for co-creators with non or  limited scripting knowledge it adds a positive friction in relations to the co-creators own professional knowledge, and support them in bringing the  noise to Abadyl.

REFERENCES

Alexander, C., S. Ishikawa, et al. (1977)  A pattern language : towns, buildings, construction. New York. Oxford University Press.

Boden, M. 1997 The Creative Mind: Myths and Mechanism, Weidenfeld &Nicholson, London.

Bardzell, J. 2009. Interaction Criticism and Aesthetics. Proceedings of CHI’09: World Conference on Human Factors in Computing Systems. ACM: New York.
Goodman, N., 1978. Ways of Worldmaking. Indianapolis, IN: Hackett Publishing
Greenaway, P. and Akademie der Bildenden Künste in Wien.(1992) Hundert Objekte zeigen die welt = Hundred objects to represent the world : 100. Verlag erd Hatje, Stuttgart.
Hoberg, C 2006 Komplexitets Max, Dialoger Förlag, Stockholm, Sweden
Johansson, M. and Linde, P. Fieldasy, 2004 Fieldasy. Pixel Raiders. Sheffield, UK.
Poggenpohl, S. and Sato, K, 2009. Design Integration. Intellect: Bristol
Sharples M  1996An Account of Writing as Creative Design, A Writer’s Reference 6th Edition Books
Sol Lewitt 1969 Sentences on Conceptual art, Art-Language, Vol. 1, No. 1 .
Tschumi, B 1994 Architecture and disjunction. Cambridge, Mass, London, MIT Press

Thanks to Jim Hall letting us use Isle of tune and all of the modifications he implemented for making our explorations possible.