Game Environments

Sound in Gaming Enviromnents

(revision initiated on nov.30 2007)

Alejo Duque

The trend

In my view the path to get down into the mud with the topic of "Sound in Games" is to check what the genre of game modding has given to us so far, this way a framework will be defined and a place where to explore present sound possibilities in the Gaming Enviroments will be by default rendered within the locus sonus lab. For such case I suggest to have a look to a 3d engines called Panda3d (other 3d engines like OGRE where considered but they dont offer sound manipulation in an integrated way, on the other hand there are core audio tools that are far from our grasp in regards to programming level like the OpenAL API, Soya3D (phyton based as Panda3d) has bindings to OpenAL making it perhaps less complex to integrate sound but in the course of exploration Panda3d seems to be the one to try first cause it integrates a more "advanced" sound manager:

http://www.panda3d.org/apiref.php?page=Audio3DManager

Panda3d can help us get started withouth depending on 3rd party platforms (read here SecondNatureSecondLife our notes on the project developed for the festival Second Nature where Locus Sonus developed and sound spatialization using Second Life). Panda3d was also the platform used during the FACE workshop with Chicago. More infos here:

http://www.panda3d.org/wiki/index.php/3DAudio

The rest of the sound in games story, as far as i can tell, is more related to techno-historical details that shape the general field of the so called "new media arts" combined, in regards to sound, with the usual studies on the psychological effects of sound in games. Below I try to develop a fast overview of that but check first this links, one is an example of todays techniques to sound game modifications i,e:

On the side of game modifications, here few examples to get started:

http://www.moddb.com/games/61/half-life-2/tutorials/24158/adding-precipitation and the other one is a selection of works for an exhibition called Forbidden Games: http://maarav.org.il/classes/PUItem.php?lang=ENG&id=869 For more info and a tool that can help us to get our hands dirty go here: http://ioquake3.org/?page=games

Last but not least, here is the wikipedia entry on audio_game: http://en.wikipedia.org/wiki/Audio_game

History Overview

Computer game/play is one of the strongest driving forces for contemporary culture, they have been around for more than 20 years and have expanded the was of making cinema, of telling stroies and for the case, producing and spatializing sound.

Even though I'm no expert in the field, I did spent few years in the "battlegrounds" playing from Taito and Casio portable consoles including playing with Pong back in the early 70'ies (juguete de contrabando*). The Atari 8bit attack or even big scale arcade games like any williams pinball to my long love addiction with Galaga. One of the "best moments" in my life as a gamer was playing until it's end the computer game Myst, one that highly introduced the so called Acoustic Landscape into the game itself along with the markwting phenomena of interactive CD-ROM games. It was said that not knowing what the experience of playing Myst was could be compared to not knowing what for previous generations was to read a book. So, if you never played Myst consider yourself a game-analphabet :). I have also to confess in this short paper I made good progress playing F.P.S (first person shooter) DOOM, where basically what one hears are painful shouts, continuous gunshots and loud explosions. My life as an spontaneous gamer then came to a halt, the last intense experience was playing with a PlaystationI I bought with some friends for entertaining few boring weekends: Mortal Kombat, Gran Turismo and FIFA (Same as with DOOM, intesive DSP (digital sound processing) for generating the tipical augmented sound effects of punches, motor engines and cheerful fans.

This short text is about the uses of sound in game enviroments, I will depart from a storyline that could help locate ourselves in games history.

The first video games where like silent films, just in 1972 the game Pong introduced sound and in 1978 Space Invaders included the first soundtrack. Sound Effects where used before Space Invaders but they didn't use sound for creating film-like atmospheric tension that developed according to the different moments of the game. In Space Invaders sounds where used for augmenting the thrill of the gaming experience. Sound accelerated in an "interactive" manner according to the amount of invaders on the screen and their proximity to ground level.

Crucial moments to consider from the timeline:

http://www.youtube.com/watch?v=XRllfMIyfT0

1952

The name of the game? Noughts and Crosses. It runs on a Cambridge computer called the Electronic Delay Storage Automatic Calculator. Don't ask me where to get one, I bet it was a "silent" game though the machine in itself and its many operators will certainly do a lot of phisical exercise while "playing".

1958

Tennis for Two, created by William Higginbotham whom also was part of the group that build the first atomic bomb. I can't avoid relating the trace of the parabole with that of the atomic missile. see Gravity's Rainbow from Thomas Pynchon or the xxxxx conference from Peenemuende: http://scrying.org/doku.php?id=pm:w.a.s.t.e:pmp

1972

"In 1972, Pong the first video game which includes sound comes onto the market." Pong, was basically a simulation of ping pong, debuted as the first game to feature sound, the "sonar-blip" of the ball hitting the paddles."

1977

Second generation consoles appear (atari 2600) this ones had also microprocessor that allowed the use of game cartridges. And also included a processor capable of sound generation, this document is the bible of sound in the atari2600 http://qotile.net/files/2600_music_guide.txt I will quote here below some of its technical capabilities:

"The TIA is the chip in the Atari 2600 that produces audio and video. The audio portion has two independent voices, each of which has a 4 bit volume control (16 values), 5 bit pitch (32 values), and a 4 bit control register which selects the type of sound you will hear. When writing software for the Atari, the standard labels for these registers are AUDV0 and AUDV1 for the volume registers, AUDF0 and AUDF1 for the pitch registers, and AUDC0 and AUDC1 for the control registers. The 5 bit pitch is very limited and the frequency values are simply divided down from the system clock, so many of the pitch values are not in-tune with others. Note that setting the pitch register to a lower value results in a higher pitch."

1978

Click Image to see/listen the game frentic sounds.

"Space Invaders hits arcades with its menacing, paranoia-inducing soundtrack. It is an example of simple, effective sound design; the thumping audio track actually accelerates in tempo as the enemy invaders draw nearer– and move faster. The following year Asteroids– another arcade game– employs a similar sound technique to achieve the same results. At the beginning of the 80's PacMan makes its debut. The game boasts many memorable sound and music elements" Nothing better than playing it, so here is an online demo: http://www.spaceinvaders.de/

1989

Pac-Man (to play from the dash-board)

This game use music in between levels, like scenes from a movie.

1985

Tetris and Super Mario Bros.

"Most early games did not feature continuous musical soundtracks due to the memory and sound output limitations of early arcade and home gaming machines; melodies could be composed of no more than a few lines of synthesized notes with monaural (non-stereo) sound and little or no harmony, so any attempt to create a sustainable soundtrack was an exercise in excruciating repetition—some games of the mid-80s fell prey to just this sort of trap: "Audio, especially game audio, is a powerful weapon. When used properly, it has the power to involve, immerse, elevate, and reward. It has the power to excite. It can make an artificial world appear to be deeper, older, and much more complex and complete than it actually is. But when misused, audio reveals its most awesome and deadly power—the power to annoy." taken from Eric Pidkameny's Leves of Sound.

Music has developed enormously in the history of video games, its been based on the use of cinematic strategies but evolved into a new dimension determined by the complexities that interactivity could offer. This is something we can see in Super Mario Bros. Music evolves as hardware allows and games begin to include specific sound leitmotivs for different characters that appear in the games. Its the times of the explosion of the games console market. theres high offer and a myriad of bad quality games so the whole systems suffers a big crash some companies go bankrupt and other just close.

1993

Myst - DOOM and the advent of violent video games.

In myst and based on computer sound cards we can really appreciate how the notion of interactive game music was not just sounds that reacted to a particular state of a player or space, but that could evolve and announce new levels of the game.

In year 2000 there was a remake of Myst that included Weather effects like thunderstorms and sunsets/sunrises were added. This version didn't sold well at the time cause the home PC's where not really ready for rendering it all smoothly.

(Myst example to demo/play)

Midi sound files played a crucial role in the history of computer games, the sound-card synthesizer offered the sound and music designers to go beyond the primitive beeps.

1995

Sony releases the 32-bit PlayStation, "the 24-channel sound chip provides CD-quality stereo sound and has built-in support for digital effects such as reverb and looping." Gamespot

Midi sounds in video games are part of a cult that dedicated to keep track of a collection of midi files that are in the range of the commodore64 to the wii plus all sort of arcade games: http://www.vgmusic.com/

producer included sountracks that the player could select beforehand so to serve as background for his/hers gaming experience, Wipe-Out offered tracks from famous electronic music producers like: Underworld, Future Sound of London, The Prodigy, and The Chemical Brothers.

2000

Sony releases PlayStation 2. "Along with the 128-bit Emotion Engine CPU, the system boasts 48 channels of sound plus 2MB of dedicated sound memory." With the ability to play DVDs and link to the Internet, the PlayStation 2 quickly became one of the most sought-after game consoles.

network link

There are also many flash based web games:

http://www.mcvideogame.com/game-esp.html

Some of them engage in political activism:

http://www.molleindustria.org

On a historical jump intending to find the connection to networked games and sound, back from when one cannot avoid to make reference to explosions or machine guns (First Person Shooter games), here is the URL from a project from jean babtiste bayle and thomas lucas that i find to be a good example of sound modding:

http://www.grandhoteldeparis.com/unreal/

(example to play/demo)

Its clearly a big jump in contrast to the text based imaginary MOO (MUD - Multi User Dungeon) which I had the chance to learn to program during a workshop entitled "Global Theater" a performance directed by Adrianne Wortzel back in 1997 after the Polar Circuit II in Tornio, Finland. The sound of a dungeon reverberates in the vacumm of the networks.

sound specific

Doing such historical revision makes sense to me if done in the scope of an anarcheological recall of media that can help trace the development of the use of sound, from Pong to the DS to the sound of electroplanktum (http://en.wikipedia.org/wiki/Electroplankton here the video: http://video.google.com/videoplay?docid=4439496341187280535&sourceid=docidfeed&hl=undefined)

Is also crucial to locate in such timeline the origins and uses of speech synthethizers. A good exmple of technological simplification could be the sample audio track of lsdj for the gameboy. "LSDJ Little Sound Dj, tool of choice for amateur & professional musicians and composers!": http://mp3death.us/tute/lesson01.php

Here is a comprehensive list of games that make use of an open sorce API to control sound in games:

http://www.openal.org/titles.html

And as a curiosity, here a listing of synths available in Second Life to buy with real money:

http://shop.onrez.com/search/?SearchString=synth&SearchIn=items_all_items&x=0&y=0

Extended Mode:

-streamlab http://www.streamlab.info

-PacManhattan -> to play in the 1st life (natural sounds and ambient)

-fijuu >> 3d engine to demo, mix between game and video as in vj

-playsh >> silent coding game in the style of the Moo. A follow up → http://www.grandhoteldeparis.com/unreal/

-fluxus and al-jazari >> from dave griffiths scheme code to live-code realtime music and visual representations

-Ludic Society >> the third life LIFE-CODING (xxxxx)

-Angus Carlyle and Corrado Morgana >> http://blog.game-play.org.uk/?q=CORRADO_MORGANA

-selectparks: Political Games http://www.selectparks.net/modules.php?name=News&new_topic=2

-An Approach to video game music >> http://www.gamestudies.org/0401/whalen/

Games and the so called flash-activism

-http://www.mcvideogame.com/game-esp.html

-http://www.molleindustria.org

Emulators

ps2onlinux:

http://www.pcsx2.net/

DOSmultiplatform:

http://dosbox.sourceforge.net

Dreamcast on linux:

http://www.lxdream.org/news/

+all the NES (nintendo emulators around)

references

http://gamestudies.org/

http://playsh.org/wiki/InspirationalProjects

http://www.sterneck.net/musik/bey-immediatism/index.php

Interview to a game developer

Play Along - An Approach to Videogame Music by Zach Whalen → http://www.gamestudies.org/0401/whalen/

Levels of Sound: http://www.vgmusic.com/information/vgpaper2.html

Video Games net.radio

http://ftp.emulationzone.org:26038/

and, not to forget!

-dig for the link to that SL clone for musicians

>> U2 in SL: http://www.u2insl.com/ (video): http://www.youtube.com/watch?v=Mro9Qzv—k8

>> REZ http://www.youtube.com/watch?v=lp89LjiYAcA&feature=related

1. Notes from the FACE workshop Game rulez

Español

El uso de la localizacion exacta de sonidos (lease fuentes sonoras) y su movimiento en el espacio es algo relativamente reciente no solo en el area de la composicion acustica, el cine o los juegos de video. Dicho interes se ha desarrollado de la mano con los avances tecnologicos en el campo de la computacion. Es decir, el area de la electroacustica no ha estado al margen del desarrollo de software particularmente de la arena de uno de los vectores que hala con mayor vigor el del desarrollo de hardware. Me refiero a la industria de los video juegos.

Sin hacer un recuento detallado o historico de dicho orden cronologico pero si con el animo de poder sobre imponer la linea de tiempo del cine con la de los video juegos y la de la electroacustica para asi tal vez descubrir los puntos de cruce de dichas practicas.