The development of optical sound recording for either independent playback or accompaniment with film in the early 20th century resulted in a number of experiments with synthetic and graphically generated sound. In Russia and the Soviet Union, simultaneous development of various methods of graphical and ornamental sound proliferated throughout the late 1920s and through the 1930s up until the beginning of WWII. The ANS Synthesizer, developed in 1938 by Evgeny Murzin in Moscow, generated sound through the electronic translation of light moving across an etched glass panel. In Germany, experiments with drawn and ornamental sound took place from the early 1930s onward.
Following WWII, the work of Norman McLaren with the support of the National Film Board of Canada continued development of his graphical sound techniques in accompaniment with hand-drawn animation. The Oramics Machine, designed by Daphne Oram in the UK and realized with the expertise of engineer Graham Wrench, utilized ten film painted film strips positioned above an array of photoelectric cells to control various sound parameters. Experiments with sound-on-disk optical sound based instruments and film sound-sync systems–such as the Optigan, the Welte Light-Tone, and the “Singing Keyboard”–utilized principles of optical sound recording and playback, are also relevant to this discourse.
Brief History of Optical Sound Recording:
Three recording formats were primarily used in the production of sound-on-film recorded through optical means: unilateral variable-area soundtracks, photographed transversally on film as a line of light varying on one side moved across the film; bilateral variable-area soundtracks, also photographed transversally onto film, with a line of light varying on both sides–this method most closely resembles modern depictions of audio signal; and variable-density soundtracks which utilize a line of light across the entire film strip, varying in thickness depending on sound controlled current variations. Sound vibrations entering a microphone are translated into electrical signals which vibrate a beam of light that is photographed onto a film strip. Regardless of soundtrack format, the recording is played back through an amplification system as light moves across the film strip, triggering a photocell with variations in light.
The development of the photoelectric cell by Heinrich Hertz in 1887 and of the Audion valve by Lee De Forest in 1907 set the scene for the development of optical sound recording. While The Jazz Singer is often credited as the first sound-on-film accompanied picture, the sound of the film was actually played back via the Vitaphone, a sound-on-disk system synced with the film. Optical sound recording devices and optical soundtracks were developed in the 1920s in various countries. Eugene Lauste’s experiments in London with sound on film between 1910 and 1912 are the earliest documented examples of sound-on-film recording, and utilized a unilateral variable-area soundtrack format that was recorded and played back using a selenium cell and headphones. Two Danish engineers, Axel Petersen and Arnold Poulsen, began development of the Ortophone in 1918, and the first film accompanied by variable density soundtrack was premiered at Palads Cinema in Copenhagen in 1923. Tri-Ergon was developed in Germany beginning in 1919 by Josef Engl, Hans Vogt, and Josef Massolle, and used the variable density recording method.
The variable-area recording Pallophotophone, or “shaking-light-sound” system was developed by Charles A. Hoxie in 1922. Earlier versions of the device were utilized to record transoceanic telegraphy signals, and an iteration of the device adapted for speech recording was notably used to record President Calvin Coolidge in 1921. The system recorded multitrack audio onto un-sprocketed 35mm film in the form of parallel unilateral variable-area waveforms, and preceded the development of stereo recording and other multitrack recording methods on magnetic film by nearly 20 years. In 1925, General Electric began manufacturing a version of the Pallophotophone, marketed as the RCA Photophone, which was better suited to provide sound accompaniment with film. The device was premiered in 1927 providing sound and music accompaniment to the film Wings. The variable-density Phonofilm system was first developed in 1919 by Lee De Forest, who was joined in 1922 by Theodore Case and assistant Earl Sponable. Case and Sponable ended their collaboration with De Forest in 1925 in disagreement over De Forest’s tendency to take full credit for the development of the Phonofilm system, and continued development of a variable-density optical sound recording system they called Movietone.
In Russia, Alexander Shorin and Pavel Tager were almost simultaneously developing devices for optical sound recording. The Tagephon was developed in 1926 by Tager in Moscow, and was a system of recording variable density sound on film. In 1927, Shorin developed the Kinap, which recorded sound on film in a transversal, unilateral format. Shorin went on to develop both a stationary and portable optical sound recording device, called the Shorinophone. The stationary Shorinophone was capable of recording up to 50 tracks of sound onto 35mm film, and could record uninterrupted audio for hours.
The portable version of the Shorinophone was manufactured in 1940, until the beginning of WWII in 1941. The introduction of magnetic tape recording technology superseded the device, and manufacturing was not continued following the war. Later optical sound recording and playback systems, including Fantasound and Perspecta, implemented sub-audio switching methods and gain control–allowing for the development of more complex and immersive playback systems–as well as discrete and spatialized multitrack recording on sound-film.
Graphical and Ornamental Sound:
Early development of optical recording resulted in experimentation with electronically generated sound in the form of shapes and patterns that were painted, drawn, or photographed onto the film and played back in place of pre-recorded sound. These methods eliminated the necessity of performers or intermediates, and allowed composers to synthesize soundtracks independently. In 1929, the first experiments with ornamental graphical sound were undertaken at the suggestion of artist Mikhail Tsekhanovsky at Shorin’s Central Laboratory of Wire Communication in Leningrad, who voiced the idea that Egyptian and Greek ornaments utilized as a soundtrack might produce some “unknown archaic music.”
Russian engineer Arseny Avraamov and his group ‘Multzvik’– based in Moscow continued experimenting with ornamental and geometric sound drawing between 1929 and 1934, resulting in a number of sound experiments and films including Ornamental Animation, Marusia Otravilas, Chinese Tune, Organ Cords, Untertonikum, Prelude, Piruet, Staccato Studies, and Dancing Etude and Flute Study. By 1936 there were several comparable trends in Russian Graphical sound, including the work of Arseny Avraamov with hand-drawn ornamental sound achieved by means of photographing drawn sound waves on an animation sound to produce a final transversal form; Nikolai Voinov’s 1931 development of the ‘Nivotone’, used to produce hand-made paper sound with final transversal soundtracks; the Variophone, developed by Evgeny Sholpo, which utilized rotating paper disks with cutouts producing sound patterns in both variable-density and variable-area form; and the Syntones method, developed between 1932-1935 by Boris Yankovsky, utilizing resynthesized spectral analysis.
The Variophone, developed by Sholpo in 1930, was a dedicated device that produced synthetic sound with the use of paper disks cut with patterns. The disks were rotated and photographed onto 35mm sound film; the rate at which the disk was spinning determined the frequency of the tone produced. The filmstrip was then played using a normal film projector, which read the soundtrack using photocell, producing purely synthetic electronic sound. Sholpo was assisted in his research by composer Georgy Rimsky-Korsakov, who used the device to realize a number of sound compositions. The first version of the Variophone was capable of producing up to six polyphonic soundtracks by shooting several monophonic parts and combining them later. By the late 1930s and 40s, some Variophone soundtracks contained up to twelve voices, recorded transversally in parallel inside the area of the optical soundtrack. The Variophone was destroyed in the 1941 Siege of Leningrad. The fourth and final version of the Variophone was never completed, and Sholpo died in 1951 following a long illness.
Rimsky-Korsakov mainly approached the Variophone as a means of synthesizing musical compositions without the use of musicians or performers. Though much of his Variophone work was lost with the onset of WWII and the rule of the Soviet Union, some has been salvaged and archived. La Suite Caburateur, likely realized between 1933 and 1934, consisted of up to six polyphonic soundtracks on a single film strip, made of lines of varying width stacked vertically on top of one another. The piece was roughly two minutes and eighteen seconds in length. Wider lines produced lower frequencies, while shorter lines produced higher frequencies, and somewhat resembling variable-area soundtrack format. Les Vautours, realized in 1941, consisted of up to twelve polyphonic tracks, all recorded within the area of the optical soundtrack. The final shapes more closely resembled variable-density soundtrack format, with variations in height controlling the overall amplitude of each individual track, and horizontal spacing between lines controlling the pitch.
These graphical sound efforts in Russia were concurrent with similar efforts being undertaken by others in Germany. Rudolf Pfenninger began developing a graphical sound system that he called “sound handwriting” in Munich between 1929 and 1930. Using an oscilloscope, Pfenninger produced graphical representations of individual notes on twelve by one inch strips of paper that could then be photographed onto an optical soundtrack to produce tones. Oskar Fischinger’s graphical sound efforts in Berlin between 1931 and 1932 closely resembled the ornamental sound experiments conducted in Russia by Araamov and others. Much of this experimentation was cut short by WWII, particularly in the Soviet Union, where many of the devices utilized to produce experimental optical sound were either destroyed or dismantled.
The ANS Synthesizer:
The ANS Synthesizer was designed by Evgeny Murzin in 1938 and named for occultist Alexander Nikolayevich Scriabin, whose creative work and ideas about combining various forms of art inspired Murzin. The instrument itself was not realized until 1958, when Murzin was finally able to establish a laboratory and gather a group of engineers and artists to provide the technical expertise Murzin lacked. One of those engineers was Stanislav Kreichi, hired in 1961 as a sound engineer and composer, who continues to maintain the ANS at its current home at Glinka State Central Museum of Musical Culture in Moscow. The ANS generated sound with the use of 144 optical soundtracks printed over 5 glass disks (720 soundtracks in total), projecting modulated light onto the back of the synthesizer interface, a large pane of glass coated with non-drying black mastic that could be scraped away to form patterns, or with the use of specially made tools to produce harmonically related tones. Individual tones are played when the black mastic is removed, with pitch relating to vertical space: low frequencies at the bottom of the pane, high frequencies at the top. The instrument possesses a ten octave range, with 72 steps per octave (as suggested by Boris Yankovsky, who was involved in the development of the ANS between 1939 and 1940). When light passes over a bank of twenty photocells positioned in front of the glass pane, each photocell sends signal to a corresponding amplifier and bandpass filter, each with independent gain control. The ANS is fully polyphonic, and all 720 individual sine tones can be triggered with a single vertical line of mastic removed along the glass pane.
A number of composers have composed pieces using the ANS, including Alfred Shnitke, Sofia Gubajdulia, Edison Denisov, Eduard Artemyev, Alexander Nemtin, the group Coil, and Stanislav Kreichi. Kreichi has utilized the ANS for the realization of various compositions for performance and sonic experiments including music for the 1961 film Into Space, which featured artist Andrew Sokolov’s cosmic paintings appearing as moving images, smoothly fading into each other and dissolving into fragments. Kreichi interpreted the movement of the cosmic objects as rhythms in his music, and tried to express this movement by tracing on the ANS score, adjusting the shapes over time to suit the visuals. Kreichi continues to compose with the ANS today.
The ANS was also used produce sound accompaniment to many other films, and was used by Eduard Artemyev to produce the soundtrack of a number of early films by Andrei Tarkovsky, including Solaris in 1972, The Mirror in 1975, and Stalker in 1979. For Solaris, Tarkovsky originally wanted the film to be accompanied by only ambient electronic sound, but eventually settled on using ‘Ich ruf’ zu dir, Herr Jesu Christ’ by Johann Sebastian Bach to represent the planet Earth and accompany the prelude, a death scene, and the end of the story. The classical theme of Earth stood as a counterpoint to the ambient electronic sounds produced by the ANS to represent the planet Solaris.
This scene doesn’t feature the ANS, it’s just visually stunning and I wanted you to see it.
Direct Animation and Drawn Sound:
The practice of painting or drawing sound directly onto the area of the optical soundtrack has frequently been practiced in tandem with direct film and animation techniques. The works of Norman McLaren, John and James Whitney, and Guy Sherman and Lynn Loo with both direct animation and synthetically generated optical sound are notable in the scope of this research for their varying and complex methods of sound generation. McLaren painted directly onto the surface of film, precisely creating shapes to produce sounds corresponding to his animation and film work. The Whitney brothers exposed the optical soundtrack to light in cut-out patterns. Sherman and Loo applied physical materials such as sand, newspaper, and various chemicals directly to the optical soundtrack to produce the sound of Sherman’s film works.
McLaren’s long-ranging work for The National Film Board of Canada resulted in a mastery of direct sound-on-film techniques, allowing McLaren to produce effective and sonically rich sound accompaniments for many of his experimental animations and films by way of painting various film strips and layering them in sync with visual elements using a Moviola. His earliest known works featuring synthetic sound generation include “Allegro” (1939) and “Rumba” (1939). “Dots,” produced in 1940, and “Loops,” also produced in 1940, sync synthetically generated sound with abstract geometric visuals, producing a rhythmic texture at times. McLaren’s sound-on-film techniques are explored in some detail a 1951 mini-documentary on his sound techniques, produced by Lorne Batchelor, entitled “Pen Point Percussion.” Shapes and lines were carefully painted transversally onto strips of film. The distance between the lines or shapes related directly to the control of pitch, while height and shape related to amplitude and timbre.
McLaren’s later works include his award winning film “Neighbors,” produced in 1952, with synthetic sound effects and electronically generated musical accompaniment achieved through his sound-on-film technique. “Two Bagatelles,” another film produced in 1952, was also accompanied by synthetic sound effects and music synced to the action of the film. “Blinkity Blank,” produced in 1955, features the sound accompaniment of Maurice Blackburn as well as elements of synthetically generated sound in sync with intermittently flashing shapes of various colors and sizes. “Rhythmetic,” produced in 1956 in collaboration with Evelyn Lambart, consisted of a growing mathematical equation with gradually appearing and changing numbers in sync with synthetically generated sounds representing each number and symbol. Another collaboration with Lambart, “Mosaic,” produced in 1965, consists of a moving array of dots, with synthetic sound synced to their various collisions. The latest of McLaren’s works featuring synthetic sound is “Synchromy,” realized in 1971, with visual effects by Ron Moore, in which a series of brightly colored shapes is synced with synthetically generated sound to produce a rhythmic musical composition.
The experimental films produced by brothers John and James Whitney between 1943 and 1945, entitled “Five Film Exercises”, also fall within the category of direct animation and sound drawing. The visual elements of the films are based on modernist composition theories, and generated with the manipulation of cut-out masks to photograph light directly onto the surface of the film, rather than by light reflected from drawings as in traditional animation. The sound scores are composed using a pendulum device to write sounds directly onto the optical soundtrack of the film with precise control. The brothers won Grand Prize at the 1949 Brussels Experimental Film Competition for the Film Exercises.
Of this sample of direct animation and sound techniques, the most experimental is certainly the work of Guy Sherwin and Lynn Loo. Sherwin’s work with film and sound utilizes various materials applied directly to the surface of the film and optical soundtrack to generate both sound and image. “Cycles #3”, a two-projector film realized in 2003, uses a black and white print set inside another tinted print, and is a development of Sherwin’s prior film “Cycles 1” and “Cycles 2”, produced between 1972 and 1977, in which paper dots are stuck directly onto the surface of a 16mm film strip and onto its optical soundtrack. These dots are converted simultaneously into picture and sound during playback. Sherwin’s 2008 film, “Mobius Loops”, is produced with the use of three 16mm projectors and projected as stacked blocks of light that are gradually colored by gels. The film and sound material is prepared by bleaching away the side of a length of black film leader, leaving a clear strip running along its length. The film strip is then twisted into a Mobius strip, and on projection, a strip of projected light switches from side to side, followed by a change in sound tone produced by the width of the bleach strip in relation to the sprocket holes on either side of the film. The sound mix for the film is provided by Lynn Loo.
“Abrasion Loops”, produced in 2007 and choreographed by Lynn Loo, utilizes two 16mm projectors, and is designed for a long white wall with a projectionist at either end projecting obliquely onto it. To produce the film, color film leader was sandblasted to produce visual effects and accompanying optical soundtrack, giving the appearance of drifting sand or grain along with the sound of shifting white noise. “Newsprint #2”, produced in 2007, utilizes newsprint adhered directly to the surface of film and optical soundtrack, layered with two identical prints using two 16mm projectors. The projectionists attempt to bring the films into synchronization with each other while intermittently pausing and running their projectors. This more abstract approach to optical sound generation produces interesting and varied results, and is utilized by a number of contemporary artists involved with direct animation on film.
The Oramics Machine:
The Oramics Machine was designed by Daphne Oram at the BBC Radiophonic Workshop and independently between 1957 and 1962 and realized with the technical expertise of Graham Wrench. The device is generally recognized as the first sound generation system designed by a woman. Ten sprocketed film strips positioned covering a series of photoelectric cells generated an electrical charge to control the pitch, vibrato (the level of pitch modulation), tremolo (the level of volume modulation), overall volume and timbre. A cathode ray tube with a single bright spot on one side triggered the photoelectric cell when light passed through the painted film. This technique is similar to Sholpo’s implementation of optical sound generation in the Variophone.
Output of the Oramics Machine was monophonic, and relying on multitrack tape recording–with which Oram was particularly masterful–to build up polyphonic sonic textures. Oram utilized the Oramics Machine in the realization of a number of compositions throughout the 1960s, the first of which was Contrasts Essonic, realized in 1968. Later compositions utilizing the Oramics Machine included Four Aspects, also realized in 1968. In the 1980s Oram was involved in the development of a software version of Oramics for the Acorn Archimedes computer. Oram suffered two strokes in the 1990s, forcing her to stop work; she died in 2003 at the age of 77. The Oramics Machine is currently on display at the Science Museum in London, along with an exhibition of Oram’s other work, and will be on display until June 2015.
Sound-On-Disk and Optical Film Sampling Instruments:
The Clavilux Color Organ, developed by Danish-born visual artist Thomas Wilfred in 1930, utilized rotating glass disks hand-painted with color patterns. Wilfred’s color organ, however, did not produce sound, and its inspiration–much like the influence of Alexander Scriabin on the development of the ANS Synthesizer–came from a group of Theosophists who wished to demonstrate spiritual and occult principles through the use of light and color. Color organs have been produced in various forms since the late 1500s, and, while these devices typically did not produce sound, their intent in marrying various senses aligns with the motivations of other engineers and composers involved in graphical sound. Oskar Fischinger’s Lumigraph also falls within this color organ tradition.
The Optophonic Piano, developed by Russian Futurist painter Vladimir Baranoff Rossiné in 1916, generated sounds and projected revolving patterns of color and light onto a wall or ceiling through a series of painted glass disks, filters, mirrors, and lenses. The three octave keyboard controlled the combination of various filters and glass disks, and the variations in opacity of the painted disk and filters were detected by a photoelectric cell to control the pitch of a single oscillator. A single continuous tone varying in pitch–similar to the Theremin– was produced in accompaniment with visual projections.
The Welte Light-Tone (Lichtonorgel), or Phototone Organ, developed in Germany by Edwin Emil Welte in 1936, was an electronic instrument that utilized electro-optically controlled tone generators. The instruments sound generation mechanism consisted of 12 glass disks printed concentrically with 18 different waveforms, with 3 different timbres for all octave registers of each single note. Each of the waveforms was recorded in unilateral variable-area format. The glass tone wheel rotated over an array of photoelectric cells, which filtered a light beam controlling timbre and pitch. Only three production models of the Lichtonorgel were ever produced. The German arm of the Welte-Mignon company in Frieburg was bombed by allied forces in 1944, destroying all of the company’s closely held designs and secrets. Welte’s instrument predates the Optigan by around forty years.
The “Singing Keyboard”, invented by Frederick Sammis in 1934, played electro-optical recordings of audio waves stored on strips of 35mm film which were controlled in pitch by a keyboard. Sammis recorded sung and spoken words on to individual strips of 35mm film. The samples were then positioned in such a way so that when a key was pressed, the film strip would be pulled over a photoelectric cell, amplified, and reproduced. Sammis’ instrument is similar in design to later instruments utilizing samples stored on magnetic tape, such as the Mellotron and Chamberlain.
The Optigan, developed by Mattel Inc. in 1971, was another optical sound-on-disk playback interface. A clear celluloid disk printed with 57 concentrically photographed variable-area soundtracks rotates over an array of photoelectric cells. When a key is pressed, a beam of light activates a photoelectric cell triggering a corresponding pitch. A rhythm section to the left of the keyboard triggers time synced rhythm accompaniment with an array of buttons. A flashing LED indicates the downbeat so that players can come in at the start of the rhythm recording in time. The Optigan is not renowned for particularly hi-fi sound quality, and has gained a sort of cult following for its characteristically grainy and lo-fi sonic attributes. A number of disks were available, including Bluegrass Banjo, Majestic Pipe Organ, Polynesian Village, and Latin Fever.
Mattel only manufactured the Optigan for a short time between 1971 and 1973 due to huge financial losses in its production. Pea Hicks, a self proclaimed Optigan enthusiast and administrator of Optigan.com, has devised a modern method of recording and producing new Optigan disks. Chilton Corporation and Opsonar (and later Vako) attempted to improve upon the Optigan with the Chilton Talentmaker and Orchestron (respectively), in order to compete with the Mellotron and other similar sample based instruments. None succeeded and production of all three optical disk based organs was entirely discontinued by 1976.
The subversion of optical sound-on-film and sound-on-disk recording techniques for the purpose of electronic sound generation produced a number of remarkably innovative technologies and graphical sound techniques with novel sonic results. Departure from conventional use allowed optical soundtracks to provide composers and artists with the ability to produce sonic and musical works through purely electronic means. At points, the practice of graphical sound production served to unify visual and sonic arts, both in its execution and use in accompaniment with visual media.