A Library of Sight and Sound

By Sean Serrell

Visitors to the Lab’s grand opening who managed to wind their way through the long corridors on the ground floor, past the flock of children having their faces painted with tiger stripes or hummingbirds, found a different sort of animal collection in the Macaulay Library. Each of the soundproof multimedia studios featured the sounds of a different animal, ranging from birds (Laughing Kookaburra and Winter Wren) to mammals (indri, bearded seal, and beluga whale) and even fish (plainfin midshipman). All of these selections came from the library’s world-famous collection of more than 160,000 animal sound recordings, but these samples provided only a glimpse of the diversity of recordings the library is accumulating.

Under Jack Bradbury’s direction, the Macaulay Library has broadened its focus considerably from its roots as a repository of animal sound recordings. The library is now incorporating video recordings into the collection, so that every component of a particular behavior, such as a courtship display, can be studied in depth as a whole. “We’re trying to build as exhaustive a collection of specimens as we can,” says Bradbury. “Having only the sound is almost like having a bird’s body without its feathers. We want the entire specimen, properly archived, with the appropriate metadata saying where it was when it was recorded and what it was doing.”

The library’s entire sound archive is currently being copied from its original analog tape format to DVDs, to preserve the recordings for long-term archives. At the same time the library is increasing the depth of its collection to include full behavioral repertoires in the same way some museums have skins representing different ages or different subpopulations.

This new commitment to whole behavior is expressed in the library’s five-year initiative to acquire audio and video footage of behaviors performed by each bird species in North America. “We defined what we call the matrix,” says Bradbury. “The rows are the species, and the columns are the different kinds of behaviors that animals do. Our goal is to fill in that matrix, starting with all of the North American birds and then branching out into other groups and other regions. But as a long-term goal, the library will be more focused on trying to fill in all behaviors of all species.” This push is being funded in part by the “Sponsor a Species” program, which will support expeditions to record new material.

The library is continuing to grow along the species row of the matrix as well, having just acquired its first recording of the California Condor, North America’s largest and one of its rarest birds. During a condor release, a volunteer working with biologists from Hopper Mountain National Wildlife Refuge took the opportunity to hide a sensitive microphone in a rock pile two feet away from some carrion to record the bird’s guttural vocalizations. Besides being rare—as of September 1, 2003, the wild population numbered only 85—the species is quite elusive. Condors can soar up to 140 miles a day foraging for food. So this recording was an exciting acquisition for the library.

 

Photo credit: Jon Reis

In a sound studio, archivist Shelagh Smith digitizes analog tapes of underwater sounds, some of which have never been described and catalogued before. Although it began as an archive for recordings of birds, the Macaulay Library has evolved into a multimedia museum showcasing the sounds and images of a wide variety of animals.

The sounds that fill the oceans and seas, often so alien to our terrestrial ears, are another major focus of the Macaulay Library. Thanks to a $2.6-million grant from the U.S. Navy, the library will soon receive about 8,800 hours of marine animal recordings from more than 50 institutions. This acquisition will fill about one-third of the dozen 480-disc jukeboxes of DVDs that will store the library’s collection. The major objectives of this grant are to improve our ability to distinguish sounds made by what the Navy calls ‘biologics’ (wildlife) from those made by ‘non-biologics’ (which could include foreign submarines and other possible security threats) and to create a master archive of underwater sounds that will help researchers assess the impact of military and other human activities on marine animals and ecosystems.

Far fewer sounds from the deep have been identified, let alone interpreted, than those made above the surface. “Because the sounds in that environment are not as familiar to us, we have to build an understanding of what we’re hearing. There are lots of mechanical-sounding noises down there that are in fact animals making sounds,” says Marc Dantzker, the library’s visual curator.

Basic discoveries about two of the most common underwater sounds have only been made in the last few years. The mysterious and ubiquitous boing, first observed by U.S. Navy sonar operators in the 1950s and thought at the time to be an enemy submarine, was just last year finally identified as a sound that might have been made by a whale. And for many years, it was assumed that the racket made by colonies of snapping shrimp was delivered through the constant impact of millions of claws clamping shut. These shrimp, the largest source of ambient sound in warm coastal waters, make a noise similar to the crackle of a campfire, but at volumes of up to 200 dB, which can seriously interfere with sonar reception. Synchronized video and audio footage recorded at 40,000 frames per second has shown that the claws themselves do not produce much noise, but they shut with such force that cavitation bubbles—tiny implosions like those produced by propellers—form and then pop after the claws close. These bubbles, which are used to stun prey and defend territory, collapse so violently that they emit light in a phenomenon called sonoluminescence.

The Macaulay Library and the Lab’s Bioacoustics Research Program (BRP) are teaming up to analyze the treasure trove of data acquired through the Navy grant. These two units have had a very close relationship for many years, the library acting as a repository for key recordings made by BRP and both programs using Raven, sound analysis software developed by

BRP to archive and research natural sounds. Now, information gleaned from these recordings will help enhance the “intelligence” of the Autonomous Recording Units that BRP researchers place on the ocean floor to record ambient sounds. With a more specific template of sounds to ignore, these units will be choosier about what they record.

Some behaviors sought by the library even defy being recorded with cameras, microphones, or hydrophones. For example, Electric Organ Discharge (EOD) waveforms made by “weakly electric fish” such as the wormjawed elephantfish, must be picked up through electrodes in order to be translated into sound. (Sound recordings themselves are basically electronic transcriptions of the audible world.) Carl Hopkins, a professor in Cornell’s Department of Neurobiology and Behavior, is working with the library to contribute his extensive recordings of EODs made by mormyrids, West African freshwater fish that use a specialized electrosensory system for electrolocation and social communication. Hopkins’s research has shown that these electrical signals allow a fish to recognize the sex and species of other nearby individuals. Some mormyrids use combinations of electrical and acoustic signals to select and pursue a mate. Considering the snapping shrimp, it’s exciting to imagine what could be learned from a synchronized set of video, underwater audio, and EOD recordings, each enriching the data provided by the others.

In addition to scientific and military applications, multimedia recordings of behavior can serve the Lab’s mission of education and conservation. For years, the Macaulay Library has produced audio field guides that have helped field researchers and birders alike to gain knowledge quickly about regional bird sounds. These include sets of biological monitoring CDs for the Caribbean, Peru, and the Andes, which scientists can use to survey and census hotspots for vanishing Neotropical species. According to audio curator Greg Budney, these CDs fill a drastic need, helping to increase the effectiveness of conservation efforts in specific areas of concern.

The library’s incorporation of visual materials allows it to address the urgent, specific needs of other niches within conservation, such as captive-breeding programs. Bradbury says that video footage of endangered species living in their natural habitat helps people to “set up captive-breeding programs in a way that maximizes the normal behavior and normal patterns of the animals as well as can possibly be done in an enclosed situation.” Sometimes even showing videos directly to captive animals may increase reproductive success, as some giant panda breeding programs in China have found. The male pandas’ libidos, already naturally feeble, often wane further in captivity, but after being shown footage of pandas mating, some males have become demonstrably more amorous.

Movies focusing on threatened habitats and the species they support are being produced by the library to be shown in classrooms and theaters at the Lab and at museums. These films, such as the one on Hawaii currently being shown in the Lab’s Bartels Family Multimedia Theater, are intended to be seamless experiences that make the viewer feel immersed in the often remote places being shown on screen. Future plans include films without spoken narrative, which will tell their stories through natural sounds and images or surround-sound audio-only programs that Budney says will also “transport” listeners.

These are not, of course, the most profound effects the new building will have on the Macaulay Library. Bringing the far-flung divisions of the Lab under one roof has fostered contact and collaboration among all parts of this institute. For the Macaulay Library, this has meant joining with the Cornell University Museum of Vertebrates to participate with international groups of natural history museums whose members share databases and other information. According to Bradbury, as soon as the collections go online next year with the help of the Lab’s Information Technologies team, this massive network of information will be at anyone’s fingertips.

“As a museum of animal sounds, and increasingly, animal videos and behavior, we can link and integrate our specimens with physical and genetic specimens, and other things in other places, so that a user can go online and say, ‘I’m interested in bullfrogs,’ and then have the sounds, the videos, and even the cross-sections of the ovaries that were done at the University of Kansas. Everything’s right there at one site, and as far as the user is concerned, it’s transparent,” says Bradbury. The Macaulay Library has recently joined the National Science Digital Library, which connects many math and science libraries across the United States.

This will allow more people to use the library, and for a more diverse range of purposes. “We’re trying to democratize the whole process and help people find exactly what they’re looking for in video or audio,” says Dantzker. Scientists, educators, moviemakers, students, military personnel, birders, and conservationists all will be able to use the collections. And when they find what they need, they’ll be able to get it nearly instantly