ENVELOPES
Natural sensory stimuli frequently consist of a high frequency waveform (i.e. the "carrier") whose amplitude (i.e. the “envelope”) varies independently on a longer timescale. Thus, the stimulus waveform and its envelope, which are both essential for perception, constitute separate sources of information about the sensory environment. As such, envelopes can contain differential temporal frequency content than the carrier. Envelopes have been found to play an important role in many sensory systems: speech recognition in the auditory system, contrast discrimination in the visual system or the determination of distance/orientation of a conspecific in the electrosensory system. To extract the envelope from a natural stimulus, a nonlinear operation such as half-wave rectification or a hilbert transform is needed.
Envelopes in the auditory system
In the auditory system, envelopes have been shown to play an important role in the perception of auditory signals. Indeed, it is the envelope that determines the reception of speech - the words are identified by the envelope. On the other hand, the fine structure of a sound signal - or the carrier - is most important for pitch perception and sound localization.
Click the play buttons to listen to the sound files.
The figure shows two different voice-recordings (signal 1: 40 yrs old male; signal 2: 16 months old female) and the decomposition into their envelopes and their underlying fine-structures (carrier). In many cases, the power spectrum of envelopes of natural stimuli follow a power-law relationship.
The importance of envelopes for the perception of speech can easily be demonstrated by reconstituting an audio sequence by modulating the carrier signal from one audio sequence with the envelope of an other audio sequence. Such chimeras provide insights into the limits and possibilities of hearing and are crucial in the optimization of sound processing in cochlear implants. In the example on the right, the carrier of the
Click the play button to listen to the sound file.
above signal 2 was modulated with the envelope from signal 1 to generate a chimera (new signal). Even more, the content of the sentence can be well understood when the carrier signal contains only one single frequency (eg a 5 Hz sine wave).
Envelopes in the electrosensory system
Envelopes in the electrosensory system of wave-type weakly electric fish occur in different situations: through movement of two fish (movement envelopes; see video below) or the interactions of three or more EODs of conspecifics in close proximity (<1 m). These signals comprise the sinusoidal variations in the amplitude of each fish’s EOD (i.e. the AM). The AM can be further modulated on a longer timescale and thus creating an envelope. These “movement” envelopes primarily contain low (<1 Hz) temporal frequencies, while signals originating from the interference of EODs in a group of individuals generally contain higher (>1 Hz) frequencies. As such, envelopes are ubiquitous features of the natural electrosensory environment that carry behaviourally relevant information such as the distance between two individuals (amplitude of the envelope) or their relative swimming speed (envelope frequency content).
Electric organ discharge (EOD) waveform from Apteronotus leptorhynchus (yellow) with AM (blue) and envelope (red) waveforms. The envelope corresponds to the depth of modulation of the EOD AM that is due to relative movement (green) between individuals. The inset (right) shows a snippet of all waveforms.
However, envelopes in the electrosensory system differ somewhat from those found in other sensory modalities (e.g. visual or auditory). In these senses, the envelope corresponds to the AM of the signal (i.e. a first order envelope). In contrast, envelopes in the electrosensory system refer to the depth of modulation of AMs and would be equivalent to second order envelopes in the auditory and visual systems. It is important to note that, unlike vision and audition, electrosensation is a sensory modality in which perturbations of a self-generated signal carry the relevant information.
The video shows the relation between the distance and the amplitude of the envelope generated by relative movement between the freely swimming fish and a conspecific in an enclosure. The signal that is perceived by this fish is measured using a small dipole. The envelope is constant in situations where there is minimal movement (a, stationary) and increases in amplitude once the swimming fish is approaching the dipole (b, looming).
The importance of envelopes can be demonstrated, because weakly electric fish display behavioral responses by modulating their EOD frequency to track the time course of movement related envelopes.
EOD spectrogram (EOD power spectrum as a function of time) showing behavioral responses to the stimulus shown on top.It can be seen that the animal modulates its EOD frequency and thereby tracking the time course of the envelope (red).
Over the range of frequencies (< 1 Hz) that movement envelopes are composed of, these responses are adapted to the natural frequency content of envelopes as both curves decay with a similar power-law exponent.
Power spectrum of the envelope signal (red) recorded in freely moving fish and the gain (white) measured for A. leptorhynchus superimposed. Note that both curves decay in a power-law fashion with similar exponents.
Further reading:
-
Mante V, Frazor RA, Bonin V, Geisler WS & Carandini M (2005) Independence of luminance and contrast in natural scenes and in the early visual system. Nature Neuroscience 8, 1690-1697.
-
Smith ZM, Delgutte B, Oxenham AJ (2002) Chimaeric sounds reveal dichotomies in auditory perception. Nature 416:87-90.
-
Simoncelli EP & Olshausen BA (2001) Natural image statistics and neural representation. Annual Review of Neuroscience 24, 1193-1216.
-
Lewicki MS (2002) Efficient coding of natural sounds. Nature Neuroscience 5, 356-363.
-
Shannon RV, Zeng FG, Kamath V, Wygonski J & Ekelid M (1995) Speech recognition with primarily temporal cues. Science 270, 303-304.
-
Shannon RV, Zeng FG & Wygonski J (1998) Speech recognition with altered spectral distribution of envelope cues. J Acoust Soc Am 104, 2467-2476.
-
Stamper SA, Fortune ES, & Chacron MJ (2013) Perception and coding of envelopes in weakly electric fishes. Journal of Experimental Biology 216:2393-402.
-
Metzen MG, & Chacron MJ (2014) Weakly electric fish display behavioral responses to envelopes naturally occurring during movement: implications for neural processing. Journal of Experimental Biology 217: 1381-91.