Introduction
Today, machine learning algorithms are used to identify and detect patterns in audio, images and videos. Audio, images and videos can be represented as signals that vary over time and space. The time domain analysis is a representation of a signal as a function of time, i.e. it shows how a signal changes over time and appear as sinusoidal waves. There are a number of independent variables that fully describes a signal and is referred to as degrees of freedom. In the time domain analysis, the degrees of freedom refer to number of samples in the signal.
In the context of applying signals to AI, frequency domain analysis is required to analyze signals with respect to frequency instead of time domain analysis. Frequency domain is used in image processing, speech recognition, feature extraction techniques in machine learning applications and in time series analysis to identify trends and patterns in data.
What is Frequency Domain?
Frequency Domain of an original time signal is a mathematical representation of a signal or data in terms of its frequency components. In other words, it is a way of analyzing a signal by examining the different frequencies that make it up and they appear as distinct impulses. In the frequency domain, a signal is represented as a sum of sinusoidal waves with different frequencies, amplitudes, and phases. In the frequency domain analysis, the degrees of freedom is related to the number of frequency components that make up the signal. The relative strengths of these frequency components can reveal important information about the signal, such as its bandwidth, dominant frequencies, and harmonic content. Some of the key design parameters associated with the frequency domain analysis include sampling rate, windowing function, signal length and the choice of Fourier Transform algorithm.
There is an inverse relationship between time and frequency domain. The desired signal and the undesired signal are separable in the frequency domain and you can use a filter to reject the undesired signal. Also, analyzing signals in the frequency domain can have better computational efficiency than analyzing them in the time domain. This is because many signal processing operations such as filtering, convolution, and correlation, can be performed more efficiently in the frequency domain.
Source: YouTube
Let’s consider an example of image sharpening. To perform image sharpening using the Fourier transforms, we first apply the transform to the image. We can then filter the image in the frequency domain to emphasize high-frequency components, which correspond to edges and details. This filtering can be done by multiplying the Fourier transform of the image by a filter function that emphasizes high-frequency components. We then apply the inverse Fourier transform to the filtered image to obtain the sharpened image in the spatial domain. The result is an image that appears clearer and more defined, with enhanced edges and details. However, it’s important to note that image sharpening can also introduce noise or artifacts into the image, so it’s important to carefully choose the filter function and adjust control parameters to achieve the desired result.
1. Multiply the input image by (-1)^(x+y) to center the transform
2. Compute F(u,v), the DFT of input
3. Multiply F(u,v) by a filter H(u,v)
4. Compute the inverse DFT of step 3
5. Obtain the real part of step 4
6. Multiply the result obtained in step 5 by (-1)^(x+y)
The frequency domain filtering is advantageous because of less computational overhead. It is faster to perform 2D Fourier Transform and a filter multiply than to perform a convolution in the spatial domain. It gives you control over the whole images where we can enhance and suppress different characteristics of the image easily. The idea of blurring an image by reducing its high frequency component or sharpening the image by increasing the magnitude of its high frequency component is easy to understand. Fourier transform states that any function that periodically repeats itself can be expressed as the sum of sines and cosines of different frequencies and different amplitudes.
Frequency Domain Features (FDF)
Frequency domain features are specific characteristics of a signal or system that can be extracted from its frequency-domain representation. These features are often used in signal processing, machine learning, and other fields to analyze and classify signals based on their frequency content. In machine learning, discriminative features are those that are most relevant for distinguishing between different classes or categories of signals. The frequency domain is a common source of discriminative features, as it can provide information about the energy present at different frequencies in a signal. For example, in speech recognition, different phonemes are characterized by different frequency patterns, and the frequency domain can be used to extract these patterns as discriminative features. Similarly, in image processing, the frequency domain can be used to identify distinctive spatial patterns in an image, which can be used to distinguish between different objects or scenes. There are many techniques for extracting discriminative features from the frequency domain. One common technique is to use the Fourier transforms or another frequency-domain transform to obtain a set of frequency components, and then to apply feature selection or feature extraction algorithms to identify the most relevant features for a given classification task.
Frequency domain features include:
- Power spectral density (PSD): Measures the distribution of power across different frequency bands in a signal and is used to analyze the frequency content of a signal and to identify dominant frequencies.
- Spectral centroid: Calculates the center of mass of a signal’s frequency spectrum, which provides information about the average frequency of the signal.
- Spectral flatness: Measures the degree to which a signal’s power is spread evenly across its frequency spectrum. Signals with a higher spectral flatness have more even power distribution across different frequencies.
- Spectral entropy: Measures the degree of randomness or unpredictability in a signal’s frequency spectrum. Signals with a higher spectral entropy have a more complex frequency content.
- Harmonic ratio: Measures the degree to which a signal contains harmonics, which are integer multiples of a fundamental frequency. Signals with a higher harmonic ratio contain more harmonics.
Node details
A node refers to a computational block or module that extracts specific features from a signal in the frequency domain. A feature is a measurable property of a signal that can be used to characterize or differentiate it from other signals. Some features that can be extracted include spectral centroid, spectral bandwidth, spectral roll-off, spectral flux, and mel-frequency cepstral coefficients (MFCCs). These features can be used in various applications such as speech recognition, music classification, and biomedical signal analysis.
Input ports
Input port is a point at which a signal enters a system or a device in the frequency domain. The input signal must first be transformed from the time-domain to the frequency-domain using a technique such as the Fourier transform or the Laplace transform. Once the signal is in the frequency domain, it can be analyzed, processed or modified using various frequency domain techniques.
Output ports
A frequency domain output port is a point at which a signal exits a system or device in the frequency domain. For example, in a filter, the frequency domain output port is where the filtered signal is obtained in the frequency domain after the input signal has been processed by the filter.
Extension
Extension refers to the process of extending a signal from its existing frequency domain representation to a larger frequency range. This is typically done to increase the resolution of the frequency domain representation or to analyze the signal or system at higher frequencies. This can be performed using techniques such as zero-padding, which involves adding zeros to the end of a signal to increase its length and thereby increase the frequency resolution of the Fourier transform. Another technique is interpolation, which involves estimating the values of the signal or system at intermediate frequencies based on its known values at discrete frequencies.
Deep learning algorithms use artificial neural networks. An Artificial Neural Network is made up of multiple processing units called nodes or neurons. The nodes are organized into layers and the layers are connected to each other by weights in the network. The number of nodes present in any given layer of a network partly depends on where in the network the layer resides and it also partly depends on the data that will eventually be processed by the nodes in a given layer and also partly depends on the design choice for the given layer by the network architect.
For the input layer, the number of nodes is directly determined by the number of input features for the single sample that will be passed as input to the network.
In the below illustration, the neural network has an input layer with two nodes. This indicates that the input data would have two input features. For example, in a dataset, a sample could represent an individual person and within the sample we could have two features, e.g. height and weight of the person. So the height and weight will be passed as input to the network and we therefore represent the two input features as two nodes in the input layer. If we are using the network for classification tasks, then in the output layer, the number of nodes has to be equal to the number of output classes. With the hidden layers, we have more freedom in choosing the number of nodes.
Consider an example of Convolutional Neural Network (CNN). CNN is a type of Artificial Neural Network that is popular for analysing images. Each node in the CNN acts as a frequency domain filter and carries out a specific task in the image processing process. The frequency content of an image refers to the rate at which the gray levels change in time. Rapidly changing brightness values correspond to high frequency terms, slowly changing brightness values correspond to low frequency terms. Filters are able to detect patterns. An image might have multiple edges, shapes, textures, objects, etc. So one type of pattern a filter could detect is edges, some could detect corners, some could detect circles, etc. The deeper the networks are; the more sophisticated these filters become. For example, one node can be used for image smoothing and another node can be used for image sharpening.