小波變換詳解

小波變換詳解

1，簡介

We can use the Fourier Transform to transform a signal from its time-domain to its frequency domain. The peaks in the frequency spectrum indicate the most occurring frequencies in the signal. The larger and sharper a peak is, the more prevalent a frequency is in a signal. The location (frequency-value) and height (amplitude) of the peaks in the frequency spectrum then can be used as input for Classifiers like Random Forest or Gradient Boosting.

我們使用傅里葉變換將一個信號從時域轉換成頻域。頻譜中的峯值表示信號中出現次數最多的頻率。頻譜中鋒越大，越尖，表示在信號中出現越普遍。頻譜中峯值的位置(頻率值)和高度(振幅)可以作爲分類器的輸入，如隨機森林或梯度增強。

The general rule is that this approach of using the Fourier Transform will work very well when the frequency spectrum is stationary. That is, the frequencies present in the signal are not time-dependent; if a signal contains a frequency of x this frequency should be present equally anywhere in the signal. The more non-stationary/dynamic a signal is, the worse the results will be. That’s too bad, since most of the signals we see in real life are non-stationary in nature. Whether we are talking about ECG signals, the stock market, equipment or sensor data, etc, etc, in real life problems start to get interesting when we are dealing with dynamic systems. A much better approach for analyzing dynamic signals is to use the Wavelet Transform instead of the Fourier Transform.

通常情況下，使用傅里葉變換能夠很好的解決問題，這是因爲信號不依賴時間變化而變化的；如果信號只包含X ，信號在任何時間段種都是相同的。這樣越是非平穩信號，那傅里葉變換的結果越是糟糕。遺憾的是，我們生活種常見的大部分信號都是這種非平穩信號。例如我們常說的ECG信號，股票信息，設備或傳感去的數據，等等。在現實生活中，當我們處理動態系統時，問題開始變得有趣。而更有效的方法就是使用小波變換來代替傅里葉變換。

Even though the Wavelet Transform is a very powerful tool for the analysis and classification of time-series and signals, it is unfortunately not known or popular within the field of Data Science. This is partly because you should have some prior knowledge (about signal processing, Fourier Transform and Mathematics) before you can understand the mathematics behind the Wavelet Transform. However, I believe it is also due to the fact that most books, articles and papers are way too theoretical and don’t provide enough practical information on how it should and can be used.

儘管小波變換能夠很有效的處理時間相關的分類和信號，但是在數據科學領域並不被人熟知。這是由於在你明白小波變換身後的數學原理前，你需要知道一些前提知識（信號處理，傅里葉變換，數學）。我認爲這也是由於大多數書籍、文章和論文都過於理論化，沒有提供足夠的實用信息來說明如何使用它。

2.1 從傅里葉變換到小波變換

In the previous paper we have seen how the Fourier Transform works. That is, by multiplying a signal with a series of sine-waves with different frequencies we are able to determine which frequencies are present in a signal. If the dot-product between our signal and a sine wave of a certain frequency results in a large amplitude this means that there is a lot of overlap between the two signals, and our signal contains this specific frequency. This is of course because the dot product is a measure of how much two vectors / signals overlap.

從之前的文章種我們知道傅里葉變換是如何工作的。就是通過很多不同頻率的正弦波疊加我們能過知道一個信號中存在哪些頻率。如果我們的信號和某個頻率的正弦波之間的點積導致振幅很大，這意味着這兩個信號之間有很多重疊，我們的信號包含這個特定的頻率。當然，這是因爲點積是衡量兩個向量/信號重疊程度的指標。

The thing about the Fourier Transform is that it has a high resolution in the frequency-domain but zero resolution in the time-domain. This means that it can tell us exactly which frequencies are present in a signal, but not at which location in time these frequencies have occurred. This can easily be demonstrated as follows:

傅里葉變換對頻域變換有很好的解決方式，但是不能解決時域變換問題。這意味着，它能告訴我們信號包含哪些頻段，但是不能告訴我們這個頻段是在信號的哪個時間段出現的。下面的例子能夠很好的說明問題：

import numpy as np
from matplotlib import pyplot as plt
​
t_n = 1
N = 100000
T = t_n / N
f_s = 1/T
 
xa = np.linspace(0, t_n, num=N)
xb = np.linspace(0, t_n/4, num=N/4)
 
frequencies = [4, 30, 60, 90]
y1a, y1b = np.sin(2*np.pi*frequencies[0]*xa), np.sin(2*np.pi*frequencies[0]*xb)
y2a, y2b = np.sin(2*np.pi*frequencies[1]*xa), np.sin(2*np.pi*frequencies[1]*xb)
y3a, y3b = np.sin(2*np.pi*frequencies[2]*xa), np.sin(2*np.pi*frequencies[2]*xb)
y4a, y4b = np.sin(2*np.pi*frequencies[3]*xa), np.sin(2*np.pi*frequencies[3]*xb)
 
composite_signal1 = y1a + y2a + y3a + y4a
composite_signal2 = np.concatenate([y1b, y2b, y3b, y4b])
 
f_values1, fft_values1 = get_fft_values(composite_signal1, T, N, f_s)
f_values2, fft_values2 = get_fft_values(composite_signal2, T, N, f_s)
 
fig, axarr = plt.subplots(nrows=2, ncols=2, figsize=(12,8))
axarr[0,0].plot(xa, composite_signal1)
axarr[1,0].plot(xa, composite_signal2)
axarr[0,1].plot(f_values1, fft_values1)
axarr[1,1].plot(f_values2, fft_values2)
(...)
plt.tight_layout()
plt.show()

代碼未復現

圖1，第一行的兩個圖表示的是一個信號包含4種頻率，不因時間的變化而變化。第二行的兩個圖表示的是信號中的4種頻率根據時間依次出現

In Figure 1 we can see at the top left a signal containing four different frequencies (, , and ) which are present at all times and on the right its frequency spectrum. In the bottom figure, we can see the same four frequencies, only the first one is present in the first quarter of the signal, the second one in the second quarter, etc. In addition, on the right side we again see its frequency spectrum.

圖1中我們能看到左上角的信號中包含4種不同的頻率(, , and ) ，所有頻率在這段時間內全部出現，右上角是這個信號的頻譜。在下面的兩個圖中，我們同樣能看到這4種頻率，但是第一種出現在1/4時間上，之後的頻率依次出現。並且我們能在右下面看到這個信號的頻譜圖。

What is important to note here is that the two frequency spectra contain exactly the same four peaks, so it can not tell us where in the signal these frequencies are present. The Fourier Transform can not distinguish between the first two signals.

值得注意的是兩個頻譜圖都包含了4峯值，但是他們都不能明確表示出在信號的哪個時間段出現的。這就使得傅里葉變換不能區分這個兩個信號。

In trying to overcome this problem, scientists have come up with the Short-Time Fourier Transform. In this approach the original signal is splitted into several parts of equal length (which may or may not have an overlap) by using a sliding window before applying the Fourier Transform. The idea is quite simple: if we split our signal into 10 parts, and the Fourier Transform detects a specific frequency in the second part, then we know for sure that this frequency has occurred between th and th of our original signal.

在試圖解決這個問題過程種，短時間傅里葉變換出現了。在使用傅里葉變換前對原始信號通過滑動窗口的形式將信號分成幾個相同長度。想法很簡單：如果我們將信號分割成10份，傅里葉變換找出特殊的信號在第二個部分出現，那麼我們就知道這個頻段出現在原始信號的 th and th

The main problem with this approach is that you run into the theoretical limits of the Fourier Transform known as the uncertainty principle. The smaller we make the size of the window the more we will know about where a frequency has occurred in the signal, but less about the frequency value itself. The larger we make the size of the window the more we will know about the frequency value and less about the time.

這種方法的主要問題是遇到了傅里葉變換的理論極限即不確定性原理，我們把窗口的大小做得越小，就越能知道信號中某個頻率出現在哪裏，但是自身的頻率值獲得的越少。窗口的大小越大，我們對頻率值的瞭解就越多，而對時間的瞭解就越少。

A better approach for analyzing signals with a dynamical frequency spectrum is the Wavelet Transform. The Wavelet Transform has a high resolution in both the frequency- and the time-domain. It does not only tell us which frequencies are present in a signal, but also at which time these frequencies have occurred. This is accomplished by working with different scales. First we look at the signal with a large scale/window and analyze ‘large’ features and then we look at the signal with smaller scales in order to analyze smaller features.

The time- and frequency resolutions of the different methods are illustrated in Figure 2.

分析這種動態頻譜最好的方法就是使用小波變換。小波變換在頻域和時域上都有很好的表現。它不僅能夠告訴我們在信號種有哪些頻率出現，而且能夠告訴我們在信號的什麼時候出現。這是通過不同的伸縮變換實現的。首先，我們看一個大尺度/窗口的信號，並分析“大”特徵，然後我們看一個小尺度的信號，以便分析更小的特徵。圖2顯示了不同方法的時間分辨率和頻率分辨率。

圖2，與原始時間序列數據集相比，不同轉換的時間和頻率分辨率的示意圖概述。

In Figure 2 we can see the time and frequency resolutions of the different transformations. The size and orientation of the blocks indicate how small the features are that we can distinguish in the time and frequency domain. The original time-series has a high resolution in the time-domain and zero resolution in the frequency domain. This means that we can distinguish very small features in the time-domain and no features in the frequency domain.

圖2我們可以看到不同變換的時間分辨率和頻率分辨率。塊的大小和方向表明了我們可以在時域和頻域區分的特徵有多小。原始的時域信號有很高的時域分辨率，頻域分辨率爲零。這意味着我們能過分辨很小的時域特徵但是不能區分頻域特徵。

Opposite to that is the Fourier Transform, which has a high resolution in the frequency domain and zero resolution in the time-domain.

同樣，在頻譜種，有很高的頻域分辨率但是時域分辨率也是零。

The Short Time Fourier Transform has medium sized resolution in both the frequency and time domain.

短時傅里葉變換在頻域和時域均具有中等分辨率。

The Wavelet Transform has:

• for small frequency values a high resolution in the frequency domain, low resolution in the time- domain,

• for large frequency values a low resolution in the frequency domain, high resolution in the time domain.

小波變換有：

• 對於小頻率值，頻域分辨率高，時域分辨率低。

• 對於大頻率值，頻域分辨率低，時域分辨率高。

In other words, the Wavelet Transforms makes a trade-off; at scales in which time-dependent features are interesting it has a high resolution in the time-domain and at scales in which frequency-dependent features are interesting it has a high resolution in the frequency domain.

And as you can imagine, this is exactly the kind of trade-off we are looking for!

換句話說，小波變換是一種權衡，在與時間相關的特徵有趣的尺度上，它在時域具有高分辨率;在與頻率相關的特徵有趣的尺度上，它在頻域具有高分辨率。、

正如你所能想象的，這正是我們正在尋找的一種取捨!

相信大家都看過油畫。 對於特別巨幅的油畫， 不知道有沒有過體會， 油畫是隻可遠觀而不可褻玩？

• 當你在足夠遠的距離觀察油畫時， 油畫所表達的內容是有層次且內容豐富的。

• 當你靠近油畫甚至貼在油畫上看時， 你只能看到一個個的小色塊， 而此時這些小色塊此時變成毫無意義的無規則排列。

那我們如何來觀看這幅油畫呢?

我們假設油畫中的每個小色塊都對應某一信號中的某個瞬時時間，那麼無數個小色塊就拼湊成了整幅畫， 相當於無數個瞬時時間片段組成了整個信號。

那麼，當我們想看看在這個信號的某個時間發生了什麼的時候， 我們不得不試圖靠近這幅畫。

假設一個信號所包含的全部頻率信息包含在一副無邊無際的油畫之中， 辣麼， 當你和油畫的距離爲無窮遠的時候你能夠毫不費力的看清楚這幅畫的全貌，（相當於傅立葉變換，可獲得此信號的全部頻率信息）， 但是你卻無法看清楚這幅畫的細節（無法得知此信號在某個時刻的瞬時頻率）。

當你慢慢走向這幅畫的時候（給傅立葉變換加窗）， 你走得越近（窗長越短）， 細節越清楚（信號的時間分辨率越來越高）， 也就是每個小色塊都越來越清晰， 但每個色塊所包含的信息量（頻率信息）也越來越少，這就是著名的海森堡測不準原理（你永遠無法得知一個信號之中任一時刻的準確頻率信息， 即瞬時頻率）， 也就是說你永遠不可能在距離油畫0 cm的位置觀測出油畫所畫的內容。加窗傅立葉變換相當於你選擇了一個固定的距離觀察這幅油畫，當距離近時（窗長減少）， 油畫的細節更加清楚， 但是油畫所表達的信息量隨之減少。

這時候就有人提出來了，這個問題還不容易嗎？ 油畫在有的地方細節多，我們就離近一點看嘛（增大變換尺度，增加時間分辨率）；有的地方細節少，我們就離得稍微遠一點看嘛（減小變化尺度，縮減時間分辨率），這樣，在細節和內容的信息量上都能兼顧，能夠更全面的欣賞這幅油畫，這就是所謂的小波變換啦！

最後總結：

• 傅立葉變換就相當於： 你只能在遠距離觀察油畫；

• 加窗傅立葉變換相當於： 你只能在固定的距離觀察油畫；

• 而小波變換相當於，你可以在任意的距離觀察油畫。

  傅里葉變換示意圖

2.2 小波變換是如何工作的

The Fourier Transform uses a series of sine-waves with different frequencies to analyze a signal. That is, a signal is represented through a linear combination of sine-waves. The Wavelet Transform uses a series of functions called wavelets, each with a different scale. The word wavelet means a small wave, and this is exactly what a wavelet is.

傅里葉變換是通過一系列頻率不同的正弦波來擬合信號。也就是說，信號是通過正弦波的線性組合來表示的。

小波變換使用一系列稱爲小波的函數，每個函數具有不同的尺度。小波這個詞的意思就是很小的波，的確是這樣的。

圖3，正弦波於小波的不同。正弦波是不限延長的，小波則是在一個時間點上的局域波

In Figure 3 we can see the difference between a sine-wave and a wavelet. The main difference is that the sine-wave is not localized in time (it stretches out from -infinity to +infinity) while a wavelet is localized in time. This allows the wavelet transform to obtain time-information in addition to frequency information.

在圖3中，我們能看到正弦波於小波的不同。最主要的不同點是正弦波是一個非時間限制的（是從負無窮到正無窮）波，而小波則是一個時間點上的局域波。這使得小波包含不僅包含頻率信息也包含時間信息。

Since the Wavelet is localized in time, we can multiply our signal with the wavelet at different locations in time. We start with the beginning of our signal and slowly move the wavelet towards the end of the signal. This procedure is also known as a convolution. After we have done this for the original (mother) wavelet, we can scale it such that it becomes larger and repeat the process. This process is illustrated in the figure below.

既然小波是在時間上局域的，我們可以疊加不同時間段的小波。我們從信號開始的時間點慢慢想信號結尾移動小波。這個方法也是所說的卷積。在對原始（母）小波做完這些，我們可以改變小波尺寸，使得它變大然後繼續上面的操作。這個過程如下圖所示。

As we can see in the figure above, the Wavelet transform of an 1-dimensional signal will have two dimensions. This 2-dimensional output of the Wavelet transform is the time-scale representation of the signal in the form of a scaleogram.

由上圖可以，經過小波變換信號從1維變成2維。這種小波變換的2維輸出是以分佈圖的形式通過時間尺度變換表示的信號。

So what is this dimension called scale? Since the term frequency is reserved for the Fourier Transform, the wavelet transform is usually expressed in scales instead. That is why the two dimensions of a scaleogram are time and scale. For the ones who find frequencies more intuitive than scales, it is possible to convert scales to pseudo-frequencies with the equation

那麼這個scale是什麼維度？由於傅里葉變換保留了頻率項，所以小波變換通常用尺度來表示。這就是爲什麼二維分佈圖有時間和尺度。對於那些認爲頻率比尺度更直觀的人來說，用這個方程可以把尺度轉換成僞頻率。

where is the pseudo-frequency, is the central frequency of the Mother wavelet and is the scaling factor.

其中 是僞頻率， 是母小波的中心頻率， 是比例因子

We can see that a higher scale-factor (longer wavelet) corresponds with a smaller frequency, so by scaling the wavelet in the time-domain we will analyze smaller frequencies (achieve a higher resolution) in the frequency domain. And vice versa, by using a smaller scale we have more detail in the time-domain. So scales are basically the inverse of the frequency.

我們可以看到越是高的比例（長的小波）對應一個較小的頻率，因此在時間維度通過修改小波的比例因子我們可以在頻域分析更小的頻率（獲得更高的分辨率，但是包含的頻率信息就少了）。反之一樣，通過使用更小的比例因子我們可以在時間維度上獲得更多的細節。所以尺度基本上是頻率的倒數。

上面的關係也就是說，如果使用更小的比例我們在時域上得到的信息少，在頻域上得到的信息多；反之，如果使用更大的比例，在時域上得到的信息多，在頻域上得到的信息少。

2.3 小波變換族中的不同類型

Another difference between the Fourier Transform and the Wavelet Transform is that there are many different families (types) of wavelets. The wavelet families differ from each other since for each family a different trade-off has been made in how compact and smooth the wavelet looks like. This means that we can choose a specific wavelet family which fits best with the features we are looking for in our signal.

傅里葉變換和小波變換的另一個不同的地方是小波中有很多不同的族。小波族之間是不同的，因爲每個小波族在小波的緊湊性和平滑性方面都做出了不同的權衡。這意味着我們可以選擇一個特定的小波族，它最適合我們在信號中尋找的特徵。

import pywt
print(pywt.families(short=False))
['Haar', 'Daubechies', 'Symlets', 'Coiflets', 'Biorthogonal', 'Reverse biorthogonal', 
'Discrete Meyer (FIR Approximation)', 'Gaussian', 'Mexican hat wavelet', 'Morlet wavelet', 
'Complex Gaussian wavelets', 'Shannon wavelets', 'Frequency B-Spline wavelets', 'Complex Morlet wavelets']

Each type of wavelets has a different shape, smoothness and compactness and is useful for a different purpose. Since there are only two mathematical conditions a wavelet has to satisfy it is easy to generate a new type of wavelet.

不同種類的小波有着不同的形狀，平滑的和緊密的，用於不同的目的。由於小波只需要滿足兩個數學條件，因此很容易產生一種新的小波。

The two mathematical conditions are the so-called normalization and orthogonalization constraints:

這兩個數學條件就是所謂的歸一化約束和正交化約束

A wavelet must have 1) finite energy and 2) zero mean. Finite energy means that it is localized in time and frequency; it is integrable and the inner product between the wavelet and the signal always exists. The admissibility condition implies a wavelet has zero mean in the time-domain, a zero at zero frequency in the time-domain. This is necessary to ensure that it is integrable and the inverse of the wavelet transform can also be calculated.

小波必須有1)有限的能量和2)零均值。

有限能量意味着它在時間和頻率上是局域的;它是可積的，小波與信號的內積始終存在。容許條件意味着小波在時域中均值爲零，在時域中頻率爲零時均值爲零。這對於保證小波變換的可積性和計算小波變換的逆是必要的。

Furthermore:

• A wavelet can be orthogonal or non-orthogonal.

• A wavelet can be bi-orthogonal or not.

• A wavelet can be symmetric or not.

• A wavelet can be complex or real. If it is complex, it is usually divided into a real part representing the amplitude and an imaginary part representing the phase.

• A wavelets is normalized to have unit energy.

此外:

• 小波可以是正交的也可以是非正交的。

• 小波可以是雙正交的，也可以不是。

• 小波可以是對稱的，也可以不是。

• 小波可以是複數，也可以是實數。如果它是複數，通常分爲表示振幅的實部和表示相位的虛部。

• 小波被歸一化爲單位能量。

Below we can see a plot with several different families of wavelets. The first row contains four Discrete Wavelets and the second row four Continuous Wavelets.

下面代碼中我們看到不同小波族的圖像。第一行包含四個離散小波，第二行包含四個連續小波。

import pywt
from matplotlib import pyplot as plt
#print(pywt.families(short=False))
discrete_wavelets = ['db5', 'sym5', 'coif5', 'bior2.4']
continuous_wavelets = ['mexh', 'morl', 'cgau5', 'gaus5']
​
list_list_wavelets = [discrete_wavelets, continuous_wavelets]
list_funcs = [pywt.Wavelet, pywt.ContinuousWavelet]
​
fig, axarr = plt.subplots(nrows=2, ncols=4, figsize=(16,8))
for ii, list_wavelets in enumerate(list_list_wavelets):
    func = list_funcs[ii]
    row_no = ii
    for col_no, waveletname in enumerate(list_wavelets):
        wavelet = func(waveletname)
        family_name = wavelet.family_name
        biorthogonal = wavelet.biorthogonal
        orthogonal = wavelet.orthogonal
        symmetry = wavelet.symmetry
        if ii == 0:
            _ = wavelet.wavefun()
            wavelet_function = _[0]
            x_values = _[-1]
        else:
            wavelet_function, x_values = wavelet.wavefun()
        if col_no == 0 and ii == 0:
            axarr[row_no, col_no].set_ylabel("Discrete Wavelets", fontsize=16)
        if col_no == 0 and ii == 1:
            axarr[row_no, col_no].set_ylabel("Continuous Wavelets", fontsize=16)
        axarr[row_no, col_no].set_title("{}".format(family_name), fontsize=16)
        axarr[row_no, col_no].plot(x_values, wavelet_function)
        axarr[row_no, col_no].set_yticks([])
        axarr[row_no, col_no].set_yticklabels([])
​
plt.tight_layout()
plt.show()

圖5 第一行包含四個離散小波，第二行包含四個連續小波。

Within each wavelet family there can be a lot of different wavelet subcategories belonging to that family. You can distinguish the different subcategories of wavelets by the number of coefficients (the number of vanishing moments) and the level of decomposition.

在每個小波族中，可以有許多不同的小波子類別屬於這個族。你可以通過係數的數量(消失矩的數量)和分解的級別來區分小波的不同子類別。

This is illustrated below in for the one family of wavelets called ‘Daubechies’.

這是在下面爲一個族的小波稱爲“Daubechies”。

db_wavelets = pywt.wavelist('db')[:5]
print(db_wavelets)
*** ['db1', 'db2', 'db3', 'db4', 'db5']
​
fig, axarr = plt.subplots(ncols=5, nrows=5, figsize=(20,16))
fig.suptitle('Daubechies family of wavelets', fontsize=16)
for col_no, waveletname in enumerate(db_wavelets):
    wavelet = pywt.Wavelet(waveletname)
    no_moments = wavelet.vanishing_moments_psi
    family_name = wavelet.family_name
    for row_no, level in enumerate(range(1,6)):
        wavelet_function, scaling_function, x_values = wavelet.wavefun(level = level)
        axarr[row_no, col_no].set_title("{} - level {}\n{} vanishing moments\n{} samples".format(
            waveletname, level, no_moments, len(x_values)), loc='left')
        axarr[row_no, col_no].plot(x_values, wavelet_function, 'bD--')
        axarr[row_no, col_no].set_yticks([])
        axarr[row_no, col_no].set_yticklabels([])
plt.tight_layout()
plt.subplots_adjust(top=0.9)
plt.show()

圖6，多貝西小波族的幾個不同的消失時刻和幾個層次的細化。

In Figure 6 we can see wavelets of the ‘Daubechies’ family (db) of wavelets. In the first column we can see the Daubechies wavelets of the first order ( db1), in the second column of the second order (db2), up to the fifth order in the fifth column. PyWavelets contains Daubechies wavelets up to order 20 (db20).

圖6中我們看到多貝西小波族，第一列中，我們可以看到一階(db1)的Daubechies小波，在二階(db2)的第二列中，一直到第五列中的第五階。小波包含多達20階(db20)的Daubechies小波。

The number of the order indicates the number of vanishing moments. So db3 has three vanishing moments and db5 has 5 vanishing moment. The number of vanishing moments is related to the approximation order and smoothness of the wavelet. If a wavelet has p vanishing moments, it can approximate polynomials of degree p – 1.

階數表示消失力矩的個數。所以db3有3個消失力矩db5有5個消失力矩。消去矩的個數與小波的逼近階數和平滑度有關。如果一個小波有p個消失矩，它可以近似p - 1次多項式

When selecting a wavelet, we can also indicate what the level of decomposition has to be. By default, PyWavelets chooses the maximum level of decomposition possible for the input signal. The maximum level of decomposition (see pywt.dwt_max_level()) depends on the length of the input signal length and the wavelet (more on this later).

在選擇小波時，我們還可以指出分解的級別。默認情況下，PyWavelets爲輸入信號選擇可能的最大分解級別。分解的最大級別(參見pywt.dwt_max_level())取決於輸入信號長度和小波的長度。

As we can see, as the number of vanishing moments increases, the polynomial degree of the wavelet increases and it becomes smoother. And as the level of decomposition increases, the number of samples this wavelet is expressed in increases.

可以看出，隨着消失矩數的增加，小波的多項式階數增加，小波變得更加平滑。隨着分解程度的增加，這個小波的樣本數以增加的形式表示。

2.4 連續小波變換與離散小波變換

As we have seen before (Figure 5), the Wavelet Transform comes in two different and distinct flavors; the Continuous and the Discrete Wavelet Transform.

正如我們之前看到的(圖5)，小波變換有兩種不同的風格;連續小波變換和離散小波變換。

Mathematically, a Continuous Wavelet Transform is described by the following equation:

數學上，連續小波變換的表達式爲:

where is the continuous mother wavelet which gets scaled by a factor of and translated by a factor of . The values of the scaling and translation factors are continuous, which means that there can be an infinite amount of wavelets. You can scale the mother wavelet with a factor of 1.3, or 1.31, and 1.311, and 1.3111 etc.

其中爲連續母波，a是尺度因子，b爲平移因子。尺度因子和平移因子的值是連續的，這意味着可能有無限多的小波。你可以用1.3，或1.31,1.311，和1.3111等因子來縮放母小波。

When we are talking about the Discrete Wavelet Transform, the main difference is that the DWT uses discrete values for the scale and translation factor. The scale factor increases in powers of two, so and the translation factor increases integer values ( ).

離散小波變換和連續小波變換主要的區別是：離散小波變換對尺度和平移因子使用離散值。縮放因子a aa的冪增加爲2（a=1,2,4…… ），平移因子b bb增加整數值(b=1,2,3)。

2.5 離散小波變換的深入：DWT是一組濾波器

In practice, the DWT is always implemented as a filter-bank. This means that it is implemented as a cascade of high-pass and low-pass filters. This is because filter banks are a very efficient way of splitting a signal of into several frequency sub-bands. Below I will try to explain the concept behind the filter-bank in a simple (and probably oversimplified) way. It is necessary in order to understand how the wavelet transform actually works and can be used in practical applications.

在實際應用中，DWT總是被當作濾波器使用。這意味着它被實現爲高通濾波器和低通濾波器的級聯。這是因爲濾波器組是一種非常有效的方法，將一個信號分解成幾個子頻帶。

下面，我將嘗試以一種簡單(可能過於簡化)的方式解釋過濾器庫背後的概念。了更好地理解小波變換的實際工作原理和在實際應用中應用的必要性。

To apply the DWT on a signal, we start with the smallest scale. As we have seen before, small scales correspond with high frequencies. This means that we first analyze high frequency behavior. At the second stage, the scale increases with a factor of two (the frequency decreases with a factor of two), and we are analyzing behavior around half of the maximum frequency. At the third stage, the scale factor is four and we are analyzing frequency behavior around a quarter of the maximum frequency. And this goes on and on, until we have reached the maximum decomposition level.

要將DWT應用於信號，我們從最小的尺度開始。像我們之前看到的，小尺度的波對應高頻。這意味着我們先分析高頻行爲。在第二階段，尺度增加一倍(頻率減少一倍)。我們在分析最大頻率的一半左右的行爲。在第三階段，尺度因子爲4，我們正在分析最大頻率的四分之一左右的頻率行爲。這樣一直進行下去，直到達到最大分解水平。

What do we mean with maximum decomposition level? To understand this we should also know that at each subsequent stage the number of samples in the signal is reduced with a factor of two. At lower frequency values, you will need less samples to satisfy the Nyquist rate so there is no need to keep the higher number of samples in the signal; it will only cause the transform to be computationally expensive. Due to this downsampling, at some stage in the process the number of samples in our signal will become smaller than the length of the wavelet filter and we will have reached the maximum decomposition level.

最大分解水平是什麼意思?爲了理解這一點，我們還應該知道，在隨後的每個階段，信號中的樣本數量都減少了一倍。在較低的頻率值下，您將需要較少的樣本來滿足奈奎斯特速率，因此無需在信號中保留較高數量的樣本；它只會導致轉換的計算開銷。由於這種下采樣，在這個過程的某個階段，我們的信號中樣本的數量會小於小波濾波器的長度，我們將達到最大分解水平。

To give an example, suppose we have a signal with frequencies up to 1000 Hz. In the first stage we split our signal into a low-frequency part and a high-frequency part, i.e. 0-500 Hz and 500-1000 Hz. At the second stage we take the low-frequency part and again split it into two parts: 0-250 Hz and 250-500 Hz.

舉個例子，假設我們有一個頻率高達1000赫茲的信號。在第一個階段，我們把信號分成低頻部分和高頻部分，即0- 500hz和500- 1000hz。在第二階段，我們取低頻部分，再次將其分爲兩部分:0-250赫茲和250-500赫茲。

At the third stage we split the 0-250 Hz part into a 0-125 Hz part and a 125-250 Hz part. This goes on until we have reached the level of refinement we need or until we run out of samples.

We can easily visualize this idea, by plotting what happens when we apply the DWT on a chirp signal. A chirp signal is a signal with a dynamic frequency spectrum; the frequency spectrum increases with time. The start of the signal contains low frequency values and the end of the signal contains the high frequencies. This makes it easy for us to visualize which part of the frequency spectrum is filtered out by simply looking at the time-axis.

在第三階段，我們將0-250赫茲的部分分爲0-125赫茲的部分和125-250赫茲的部分。

這種情況會一直持續下去，直到我們達到了所需的精化水平，或者直到我們的樣本耗盡爲止。

我們可以很容易地把這個概念形象化，通過繪製當我們對chirp 信號應用DWT時會發生什麼。chirp信號是一種具有動態頻譜的信號；頻譜隨時間而增加。信號的開始包含低頻值，信號的結束包含高頻值。這使得我們很容易通過簡單地觀察時間軸來直觀地看到頻譜的哪一部分被過濾掉了。

import pywt
​
x = np.linspace(0, 1, num=2048)
chirp_signal = np.sin(250 * np.pi * x**2)
    
fig, ax = plt.subplots(figsize=(6,1))
ax.set_title("Original Chirp Signal: ")
ax.plot(chirp_signal)
plt.show()
    
data = chirp_signal
waveletname = 'sym5'
​
fig, axarr = plt.subplots(nrows=5, ncols=2, figsize=(6,6))
for ii in range(5):
    (data, coeff_d) = pywt.dwt(data, waveletname)
    axarr[ii, 0].plot(data, 'r')
    axarr[ii, 1].plot(coeff_d, 'g')
    axarr[ii, 0].set_ylabel("Level {}".format(ii + 1), fontsize=14, rotation=90)
    axarr[ii, 0].set_yticklabels([])
    if ii == 0:
        axarr[ii, 0].set_title("Approximation coefficients", fontsize=14)
        axarr[ii, 1].set_title("Detail coefficients", fontsize=14)
    axarr[ii, 1].set_yticklabels([])
plt.tight_layout()
plt.show()

圖7，應用於chirp信號的sym5小波(1 - 5級)的近似係數和細節係數，從1階到5階。在左邊，我們可以看到一個示意圖表示的高通和低通濾波器應用於信號的每一級。

In Figure 7 we can see our chirp signal, and the DWT applied to it subsequently. There are a few things to notice here:

• In PyWavelets the DWT is applied with pywt.dwt()

• The DWT return two sets of coefficients; the approximation coefficients and detail coefficients.

• The approximation coefficients represent the output of the low pass filter (averaging filter) of the DWT.

• The detail coefficients represent the output of the high pass filter (difference filter) of the DWT.

• By applying the DWT again on the approximation coefficients of the previous DWT, we get the wavelet transform of the next level.

• At each next level, the original signal is also sampled down by a factor of 2.

在圖7我們看到chirp信號以及後續的DWT變換。有一下幾點需要注意：

• 在小波中，DWT應用pywt.dwt()。

• DWT返回兩組係數；近似係數和細節係數。

• 近似係數表示小波變換的低通濾波器(平均濾波器)的輸出。

• 細節係數表示小波變換的高通濾波器(差分濾波器)的輸出。

• 通過對上一級小波變換的近似係數進行小波變換，得到下一級小波變換。

• 每下一層，原始信號的採樣率也下降2倍。

So now we have seen what it means that the DWT is implemented as a filter bank; At each subsequent level, the approximation coefficients are divided into a coarser low pass and high pass part and the DWT is applied again on the low-pass part.

至此我們看到了DWT作爲濾波器是什麼意思了。在每一階，將近似係數分爲較粗的低通和高通部分，並對低通部分再次進行小波變換。

As we can see, our original signal is now converted to several signals each corresponding to different frequency bands. Later on we will see how the approximation and detail coefficients at the different frequency sub-bands can be used in applications like removing high frequency noise from signals, compressing signals, or classifying the different types signals.

正如我們所見，我們的原始信號現在被轉換成幾個信號，每個信號對應不同的頻段。稍後，我們將看到在不同頻率子帶上的近似係數和細節係數如何應用於從信號中去除高頻噪聲、壓縮信號或對不同類型的信號進行分類等應用。

PS: We can also use pywt.wavedec() to immediately calculate the coefficients of a higher level. This functions takes as input the original signal and the level and returns the one set of approximation coefficients (of the n-th level) and n sets of detail coefficients (1 to n-th level).

備註：我們可以使用pywt.wavedec()函數立即計算出高階的係數。這個方法需要輸入原始信號，階數並且返回一組近似係數（對應n階的）和n組細節係數（對應1到n階）

3.1利用連續小波變換實現狀態空間的可視化

So far we have seen what the wavelet transform is, how it is different from the Fourier Transform, what the difference is between the CWT and the DWT, what types of wavelet families there are, what the impact of the order and level of decomposition is on the mother wavelet, and how and why the DWT is implemented as a filter-bank.

至此我們看到了什麼是小波變換，它與傅里葉變換的區別是什麼，而其中CWT和DWT的區別是什麼，小波族的類型都有什麼，分解的階數和層次對母波的影響是什麼，小波變換如何以及爲什麼被實現爲一個濾波器庫。

We have also seen that the output of a wavelet transform on a 1D signal results in a 2D scaleogram. Such a scaleogram gives us detailed information about the state-space of the system, i.e. it gives us information about the dynamic behavior of the system.

在2D散點圖中我們已經看到了小波變換1維信號的輸出。這樣的比例圖爲我們提供了關於系統狀態空間的詳細信息，例如，它給我們展示系統中動態特徵的信息

The el-Nino dataset is a time-series dataset used for tracking the El Nino and contains quarterly measurements of the sea surface temperature from 1871 up to 1997. In order to understand the power of a scaleogram, let us visualize it for el-Nino dataset together with the original time-series data and its Fourier Transform.

el-Nino數據集是一個時間序列的數據集用於追蹤記錄EI Nino現象和包括從1871年至1997年每季度的海面溫度測量。爲了理解尺度圖的威力，讓我們將它與原始的時間序列數據及其傅里葉變換一起可視化。

def plot_wavelet(time, signal, scales, 
                 waveletname = 'cmor', 
                 cmap = plt.cm.seismic, 
                 title = 'Wavelet Transform (Power Spectrum) of signal', 
                 ylabel = 'Period (years)', 
                 xlabel = 'Time'):
    
    dt = time[1] - time[0]
    [coefficients, frequencies] = pywt.cwt(signal, scales, waveletname, dt)
    power = (abs(coefficients)) ** 2
    period = 1. / frequencies
    levels = [0.0625, 0.125, 0.25, 0.5, 1, 2, 4, 8]
    contourlevels = np.log2(levels)
    
    fig, ax = plt.subplots(figsize=(15, 10))
    im = ax.contourf(time, np.log2(period), np.log2(power), contourlevels, extend='both',cmap=cmap)
    
    ax.set_title(title, fontsize=20)
    ax.set_ylabel(ylabel, fontsize=18)
    ax.set_xlabel(xlabel, fontsize=18)
    
    yticks = 2**np.arange(np.ceil(np.log2(period.min())), np.ceil(np.log2(period.max())))
    ax.set_yticks(np.log2(yticks))
    ax.set_yticklabels(yticks)
    ax.invert_yaxis()
    ylim = ax.get_ylim()
    ax.set_ylim(ylim[0], -1)
    
    cbar_ax = fig.add_axes([0.95, 0.5, 0.03, 0.25])
    fig.colorbar(im, cax=cbar_ax, orientation="vertical")
    plt.show()
​
def plot_signal_plus_average(time, signal, average_over = 5):
    fig, ax = plt.subplots(figsize=(15, 3))
    time_ave, signal_ave = get_ave_values(time, signal, average_over)
    ax.plot(time, signal, label='signal')
    ax.plot(time_ave, signal_ave, label = 'time average (n={})'.format(5))
    ax.set_xlim([time[0], time[-1]])
    ax.set_ylabel('Signal Amplitude', fontsize=18)
    ax.set_title('Signal + Time Average', fontsize=18)
    ax.set_xlabel('Time', fontsize=18)
    ax.legend()
    plt.show()
    
def get_fft_values(y_values, T, N, f_s):
    f_values = np.linspace(0.0, 1.0/(2.0*T), N//2)
    fft_values_ = fft(y_values)
    fft_values = 2.0/N * np.abs(fft_values_[0:N//2])
    return f_values, fft_values
​
def plot_fft_plus_power(time, signal):
    dt = time[1] - time[0]
    N = len(signal)
    fs = 1/dt
    
    fig, ax = plt.subplots(figsize=(15, 3))
    variance = np.std(signal)**2
    f_values, fft_values = get_fft_values(signal, dt, N, fs)
    fft_power = variance * abs(fft_values) ** 2     # FFT power spectrum
    ax.plot(f_values, fft_values, 'r-', label='Fourier Transform')
    ax.plot(f_values, fft_power, 'k--', linewidth=1, label='FFT Power Spectrum')
    ax.set_xlabel('Frequency [Hz / year]', fontsize=18)
    ax.set_ylabel('Amplitude', fontsize=18)
    ax.legend()
    plt.show()
​
dataset = "http://paos.colorado.edu/research/wavelets/wave_idl/sst_nino3.dat"
df_nino = pd.read_table(dataset)
N = df_nino.shape[0]
t0=1871
dt=0.25
time = np.arange(0, N) * dt + t0
signal = df_nino.values.squeeze()
​
scales = np.arange(1, 128)
plot_signal_plus_average(time, signal)
plot_fft_plus_power(time, signal)
plot_wavelet(time, signal, scales)

圖8，el-Nino數據（最上），經過傅里葉變換後（中間），連續小波變換（底部）

In Figure 8 we can see in the top figure the el-Nino dataset together with its time average, in the middle figure the Fourier Transform and at the bottom figure the scaleogram produced by the Continuous Wavelet Transform.

在圖8中，我們能在頂圖中看到有時間均值的el-Nino數據集，在中間的圖是傅里葉變換的結果，在最底下的圖是連續小波變換得到的尺度圖。

In the scaleogram we can see that most of the power is concentrated in a 2-8 year period. If we convert this to frequency (T = 1 / f) this corresponds with a frequency of 0.125 – 0.5 Hz. The increase in power can also be seen in the Fourier transform around these frequency values. The main difference is that the wavelet transform also gives us temporal information and the Fourier Transform does not. For example, in the scaleogram we can see that up to 1920 there were many fluctuations, while there were not so much between 1960 – 1990. We can also see that there is a shift from shorter to longer periods as time progresses. This is the kind of dynamic behavior in the signal which can be visualized with the Wavelet Transform but not with the Fourier Transform.

在標度圖中我們可以看到大部分的能量集中在2-8年的時間內。如果我們把它轉換成頻率(T = 1 / f)它對應的頻率是0.125 - 0.5 Hz。功率的增加也可以在這些頻率值附近的傅里葉變換中看到。主要的區別是小波變換也能給出時間信息，而傅里葉變換不能。例如，在標度圖中我們可以看到，在1920年之前有很多波動，而在1960 - 1990年之間沒有那麼多波動。我們還可以看到，隨着時間的推移，從較短的週期向較長的週期轉變。這是一種信號的動態行爲可以用小波變換來表示但不能用傅里葉變換來表示。

This should already make clear how powerful the wavelet transform can be for machine learning purposes. But to make the story complete, let us also look at how this can be used in combination with a Convolutional Neural Network to classify signals.

應該已經清楚地表明小波變換對於機器學習的作用有多大。但是爲了讓這篇文章更完整，讓我們看看如何結合卷積神經網絡來對信號進行分類。

3.2利用連續小波變換和卷積神經網絡對信號進行分類

In section 3.1 we have seen that the wavelet transform of a 1D signal results in a 2D scaleogram which contains a lot more information than just the time-series or just the Fourier Transform. We have seen that applied on the el-Nino dataset, it can not only tell us what the period is of the largest oscillations, but also when these oscillations were present and when not.

在3.1節我們已經看到一維信號的小波變換得到二維尺度圖，它包含了比時間序列或傅里葉變換多得多的信息。我們看到小波變換在el-Nino數據集上的應用，它不僅能告訴我們最大振盪的週期是多少，還能告訴我們這些振盪是什麼時候出現的，什麼時候沒有。

Such a scaleogram can not only be used to better understand the dynamical behavior of a system, but it can also be used to distinguish different types of signals produced by a system from each other. If you record a signal while you are walking up the stairs or down the stairs, the scaleograms will look different. ECG measurements of people with a healthy heart will have different scaleograms than ECG measurements of people with arrhythmia. Or measurements on a bearing, motor, rotor, ventilator, etc when it is faulty vs when it not faulty. The possibilities are limitless!

這種尺度圖不僅能用於更好的理解分析系統的動態特點，而且它還可以用來區分系統產生的不同類型的信號。如果你記錄你上樓和下樓的信號，這個尺度圖就會不同。正常人的心電圖和心律不齊的人的心電圖的尺度圖會不一樣。或測量一個軸承、電機、轉子、通風機等，當它有故障時與它沒有故障時。可能性是無限的!

So by looking at the scaleograms we can distinguish a broken motor from a working one, a healthy person from a sick one, a person walking up the stairs from a person walking down the stairs, etc etc. But if you are as lazy as me, you probably don’t want to sift through thousands of scaleograms manually. One way to automate this process is to build a Convolutional Neural Network which can automatically detect the class each scaleogram belongs to and classify them accordingly.

所以通過看比例圖我們可以區分出一個壞的馬達和一個工作的馬達，一個健康的人和一個生病的人，一個上樓的人和一個下樓的人等等。但是如果你像我一樣懶，您可能不想手動篩選數千個標度圖。實現這一過程自動化的一種方法是建立卷積神經網絡，該網絡可以自動檢測每個尺度圖所屬的類並對其進行分類。

In the next few sections we will load a dataset (containing measurement of people doing six different activities), visualize the scaleograms using the CWT and then use a Convolutional Neural Network to classify these scaleograms.

在接下來的幾章裏我會加載數據（包含對從事六種不同活動的人的測量），通過CWT可視化尺度圖，並且利用卷積神經網絡對尺度圖進行分類。

3.2.1加載UCI-HAR時序數據集

Let us try to classify an open dataset containing time-series using the scaleograms and a CNN. The Human Activity Recognition Dataset (UCI-HAR) contains sensor measurements of people while they were doing different types of activities, like walking up or down the stairs, laying, standing, walking, etc. There are in total more than 10.000 signals where each signal consists of nine components (x acceleration, y acceleration, z acceleration, x,y,z gyroscope, etc). This is the perfect dataset for us to try our use case of CWT + CNN!

讓我們嘗試利用尺度圖和CNN來對一個開源的時序數據集進行分類。 Human Activity Recognition Dataset (UCI-HAR) 包含通過傳感器測量的人類行爲，而他們正在做不同類型的活動，如上樓或下樓，躺下，站着，步行等。現有10000多個信號，每個信號由9個分量(x加速度、y加速度、z加速度、x、y、z陀螺儀等)組成。這個是我們利用CWT和CNN最完美的數據集。

---

 

UCI-HAR數據集是由30名年齡在19歲至48歲之間的受試者收集的，他們穿着齊腰的智能手機記錄運動數據，進行了六項標準活動：

1. Walking

2. Walking Upstairs

3. Walking Downstairs

4. Sitting

5. Standing

6. Laying

   移動數據是來自智能手機的x、y和z加速計數據（線性加速度）和陀螺儀數據（角速度），記錄頻率爲50赫茲（即每秒50個數據點）。每個受試者依次執行兩次活動，分別將設備置於左手邊和右手邊。原始數據經過了一些預處理：

   • 使用噪音過濾器預處理加速計和陀螺儀信號。

   • 將數據分割爲2.56秒（128個數據點）的固定窗口，窗口間有50%的重疊。

   • 將加速度計數據分解爲重力和身體活動兩部分。

      

After we have downloaded the data, we can load it into a numpy nd-array in the following way:

下載完數據集後，我們可以如下利用numpy將數據集轉換爲nd-array格式

def read_signals_ucihar(filename):
    with open(filename, 'r') as fp:
        data = fp.read().splitlines()
        data = map(lambda x: x.rstrip().lstrip().split(), data)
        data = [list(map(float, line)) for line in data]
    return data
​
def read_labels_ucihar(filename):        
    with open(filename, 'r') as fp:
        activities = fp.read().splitlines()
        activities = list(map(int, activities))
    return activities
​
def load_ucihar_data(folder):
    train_folder = folder + 'train/InertialSignals/'
    test_folder = folder + 'test/InertialSignals/'
    labelfile_train = folder + 'train/y_train.txt'
    labelfile_test = folder + 'test/y_test.txt'
    train_signals, test_signals = [], []
    for input_file in os.listdir(train_folder):
        signal = read_signals_ucihar(train_folder + input_file)
        train_signals.append(signal)
    train_signals = np.transpose(np.array(train_signals), (1, 2, 0))
    for input_file in os.listdir(test_folder):
        signal = read_signals_ucihar(test_folder + input_file)
        test_signals.append(signal)
    test_signals = np.transpose(np.array(test_signals), (1, 2, 0))
    train_labels = read_labels_ucihar(labelfile_train)
    test_labels = read_labels_ucihar(labelfile_test)
    return train_signals, train_labels, test_signals, test_labels
​
folder_ucihar = './data/UCI_HAR/' 
train_signals_ucihar, train_labels_ucihar, test_signals_ucihar, test_labels_ucihar = load_ucihar_data(folder_ucihar)

The training set contains 7352 signals where each signal has 128 measurement samples and 9 components. The signals from the training set are loaded into a numpy ndarray of size (7352 , 128, 9) and the signals from the test set into one of size (2947 , 128, 9).

訓練集包含7352個信號，每個信號包含128個測量樣本和9個分量。將訓練集轉換爲格式（7352，128，9）的nd-array，並且測試集轉換爲（2947 , 128, 9）

Since the signal consists of nine components we have to apply the CWT nine times for each signal. Below we can see the result of the CWT applied on two different signals from the dataset. The left one consist of a signal measured while walking up the stairs and the right one is a signal measured while laying down.

由於信號由9個分量組成，我們必須對每個信號應用9次CWT,下面我們可以看到CWT應用於來自數據集的兩個不同信號的結果。左邊的是上樓時測量的信號，右邊的是躺下時測量的信號。

圖9，CWT應用於UCI HAR數據集的兩個信號。每個信號有九個不同的分量。在左邊我們可以看到上樓時測量到的信號，在右邊我們可以看到鋪設時測量到的信號。

3.2.2 對數據集機型CWT並且將數據轉換成正確的格式

Since each signal has nine components, each signal will also have nine scaleograms. So the next question to ask is, how do we feed this set of nine scaleograms into a Convolutional Neural Network? There are several options we could follow:

因爲每個信號有9個分量，所以每個信號也有9個標度圖。下一個要問的問題是，我們如何將這9個尺度圖輸入卷積神經網絡?我們有幾個選擇:

1. Train a CNN for each component separately and combine the results of the nine CNN’s in some sort of an ensembling method. I suspect that this will generally result in a poorer performance since the inter dependencies between the different components are not taken into account.

2. Concatenate the nine different signals into one long signal and apply the CWT on the concatenated signal. This could work but there will be discontinuities at location where two signals are concatenated and this will introduced noise in the scaleogram at the boundary locations of the component signals.

3. Calculate the CWT first and thén concatenate the nine different CWT images into one and feed that into the CNN. This could also work, but here there will also be discontinuities at the boundaries of the CWT images which will feed noise into the CNN. If the CNN is deep enough, it will be able to distinguish between these noisy parts and actually useful parts of the image and choose to ignore the noise. But I still prefer option number four:

4. Place the nine scaleograms on top of each other and create one single image with nine channels. What does this mean? Well, normally an image has either one channel (grayscale image) or three channels (color image), but our CNN can just as easily handle images with nine channels. The way the CNN works remains exactly the same, the only difference is that there will be three times more filters compared to an RGB image.

1. 針對每個組件分別訓練一個CNN，並以某種綜合方法將9個CNN的結果組合起來。我懷疑這通常會導致較差的性能，因爲沒有考慮到不同組件之間的相互依賴關係。

2. 將9個不同的信號串接成一個長信號，並對串接的信號進行CWT處理。這是可行的，但在連接兩個信號的位置會有不連續，這將會在分量信號的邊界位置的尺度圖中引入噪聲。

3. 首先計算CWT，然後將九幅不同的CWT圖像拼接成一幅，並將其輸入CNN。這也可以工作，但在這裏也會有不連續的CWT圖像的邊界，這將提供噪聲到CNN。如果CNN足夠深，它將能夠區分這些噪聲部分和圖像的實際有用部分，並選擇忽略噪聲。但我還是傾向於選擇四:

4. 將9個尺度圖相互疊加，創建一個帶有9個通道的圖像。這是什麼意思?通常一個圖像要麼有一個通道(灰度圖像)要麼有三個通道(彩色圖像)，但是我們的CNN可以很容易地處理9個通道的圖像。CNN的工作方式是完全一樣的，唯一的區別是，與RGB圖像相比，將有三倍多的過濾器。

圖10，我們將使用9個分量中的9個CWT來創建一個用於卷積神經網絡輸入的9通道圖像

Below we can see the Python code on how to apply the CWT on the signals in the dataset, and reformat it in such a way that it can be used as input for our Convolutional Neural Network. The total dataset contains over 10.000 signals, but we will only use 5.000 signals in our training set and 500 signals in our test set.

下面我們會看到Python是如何將數據集中的信號進行CWT的，並且將轉換結果轉換成下面卷積神經網絡輸入用的格式。整個數據集包含超過10000個信號，但是我們只用5000個信號用於訓練，500個信號用於測試。

scales = range(1,128)
waveletname = 'morl'
train_size = 5000
test_size= 500
​
train_data_cwt = np.ndarray(shape=(train_size, 127, 127, 9))
​
for ii in range(0,train_size):
    if ii % 1000 == 0:
        print(ii)
    for jj in range(0,9):
        signal = uci_har_signals_train[ii, :, jj]
        coeff, freq = pywt.cwt(signal, scales, waveletname, 1)
        coeff_ = coeff[:,:127]
        train_data_cwt[ii, :, :, jj] = coeff_
​
test_data_cwt = np.ndarray(shape=(test_size, 127, 127, 9))
for ii in range(0,test_size):
    if ii % 100 == 0:
        print(ii)
    for jj in range(0,9):
        signal = uci_har_signals_test[ii, :, jj]
        coeff, freq = pywt.cwt(signal, scales, waveletname, 1)
        coeff_ = coeff[:,:127]
        test_data_cwt[ii, :, :, jj] = coeff_
​
uci_har_labels_train = list(map(lambda x: int(x) - 1, uci_har_labels_train))
uci_har_labels_test = list(map(lambda x: int(x) - 1, uci_har_labels_test))
​
x_train = train_data_cwt
y_train = list(uci_har_labels_train[:train_size])
x_test = test_data_cwt
y_test = list(uci_har_labels_test[:test_size])

As you can see above, the CWT of a single signal-component (128 samples) results in an image of 127 by 127 pixels. So the scaleograms coming from the 5000 signals of the training dataset are stored in an numpy ndarray of size (5000, 127, 127, 9) and the scaleograms coming from the 500 test signals are stored in one of size (500, 127, 127, 9).

上面代碼所示，單個信號分量(128個樣本)的CWT得到127×127像素的圖像。因此來自訓練數據集的5000個信號的尺度圖存儲在一個大小爲(5000、127、127、9)的numpy nd-array中，來自500個測試信號的尺度圖存儲在一個大小爲(500、127、127、9)的numpy nd-array中。

3.2.3利用CWT進行卷積神經網絡訓練

Now that we have the data in the right format, we can start with the most interesting part of this section: training the CNN! For this part you will need the keras library, so please install it first.

現在我們已經將數據集轉換爲正確的格式，我們可以從本節最有趣的部分開始:訓練CNN！這節中你將要用到keras庫，首先請下載安裝這庫。

import keras
from keras.layers import Dense, Flatten
from keras.layers import Conv2D, MaxPooling2D
from keras.models import Sequential
from keras.callbacks import History 
history = History()

img_x = 127
img_y = 127
img_z = 9
input_shape = (img_x, img_y, img_z)

num_classes = 6
batch_size = 16
num_classes = 7
epochs = 10

x_train = x_train.astype('float32')
x_test = x_test.astype('float32')

y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)


model = Sequential()
model.add(Conv2D(32, kernel_size=(5, 5), strides=(1, 1),
                 activation='relu',
                 input_shape=input_shape))
model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))
model.add(Conv2D(64, (5, 5), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(1000, activation='relu'))
model.add(Dense(num_classes, activation='softmax'))

model.compile(loss=keras.losses.categorical_crossentropy,
              optimizer=keras.optimizers.Adam(),
              metrics=['accuracy'])


model.fit(x_train, y_train,
          batch_size=batch_size,
          epochs=epochs,
          verbose=1,
          validation_data=(x_test, y_test),
          callbacks=[history])

train_score = model.evaluate(x_train, y_train, verbose=0)
print('Train loss: {}, Train accuracy: {}'.format(train_score[0], train_score[1]))
test_score = model.evaluate(x_test, y_test, verbose=0)
print('Test loss: {}, Test accuracy: {}'.format(test_score[0], test_score[1]))

*** Epoch 1/10
*** 5000/5000 [==============================] - 235s 47ms/step - loss: 0.3963 - acc: 0.8876 - val_loss: 0.6006 - val_acc: 0.8780
*** Epoch 2/10
*** 5000/5000 [==============================] - 228s 46ms/step - loss: 0.1939 - acc: 0.9282 - val_loss: 0.3952 - val_acc: 0.8880
*** Epoch 3/10
*** 5000/5000 [==============================] - 224s 45ms/step - loss: 0.1347 - acc: 0.9434 - val_loss: 0.4367 - val_acc: 0.9100
*** Epoch 4/10
*** 5000/5000 [==============================] - 228s 46ms/step - loss: 0.1971 - acc: 0.9334 - val_loss: 0.2662 - val_acc: 0.9320
*** Epoch 5/10
*** 5000/5000 [==============================] - 231s 46ms/step - loss: 0.1134 - acc: 0.9544 - val_loss: 0.2131 - val_acc: 0.9320
*** Epoch 6/10
*** 5000/5000 [==============================] - 230s 46ms/step - loss: 0.1285 - acc: 0.9520 - val_loss: 0.2014 - val_acc: 0.9440
*** Epoch 7/10
*** 5000/5000 [==============================] - 232s 46ms/step - loss: 0.1339 - acc: 0.9532 - val_loss: 0.2884 - val_acc: 0.9300
*** Epoch 8/10
*** 5000/5000 [==============================] - 237s 47ms/step - loss: 0.1503 - acc: 0.9488 - val_loss: 0.3181 - val_acc: 0.9340
*** Epoch 9/10
*** 5000/5000 [==============================] - 250s 50ms/step - loss: 0.1247 - acc: 0.9504 - val_loss: 0.2403 - val_acc: 0.9460
*** Epoch 10/10
*** 5000/5000 [==============================] - 238s 48ms/step - loss: 0.1578 - acc: 0.9508 - val_loss: 0.2133 - val_acc: 0.9300
*** Train loss: 0.11115437872409821, Train accuracy: 0.959
*** Test loss: 0.21326758581399918, Test accuracy: 0.93

As you can see, combining the Wavelet Transform and a Convolutional Neural Network leads to an awesome and amazing result!

We have an accuracy of 94% on the Activity Recognition dataset. Much higher than you can achieve with any other method. In section 3.5 we will use the Discrete Wavelet Transform instead of Continuous Wavelet Transform to classify the same dataset and achieve similarly amazing results!

正如你所看到的，小波變換和卷積神經網絡結合得到了一個很棒的結果！

我們對活動識別數據集的準確率爲94%。比任何其他方法都要高得多。

在3.5節中，我們將使用離散小波變換代替連續小波變換對相同的數據集進行分類，得到同樣驚人的結果!

3.3用小波分解信號

In Section 2.5 we have seen how the DWT is implemented as a filter-bank which can deconstruct a signal into its frequency sub-bands. In this section, let us see how we can use PyWavelets to deconstruct a signal into its frequency sub-bands and reconstruct the original signal again.

在2.5節，我們已經看到小波變換是如何作爲一個濾波器庫來實現的，它可以將一個信號分解成它的子頻帶。這節中，讓我們看看如何使用小波將一個信號分解成它的子頻帶，並重新構造原始信號。

PyWavelets offers two different ways to deconstruct a signal.

1. We can either apply pywt.dwt() on a signal to retrieve the approximation coefficients. Then apply the DWT on the retrieved coefficients to get the second level coefficients and continue this process until you have reached the desired decomposition level.

2. Or we can apply pywt.wavedec() directly and retrieve all of the the detail coefficients up to some level . This functions takes as input the original signal and the level and returns the one set of approximation coefficients (of the n-th level) and n sets of detail coefficients (1 to n-th level).

PyWavelets 提供了2中解析信號的方法。

1. 我們可以對信號應用pywt.dwt()來檢索近似係數。然後對檢索到的係數應用DWT以獲得第二級係數，並繼續此過程，直到達到所需的分解級別

2. 或我們可以應用pywt.wavedec()直接和檢索所有的細節係數達到某種程度上n。這個函數作爲輸入原始信號和n水平並返回一組近似係數(第n個水平)和n組細節係數(1到n級)。

(cA1, cD1) = pywt.dwt(signal, 'db2', 'smooth')
reconstructed_signal = pywt.idwt(cA1, cD1, 'db2', 'smooth')

fig, ax = plt.subplots(figsize=(8,4))
ax.plot(signal, label='signal')
ax.plot(reconstructed_signal, label='reconstructed signal', linestyle='--')
ax.legend(loc='upper left')
plt.show()

圖12，一個信號和重建的信號

Above we have deconstructed a signal into its coefficients and reconstructed it again using the inverse DWT.

面我們將一個信號分解成它的係數，然後再用反DWT對其進行重構。

The second way is to use pywt.wavedec() to deconstruct and reconstruct a signal and it is probably the most simple way if you want to get higher-level coefficients.

第二種方式是使用pywt.wavedec()來解構和重構一個信號，如果您想獲得更高級別的係數，這可能是最簡單的方法。

coeffs = pywt.wavedec(signal, 'db2', level=8)
reconstructed_signal = pywt.waverec(coeffs, 'db2')

fig, ax = plt.subplots(figsize=(8,4))
ax.plot(signal[:1000], label='signal')
ax.plot(reconstructed_signal[:1000], label='reconstructed signal', linestyle='--')
ax.legend(loc='upper left')
ax.set_title('de- and reconstruction using wavedec()')
plt.show()

圖13，一個信號和其重建的信號

3.4使用DWT去除噪聲（高頻）噪聲

In the previous section we have seen how we can deconstruct a signal into the approximation (low pass) and detail (high pass) coefficients. If we reconstruct the signal using these coefficients we will get the original signal back.

在前一節中，我們已經看到了如何將信號分解爲近似(低通)和細節(高通)係數。如果我們用這些係數重建信號，我們將得到原始信號。

But what happens if we reconstruct while we leave out some detail coefficients? Since the detail coefficients represent the high frequency part of the signal, we will simply have filtered out that part of the frequency spectrum. If you have a lot of high-frequency noise in your signal, this one way to filter it out.

如果我們在重建的過程中丟掉一些細節係數會怎麼樣呢？既然細節係數代表的是信號中的高頻部分，我們只需要過濾掉這部分頻譜，如果你的信號中有很多高頻噪聲，這是一種過濾的方法。

Leaving out of the detail coefficients can be done using pywt.threshold(), which removes coefficient values higher than the given threshold.

可以使用pywt.threshold()刪除細節係數，它刪除了高於給定閾值的係數值。

Lets demonstrate this using NASA’s Femto Bearing Dataset. This is a dataset containing high frequency sensor data regarding accelerated degradation of bearings.

讓我們用NASA的飛向軸承數據集來證明這一點。這是一個包含關於軸承加速退化的高頻傳感器數據集。

DATA_FOLDER = './FEMTO_bearing/Training_set/Bearing1_1/'
filename = 'acc_01210.csv'
df = pd.read_csv(DATA_FOLDER + filename, header=None)
signal = df[4].values

def lowpassfilter(signal, thresh = 0.63, wavelet="db4"):
    thresh = thresh*np.nanmax(signal)
    coeff = pywt.wavedec(signal, wavelet, mode="per" )
    coeff[1:] = (pywt.threshold(i, value=thresh, mode="soft" ) for i in coeff[1:])
    reconstructed_signal = pywt.waverec(coeff, wavelet, mode="per" )
    return reconstructed_signal

fig, ax = plt.subplots(figsize=(12,8))
ax.plot(signal, color="b", alpha=0.5, label='original signal')
rec = lowpassfilter(signal, 0.4)
ax.plot(rec, 'k', label='DWT smoothing}', linewidth=2)
ax.legend()
ax.set_title('Removing High Frequency Noise with DWT', fontsize=18)
ax.set_ylabel('Signal Amplitude', fontsize=16)
ax.set_xlabel('Sample No', fontsize=16)
plt.show()

圖14，一個高頻信號和它的DWT平滑變換圖

As we can see, by deconstructing the signal, setting some of the coefficients to zero, and reconstructing it again, we can remove high frequency noise from the signal.

正如我們所看到的，通過解構信號，把一些係數設爲零，然後重新構造，我們可以從信號中去除高頻噪聲。

People who are familiar with signal processing techniques might know there are a lot of different ways to remove noise from a signal. For example, the Scipy library contains a lot of smoothing filters (one of them is the famous Savitzky-Golay filter) and they are much simpler to use. Another method for smoothing a signal is to average it over its time-axis.

熟悉信號處理的人也許知道有很多不同的方法去除信號中的噪聲。例如，例如，Scipy庫包含許多平滑過濾器(其中之一是著名的Savitzky-Golay過濾器)，它們使用起來要簡單得多.另一種平滑信號的方法是對其時間軸進行平均。

So why should you use the DWT instead? The advantage of the DWT again comes from the many wavelet shapes there are available. You can choose a wavelet which will have a shape characteristic to the phenomena you expect to see. In this way, less of the phenomena you expect to see will be smoothed out.

那麼爲什麼要使用DWT呢？小波變換的優勢又一次來自於現有的許多小波形狀。你可以選擇一個小波，它將有一個形狀特徵的現象，你希望看到。這樣，你所期望看到的現象就會減少。

3.4使用離散小波變換進行信號分類

In Section 3.2 we have seen how we can use a the CWT and a CNN to classify signals. Of course it is also possible to use the DWT to classify signals. Let us have a look at how this could be done.

在3.2節，我們看到是如何使用連續小波變換和卷積神經網絡來對信號進行分類的。當然用離散小波變換來分類信號也是可以的。讓我們看一下是如何實現的。

3.5.1離散小波分類的思想

The idea behind DWT signal classification is as follows: The DWT is used to split a signal into different frequency sub-bands, as many as needed or as many as possible. If the different types of signals exhibit different frequency characteristics, this difference in behavior has to be exhibited in one of the frequency sub-bands. So if we generate features from each of the sub-band and use the collection of features as an input for a classifier (Random Forest, Gradient Boosting, Logistic Regression, etc) and train it by using these features, the classifier should be able to distinguish between the different types of signals.

離散小波變換分類思想如下：DWT用於將一個信號分割成不同的子頻帶，儘可能多的子頻帶或儘可能多的子頻帶。如果不同類型的信號表現出不同的頻率特性，這種行爲上的差異必須表現在一個頻率子帶中。因此，如果我們從每個子帶生成特徵，並將這些特徵集合作爲分類器的輸入(Random Forest, Gradient boost, Logistic Regression等)，並利用這些特徵訓練分類器，那麼分類器應該能夠區分不同類型的信號。

圖15，將信號進行離散小波變換，我們可以分解它的頻率子帶。並從每個子帶中提取特徵，作爲分類器的輸入。

3.5.2爲每個子帶生成特徵

So what kind of features can be generated from the set of values for each of the sub-bands? Of course this will highly depend on the type of signal and the application. But in general, below are some features which are most frequently used for signals.

那麼，可以從每個子帶的值集生成什麼樣的特性呢?當然，這在很大程度上取決於信號的類型和應用程序。但是一般來說，下面是一些最常用於信號的特性。

• Auto-regressive model coefficient values

• (Shannon) Entropy values; entropy values can be taken as a measure of complexity of the signal.

• Statistical features like:

  • variance

  • standard deviation

  • Mean

  • Median

  • 25th percentile value

  • 75th percentile value

  • Root Mean Square value; square of the average of the squared amplitude values

  • The mean of the derivative

  • Zero crossing rate, i.e. the number of times a signal crosses y = 0

  • Mean crossing rate, i.e. the number of times a signal crosses y = mean(y)

• 自迴歸模型係數值

• (Shannon)熵值;熵值可以用來衡量信號的複雜度。

• 統計特徵如：

  方差

  標準差

  均值

  中值

  第25百分位值

  第75個百分位值

  均方根值;振幅平方值平均值的平方

  導數的均值

  過零率，即信號通過y = 0的次數

These are just some ideas you could use to generate the features out of each sub-band. You could use some of the features described here, or you could use all of them. Most classifiers in the scikit-learn package are powerful enough to handle a large number of input features and distinguish between useful ones and non-useful ones. However, I still recommend you think carefully about which feature would be useful for the type of signal you are trying to classify.

這些只是一些想法，你可以用來產生的特點，從每個子帶。您可以使用這裏描述的一些特性，也可以使用所有這些特性。scikit-learn包中的大多數分類器都足夠強大，可以處理大量的輸入特性，並區分有用的和無用的。但是，我仍然建議您仔細考慮哪些特性對您要分類的信號類型有用。

Let’s see how this could be done in Python for a few of the above mentioned features:

讓我們看看如何在Python中實現上面提到的一些特性:

def calculate_entropy(list_values):
    counter_values = Counter(list_values).most_common()
    probabilities = [elem[1]/len(list_values) for elem in counter_values]
    entropy=scipy.stats.entropy(probabilities)
    return entropy

def calculate_statistics(list_values):
    n5 = np.nanpercentile(list_values, 5)
    n25 = np.nanpercentile(list_values, 25)
    n75 = np.nanpercentile(list_values, 75)
    n95 = np.nanpercentile(list_values, 95)
    median = np.nanpercentile(list_values, 50)
    mean = np.nanmean(list_values)
    std = np.nanstd(list_values)
    var = np.nanvar(list_values)
    rms = np.nanmean(np.sqrt(list_values**2))
    return [n5, n25, n75, n95, median, mean, std, var, rms]

def calculate_crossings(list_values):
    zero_crossing_indices = np.nonzero(np.diff(np.array(list_values) > 0))[0]
    no_zero_crossings = len(zero_crossing_indices)
    mean_crossing_indices = np.nonzero(np.diff(np.array(list_values) > np.nanmean(list_values)))[0]
    no_mean_crossings = len(mean_crossing_indices)
    return [no_zero_crossings, no_mean_crossings]

def get_features(list_values):
    entropy = calculate_entropy(list_values)
    crossings = calculate_crossings(list_values)
    statistics = calculate_statistics(list_values)
    return [entropy] + crossings + statistics

Above we can see

• a function to calculate the entropy value of an input signal,

• a function to calculate some statistics like several percentiles, mean, standard deviation, variance, etc,

• a function to calculate the zero crossings rate and the mean crossings rate,

• and a function to combine the results of these three functions.

上面代碼可以看到：

• 計算輸入信號熵值的函數，

• 用來計算一些統計數據的函數，如幾個百分位數、平均值、標準差、方差等，

• 計算過零率和平均過零率的函數，

• 並將這三個函數的結果結合起來。

The final function returns a set of 12 features for any list of values. So if one signal is decomposed into 10 different sub-bands, and we generate features for each sub-band, we will end up with 10*12 = 120 features per signal.

最後一個函數爲任何值列表返回一組12個特性。因此，如果將一個信號分解成10個不同的子帶，我們爲每個子帶生成特徵，我們將得到每個信號的10*12 = 120個特徵。

3.5.3利用特徵和scikit-learn分類器對兩個ECG數據集進行分類。

So far so good. The next step is to actually use the DWT to decompose the signals in the training set into its sub-bands, calculate the features for each sub-band, use the features to train a classifier and use the classifier to predict the signals in the test-set.

目前爲止一切順利。下一步是實際使用DWT將訓練集中的信號分解爲子帶，計算每個子帶的特徵，利用特徵訓練分類器，利用分類器預測測試集中的信號。

We will do this for two time-series datasets:

1. The UCI-HAR dataset which we have already seen in section 3.2. This dataset contains smartphone sensor data of humans while doing different types of activities, like sitting, standing, walking, stair up and stair down.

2. PhysioNet ECG Dataset (download from here) which contains a set of ECG measurements of healthy persons (indicated as Normal sinus rhythm, NSR) and persons with either an arrhythmia (ARR) or a congestive heart failure (CHF). This dataset contains 96 ARR measurements, 36 NSR measurements and 30 CHF measurements.

我們會使用2種時序數據

1. 我們已經在3.2節看到過的UCI-HAR數據集。該數據集包含了人類在進行不同類型的活動時的智能手機傳感器數據，如坐、站、走、上、下樓梯。

2. PhysioNet ECG數據集，其中包括一組健康人(指正常竇性心律，NSR)和心律失常(ARR)或充血性心力衰竭(CHF)患者的心電圖測量。該數據集包含96個ARR測量值、36個NSR測量值和30個CHF測量值。

After we have downloaded both datasets, and placed them in the right folders, the next step is to load them into memory. We have already seen how we can load the UCI-HAR dataset in section 3.2, and below we can see how to load the ECG dataset.

下載了這兩個數據集，並將它們放在正確的文件夾中之後，下一步是將它們加載到內存中。我們已經在3.2節中看到了如何加載UCI-HAR數據集，下面我們將看到如何加載ECG數據集。

def load_ecg_data(filename):
    raw_data = sio.loadmat(filename)
    list_signals = raw_data['ECGData'][0][0][0]
    list_labels = list(map(lambda x: x[0][0], raw_data['ECGData'][0][0][1]))
    return list_signals, list_labels


##########

filename = './data/ECG_data/ECGData.mat'
data_ecg, labels_ecg = load_ecg_data(filename)
training_size = int(0.6*len(labels_ecg))
train_data_ecg = data_ecg[:training_size]
test_data_ecg = data_ecg[training_size:]
train_labels_ecg = labels_ecg[:training_size]
test_labels_ecg = labels_ecg[training_size:]

The ECG dataset is saved as a MATLAB file, so we have to use scipy.io.loadmat() to open this file in Python and retrieve its contents (the ECG measurements and the labels) as two separate lists.

The UCI HAR dataset is saved in a lot of .txt files, and after reading the data we save it into an numpy ndarray of size (no of signals, length of signal, no of components) = (10299 , 128, 9)

ECG數據集保存爲MATLAB文件，因此我們必須使用scipy.io.loadmat()在Python中打開該文件，並將其內容(ECG測量值和標籤)作爲兩個單獨的列表檢索。

UCI HAR數據集保存在許多.txt文件中，讀取數據後，我們將其保存爲numpy ndarray，大小(no of signals, length of signal, no of components) = (10299, 128, 9)

Now let us have a look at how we can get features out of these two datasets.

現在讓我們看看如何從這兩個數據集中獲得特性。

def get_uci_har_features(dataset, labels, waveletname):
    uci_har_features = []
    for signal_no in range(0, len(dataset)):
        features = []
        for signal_comp in range(0,dataset.shape[2]):
            signal = dataset[signal_no, :, signal_comp]
            list_coeff = pywt.wavedec(signal, waveletname)
            for coeff in list_coeff:
                features += get_features(coeff)
        uci_har_features.append(features)
    X = np.array(uci_har_features)
    Y = np.array(labels)
    return X, Y

def get_ecg_features(ecg_data, ecg_labels, waveletname):
    list_features = []
    list_unique_labels = list(set(ecg_labels))
    list_labels = [list_unique_labels.index(elem) for elem in ecg_labels]
    for signal in ecg_data:
        list_coeff = pywt.wavedec(signal, waveletname)
        features = []
        for coeff in list_coeff:
            features += get_features(coeff)
        list_features.append(features)
    return list_features, list_labels

X_train_ecg, Y_train_ecg = get_ecg_features(train_data_ecg, train_labels_ecg, 'db4')
X_test_ecg, Y_test_ecg = get_ecg_features(test_data_ecg, test_labels_ecg, 'db4')

X_train_ucihar, Y_train_ucihar = get_uci_har_features(train_signals_ucihar, train_labels_ucihar, 'rbio3.1')
X_test_ucihar, Y_test_ucihar = get_uci_har_features(test_signals_ucihar, test_labels_ucihar, 'rbio3.1')

What we have done above is to write functions to generate features from the ECG signals and the UCI HAR signals. There is nothing special about these functions and the only reason we are using two seperate functions is because the two datasets are saved in different formats. The ECG dataset is saved in a list, and the UCI HAR dataset is saved in a 3D numpy ndarray.

我們在上面所做的就是編寫函數來從ECG信號和UCI HAR信號中生成特徵。這些函數沒有什麼特別之處，我們使用兩個單獨的函數的唯一原因是這兩個數據集以不同的格式保存。ECG數據集保存在列表中，UCI HAR數據集保存在3D numpy ndarray中。

For the ECG dataset we iterate over the list of signals, and for each signal apply the DWT which returns a list of coefficients. For each of these coefficients, i.e. for each of the frequency sub-bands, we calculate the features with the function we have defined previously. The features calculated from all of the different coefficients belonging to one signal are concatenated together, since they belong to the same signal.

對於ECG數據集，我們遍歷信號列表，並對每個信號應用DWT, DWT返回係數列表。對於每一個係數，即每一個頻率子帶，我們用之前定義的函數計算特徵。從屬於一個信號的所有不同係數計算出來的特徵被連接在一起，因爲它們屬於同一個信號。

The same is done for the UCI HAR dataset. The only difference is that we now have two for-loops since each signal consists of nine components. The features generated from each of the sub-band from each of the signal component are concatenated together.

UCI HAR數據集也是如此。唯一的區別是我們現在有兩個for循環，因爲每個信號由9個組件組成。將來自每個信號組件的每個子帶生成的特性連接在一起。

Now that we have calculated the features for the two datasets, we can use a GradientBoostingClassifier from the scikit-learn library and train it.

現在我們已經計算了這兩個數據集的特性，我們可以使用scikit-learn庫中的GradientBoostingClassifier並訓練它。

cls = GradientBoostingClassifier(n_estimators=2000)
cls.fit(X_train_ecg, Y_train_ecg)
train_score = cls.score(X_train_ecg, Y_train_ecg)
test_score = cls.score(X_test_ecg, Y_test_ecg)
print("Train Score for the ECG dataset is about: {}".format(train_score))
print("Test Score for the ECG dataset is about: {.2f}".format(test_score))

### 

cls = GradientBoostingClassifier(n_estimators=2000)
cls.fit(X_train_ucihar, Y_train_ucihar)
train_score = cls.score(X_train_ucihar, Y_train_ucihar)
test_score = cls.score(X_test_ucihar, Y_test_ucihar)
print("Train Score for the UCI-HAR dataset is about: {}".format(train_score))
print("Test Score for the UCI-HAR dataset is about: {.2f}".format(test_score))

*** Train Score for the ECG dataset is about: 1.0
*** Test Score for the ECG dataset is about: 0.93
*** Train Score for the UCI_HAR dataset is about: 1.0
*** Test Score for the UCI-HAR dataset is about: 0.95

As we can see, the results when we use the DWT + Gradient Boosting Classifier are equally amazing!

This approach has an accuracy on the UCI-HAR test set of 95% and an accuracy on the ECG test set of 93%.

正如我們所看到的，當我們使用DWT +梯度增強分類器時，結果同樣令人驚訝!

該方法對UCI-HAR檢測集的準確率爲95%，對ECG檢測集的準確率爲93%。

3.6小波變換、傅立葉變換與遞歸神經網絡分類精度的比較

So far, we have seen throughout the various blog-posts, how we can classify time-series and signals in different ways. In a previous post we have classified the UCI-HAR dataset using Signal Processing techniques like The Fourier Transform. The accuracy on the test-set was ~91%.

到目前爲止，我們已經在各種各樣的博客文章中看到，我們如何以不同的方式對時間序列和信號進行分類。之前用傅里葉變換對UCI-HAR數據集進行分類的測試準確率達到91%

In another blog-post, we have classified the same UCI-HAR dataset using Recurrent Neural Networks. The highest achieved accuracy on the test-set was ~86%.

在之前的文章種，同樣對UCI-HAR數據集用遞歸神經網絡進行分類，測試最高的測試準確率達到86%

In this blog-post we have achieved an accuracy of ~94% on the test-set with the approach of CWT + CNN and an accuracy of ~95% with the DWT + GradientBoosting approach!

這篇文章種，利用連續小波變換加捲積神經網絡達到的準確率是94%，而使用離散小波變換加梯度增強分類器的準確度達到95%

It is clear that the Wavelet Transform has far better results. We could repeat this for other datasets and I suspect there the Wavelet Transform will also outperform.

很顯然小波變換能得到更好的結果。我們可以用其他數據集進行測試，可以猜測小波變換也會表現得更好。

What we can also notice is that strangely enough Recurrent Neural Networks perform the worst. Even the approach of simply using the Fourier Transform and then the peaks in the frequency spectrum has an better accuracy than RNN’s.

我們還可以注意到，奇怪的是，遞歸神經網絡表現最差。即使是簡單地利用傅里葉變換和頻譜峯值的方法，也比RNN具有更高的精度。

This really makes me wonder, what the hell Recurrent Neural Networks are actually good for. It is said that a RNN can learn ‘temporal dependencies in sequential data’. But, if a Fourier Transform can fully describe any signal (no matter its complexity) in terms of the Fourier components, then what more can be learned with respect to ‘temporal dependencies’.

這讓我很好奇，遞歸神經網絡到底有什麼用。據說RNN可以學習“序列數據中的時間依賴性”。但是，如果傅里葉變換可以用傅里葉分量完全描述任何信號(無論其複雜度如何)，那麼關於“時間依賴性”我們還能學到什麼。

4最後總結

In this blog post we have seen how we can use the Wavelet Transform for the analysis and classification of time-series and signals (and some other stuff). Not many people know how to use the Wavelet Transform, but this is mostly because the theory is not beginner-friendly and the wavelet transform is not readily available in open source programming languages.

在這篇文章中我們看到使用小波變換對時域序列和信號進行分析和分類。並不是很多人知道如何使用小波變換，但這主要是因爲該理論對初學者不友好，而且小波變換在開源編程語言中也不容易得到。

Among Data Scientists, the Wavelet Transform remains an undiscovered jewel and I recommend fellow Data Scientists to use it more often. I am very thankful to the contributors of the PyWavelets package who have implemented a large set of Wavelet families and higher level functions for using these wavelets. Thanks to them, Wavelets can now more easily be used by people using the Python programming language.

在數據科學家中，小波變換仍然是一顆未被發現的寶石，我建議其他數據科學家更經常地使用它。我非常感謝PyWavelets包的貢獻者，他們使用這些小波實現了一組大的小波族和更高級別的函數。多虧了它們，現在使用Python編程語言的人可以更容易地使用小波。

END