DNN-based Source Enhancement to Increase Objective Sound Quality Assessment Score

I INTRODUCTION

Sound-source enhancement has been studied for many years because of the high demand for its use for various practical applications such as automatic speech recognition , hands-free telecommunication , hearing aids , and immersive audio field representation . In this study, we aimed at generating an enhanced target source with high listening quality because the processed sounds are assumed perceived by humans.

Recently, deep learning has been successfully used for sound-source enhancement . In many of these conventional studies, deep neural networks (DNNs) were used as a regression function to estimate time-frequency (T-F) masks and/or amplitude-spectra of the target source . The parameters of the DNNs were trained using back-propagation to minimize an analytically tractable objective function such as the mean squared error (MSE) between supervised outputs and DNN outputs. In recent studies, advanced analytical objective functions were used such as the maximum-likelihood (ML) , the combination of multi-types of MSE , the Kullback-Leibler and/or Itakura-Saito divergence , the modified short-time intelligibility measure (STOI) , the clustering cost , and the discriminative cost of a clean target source and output signal using a generative adversarial network (GAN) .

When output sound is perceived by humans, the objective function that reflects human perception may not be analytically tractable, i.e., it is a black-box function. In the past few years, objective sound quality assessment (OSQA) scores, such as the perceptual evaluation of speech quality (PESQ) and STOI , have been commonly used to evaluate output sound quality. Thus, it might be better to construct DNNs to increase OSQA scores directly. However, since typical OSQA scores are not analytically defined (i.e., they are black-box functions), the gradient of the objective function cannot be calculated by simply applying back-propagation.

We previously proposed a DNN training method to estimate T-F masks and increase OSQA scores . To overcome the problem that the objective function to maximize the OSQA scores is not analytically tractable, we developed a DNN-training method on the basis of the black-box optimization framework , as used in predicting the winning percentage of the game Go . The basic idea of block-box optimization is estimating a gradient from randomly simulated output. For example, in the training of a DNN for the Go-playing computer, the computer determines a “move” (where to put a Go-stone) depending on the DNN output. Then, when the computer won the game, a gradient is calculated to increase the selection probability of the selected “moves”. We adopt this strategy to increase the OSQA scores; some output signals are randomly simulated and a DNN is trained to increase the generation probability of the simulated output signals that achieved high OSQA scores. For the first trial, we prepared a finite number of T-F mask templates and trained DNNs to select the best template that maximizes the OSQA score. Although we found that the OSQA scores increased using this method, the output performances would improve by extending the method to a more flexible T-F mask design scheme from the template-selection scheme.

In this study, to arbitrarily estimate T-F masks, we modified the DNN source enhancement architecture to estimate the latent parameters in a continuous probability density function (PDF) of the T-F mask processing output signals, as shown in Fig. 1. To calculate the gradient of the objective function, we adopt the policy gradient method as a black-box optimization scheme. With our method, the estimated latent parameters construct a continuous PDF as the “policy” of T-F-mask estimation to increase OSQA scores. On the basis of this policy, the output signals are directly simulated using the sampling algorithm. Then, the gradient of the DNN is estimated to increase/decrease the generation probability of output signals with high/low OSQA scores, respectively. The sampling from continuous PDF causes the estimate of the gradient to fluctuate, resulting in unstable training behavior. To avoid this problem, we additionally formulate two tricks: i) score normalization to reduce the variance in the estimated gradient, and ii) a sampling algorithm to simulate output signals to satisfy the constraint of T-F mask processing.

The rest of this paper is organized as follows. Section II introduces DNN source enhancement based on the ML approach. In Section III, we propose our DNN training method to increase OSQA scores on the basis of the black-box optimization. After investigating the sound quality of output signals through several experiments in Section IV, we conclude this paper in Section V.

II CONVENTIONAL METHOD

where $\omega=\{1,2,...,\Omega\}$ and $\tau=\{1,2,...,T\}$ denote the frequency and time indices, respectively.

In sound-source enhancement using T-F masks, the output signal $\hat{S}_{\omega,\tau}$ is obtained by multiplying a T-F mask by $X_{\omega,\tau}$ as

where $0\leq G_{\omega,\tau}\leq 1$ is a T-F mask. The IRM $G_{\omega,\tau}^{\mbox{\scriptsize IRM}}$ is an implementation of T-F mask, which is defined by

The IRM maximizes the signal-to-noise-ratio (SNR) when the phase spectrum of $S_{\omega,\tau}$ coincides with that of $N_{\omega,\tau}$. However, this assumption is almost never satisfied in most practical cases. To compensate for this mismatch, the phase sensitive spectrum approximation (PSA) was proposed

where $\theta_{\omega,\tau}^{(S)}$ and $\theta_{\omega,\tau}^{(X)}$ are the phase spectra of $S_{\omega,\tau}$ and $X_{\omega,\tau}$, respectively. Since the PSA $G_{\omega,\tau}^{\mbox{\scriptsize PSA}}$ is a T-F mask that minimizes the squared error between $S_{\omega,\tau}$ and $\hat{S}_{\omega,\tau}$ on the complex plane, we use this as a T-F masking scheme.

II-B Maximum-likelihood-based DNN training for T-F mask estimation

In many conventional studies of DNN-based source enhancement, DNNs were used as a mapping function to estimate T-F masks. In this section, we explain DNN training based on ML estimation, on which the proposed method is based. Since the ML-based approach explicitly models the PDF of the target source, it becomes possible to simulate output signals by generating random numbers from the PDF.

In ML-based training, the DNNs are constructed to estimate the parameters of the conditional PDF of the target source providing the observation is given by $p(\bm{S}_{\tau}|\bm{X}_{\tau},\Theta)$. Here, $\Theta$ denotes the DNN parameters. Its example on a fully connected DNN is described later (after (16)). The target and observation source are assumed to be vectorized for all frequency bins as

where $\top$ is transposition. Then $\Theta$ is trained to maximize the expectation of the log-likelihood as

where the objective function $\mathcal{J}^{\mbox{\scriptsize ML}}(\Theta)$ is defined by

The back-propagation algorithm is used in training $\Theta$ to maximize (9). When $p(\bm{S}_{\tau}|\bm{X}_{\tau},\Theta)$ is composed of differentiable functions with respect to $\Theta$, the gradient is calculated as

where $\nabla_{x}$ is a partial differential operator with respect to $x$.

To calculate (10), $p(\bm{S}_{\tau}|\bm{X}_{\tau},\Theta)$ is modeled by assuming that the estimation error of $S_{\omega,\tau}$ is independent for all frequency bins and follows the zero-mean complex Gaussian distribution with the variance $\sigma_{\omega,\tau}^{2}$. The assumption is based on state-of-the-art methods, which train DNNs to minimize the MSE between $S_{\omega,\tau}$ and $\hat{G}_{\omega,\tau}X_{\omega,\tau}$ on the complex plane . The minimum-MSE (MMSE) on the complex plane is equivalent to assuming that the errors are independent for all frequency bins and follow the zero-mean complex Gaussian distribution with variance 1. Our assumption relaxes the assumption of the conventional methods; the variances of each frequency bin vary according to the error values to maximize the likelihood. Thus, since $\hat{S}_{\omega,\tau}$ is given by $\hat{G}_{\omega,\tau}X_{\omega,\tau}$, $p(\bm{S}_{\tau}|\bm{X}_{\tau},\Theta)$ is modeled by the following complex Gaussian distribution as

In this model, it can be regarded that the MSE between $S_{\omega,\tau}$ and $\hat{S}_{\omega,\tau}$ on the complex plane is extended to the likelihood of $S_{\omega,\tau}$ defined on the complex Gaussian distribution, the mean and variance parameters of which are $\hat{S}_{\omega,\tau}$ and $\sigma_{\omega,\tau}^{2}$, respectively. (11) includes unknown parameters: the T-F mask $\hat{G}_{\omega,\tau}$ and error variance $\sigma_{\omega,\tau}^{2}$. Thus, we construct DNNs to estimate $\hat{G}_{\omega,\tau}$ and $\sigma_{\omega,\tau}^{2}$ from $\bm{X}_{\tau}$, as shown in Fig. 2. The vectorized T-F masks and error variances for all frequency bins are defined as

Here $\bm{x}_{\tau}$ is the input vector of DNNs that is prepared by concatenating several frames of observations to account for previous and future $Q$ frames as $\bm{x}_{\tau}=(\bm{X}_{\tau-Q},...,\bm{X}_{\tau},...,\bm{X}_{\tau+Q})^{\top}$, and $\bm{G}(\bm{x}_{\tau})$ and $\bm{\sigma}(\bm{x}_{\tau})$ are estimated by

III PROPOSED METHOD

Our proposed DNN-training method increases OSQA scores. With the proposed method, the policy gradient method is used to statistically calculate the gradient with respect to $\Theta$ by using a sampling algorithm, even though the objective function is not differentiable. However, sampling-based gradient estimation would frequently make the DNN training behavior become unstable. To avoid this problem, we introduce two tricks: i) score normalization that reduces the variance in the estimated gradient (in Sec. III-B), and ii) a sampling algorithm to simulate output signals to satisfy the constraint of T-F mask processing (in Sec. III-C). Finally, the overall training procedure of the proposed method is summarized in Sec. III-D.

Let $\mathcal{B}(\hat{\bm{S}},\bm{X})$ be a scoring function that quantifies the sound quality of the estimated sound signal $\hat{\bm{S}}:=(\hat{S}_{1},...,\hat{S}_{\Omega})^{\top}$ defined by (2). To implement $\mathcal{B}(\hat{\bm{S}},\bm{X})$, subjective evaluation is simple. However, it would be difficult to use in practical implementation because DNN training requires a massive amount of listening-test results. Thus, $\mathcal{B}(\hat{\bm{S}},\bm{X})$ quantifies the sound quality based on OSQA scores, as shown in Fig. 1, and the details of its implementation are discussed in Sec. III-B. We assume $\mathcal{B}(\hat{\bm{S}},\bm{X})$ is non-differentiable with respect to $\Theta$, because most OSQA scores are black-box functions.

Let us consider the expectation maximization of $\mathcal{B}(\hat{\bm{S}},\bm{X})$ as a metric of performance of the sound-source enhancement that increases OSQA scores as

Since the output signal $\hat{\bm{S}}$ is calculated from the observation $\bm{X}$, we decompose the joint PDF $p(\hat{\bm{S}},\bm{X})$ into the conditional PDF of the output signal given the observation $p(\hat{\bm{S}}|\bm{X})$ and the marginal PDF of the observation $p(\bm{X})$ as $p(\hat{\bm{S}},\bm{X})=p(\hat{\bm{S}}|\bm{X})p(\bm{X})$. Then, (17) can be reformed as

Since $\mathcal{B}(\hat{\bm{S}},\bm{X})$ is non-differentiable with respect to $\Theta$, the gradient of (20) cannot be analytically obtained by simply applying back-propagation. Hence, we apply the policy-gradient method , which can statistically calculate the gradient of a black-box objective function. By assuming that the function form of $\mathcal{B}(\hat{\bm{S}},\bm{X})$ is smooth, $\mathcal{B}(\hat{\bm{S}},\bm{X})$ is a continuous function and its derivative exists. In addition, we assume $p(\hat{\bm{S}}|\bm{X},\Theta)$ is composed with differentiable functions with respect to $\Theta$. Then, the gradient of (20) can be calculated using a log-derivative trick $\nabla_{x}p(\bm{x})=p(\bm{x})\nabla_{x}\ln p(\bm{x})$ as

Since the expectation in (22) cannot be analytically calculated, the expectation with respect to $\bm{X}$ is approximated by averaging the training data, and the average of $\hat{\bm{S}}$ is calculated using the sampling algorithm as

where $T^{(i)}$ is the frame length of the $i$-th utterance, and we assume that the output signal of each time frame is calculated independently. The details of the deviation of (25) are described in the Appendix -A.

III-B Scoring-function design for stable training

III-C Sampling-algorithm to simulate T-F-mask-processed output signal

III-D Training procedure

We describe the overall training procedure of the proposed method, as shown in Fig. 4. Hereafter, to simplify the sampling algorithm, we use the complex Gaussian distribution as $p(\hat{\bm{S}}|\bm{X},\Theta)$ described in (11)–(16).

where $\bm{I}$ is the $2\times 2$ identity matrix, and $\Re$ and $\Im$ denote the real and imaginary parts of the complex number, respectively. After that, T-F mask $\hat{G}_{\omega,\tau}^{(i,k)}$ is calculated using (29). To accelerate the algorithm convergence, we additionally use the $\epsilon$-greedy algorithm to calculate $\hat{G}_{\omega,\tau}^{(i,k)}$. With probability $1-\epsilon$ applied to each time-frequency bin, the maximum a posteriori (MAP) T-F mask $\hat{G}_{\omega,\tau}^{(i)}$ estimated using DNNs is used instead of $\hat{G}_{\omega,\tau}^{(i,k)}$ as

In addition, a large gradient value $\nabla_{\Theta}\mathcal{J}(\Theta)$ leads to unstable training. One reason for the large gradient is that the log-likelihood $\nabla_{\Theta}\ln p(\hat{\bm{S}}_{\tau}^{(i,k)}|\bm{X}_{\tau}^{(i)},\Theta)$ in (26) becomes large. To reduce the gradient of the log-likelihood, the difference between the mean T-F mask $\hat{G}_{\omega,\tau}^{(i)}$ and simulated T-F mask $\hat{G}_{\omega,\tau}^{(i,k)}$ is truncated to confine it within the range of $[-\lambda,\lambda]$ as

IV EXPERIMENTS

We conducted objective experiments to evaluate the performance of the proposed method. The experimental conditions are described in Sec. IV-A. To investigate whether a DNN source-enhancement function can be trained to increase OSQA scores, we first investigated the relationship between the number of updates and OSQA scores (Sec. IV-B). Second, the source enhancement performance of the proposed method was compared with those of conventional methods by using several objective measurements (Sec. IV-C). Finally, subjective evaluations for sound quality and ineligibility were conducted (Sec. IV-D). For comparison methods, we used four DNN source-enhancement methods; two T-F-mask mapping functions trained using an MMSE-based objective function and the ML-based objective function described in Sec. II-B, and two T-F-mask selection functions trained for increasing the PESQ and STOI .

The ATR Japanese speech database was used as the training dataset of the target source. The dataset consists of 6640 utterances spoken by 11 males and 11 females. The utterances were randomly separated into 5976 for the development set and 664 for the validation set. As the training dataset of noise, a noise dataset of CHiME-3 was used that consisted of four types of background noise files including noise in cafes, street junctions, public transport, and pedestrian areas . The noisy-mixture dataset was generated by mixing clean speech utterances with various noisy and SNR conditions using the following procedure; i) the noise is randomly selected from noise dataset, ii) the amplitude of noise is adjusted to be the desired SNR-level, and iii) the speech and noise source is added in the time-domain. As the test dataset, a Japanese speech database consisting of 300 utterances spoken by 3 males and 3 females was used for target-source dataset, and an ambient noise database recorded at airports (Airp.), amusement parks (Amuse.), offices (Office), and party rooms (Party) was used as the noisy dataset. All samples were recorded at the sampling rate of 16 kHz. The SNR levels of the training/test dataset were -6, 0, 6, and 12 dB.

IV-A2 DNN architecture and setup

For the proposed and all conventional methods, a fully connected DNN was used that has 3 hidden layers and 1024 hidden units. All input vectors were mean-and-variance normalized using the training data statistics. The activation functions for the T-F mask $\phi_{g}$, variance $\phi_{\sigma}$, and hidden units $\phi_{h}$ were the sigmoid function, exponential function, and rectified linear unit (ReLU), respectively. The context window size was $Q=5$, and the variance regularization parameter in (15) was $C_{\sigma}=10^{-4}$ In preliminary experiments using candidate values $C_{\sigma}\in\{10^{-2},10^{-3},10^{-4}\}$, there were no distinct differences in training stability and results. Thus, to eliminate the effect of regularization, we used the minimum parameter of the candidate values. . The Adam method was used as a gradient method. To avoid over-fitting, input vectors and DNN outputs, i.e., the T-F masks and error variances, were compressed using a $\mathsf{B}=64$ Mel-transformation matrix, and the estimated T-F masks and error variances were transformed into a linear frequency domain using the Mel-transform’s pseudo-inverse .

A PSA objective function was used as the MMSE-based objective function. Since the PSA objective function does not use the variance parameter $\bm{\sigma}(\bm{x}_{\tau})$, DNNs estimate only T-F masks $\bm{G}(\bm{x}_{\tau})$. For the ML-based objective function, we used (9) with the complex Gaussian distribution described in Sec. II-B. To train both methods, the dropout algorithm was used and initialized by layer-by-layer pre-training . An early-stopping algorithm was used for fine-tuning with the initial step-size $10^{-4}$ and the step-size threshold $10^{-7}$, and L2 normalization with the parameter $10^{-4}$ was used as a regularization algorithm.

For the T-F-mask selection-based method , to improve the flexibility of T-F-mask selection, we used 128 T-F-mask templates. The DNN architecture, except for the output layer, is the same as MMSE- and ML-based methods.

For the proposed method, DNN parameters were initialized by ML-based training, and their step-size was $10^{-6}$. To calculate $\nabla_{\Theta}\mathcal{J}(\Theta)$, the iteration parameters $\mathcal{I}=10$ and $K=20$ were used. The $\epsilon$-greedy parameter $\epsilon$ was 0.05, and the clipping parameter $\lambda$ was determined as $0.05$ according to preliminary informal experiments We tested some possible combinations of these parameters by grid-search. Then, we found that the listed parameters achieved a stable training and realistic computational time (2 days using an Intel Xeon Processor E5-2630 v3 CPU and a Tesla M-40 GPU). . As the OSQA scores, we used the PSEQ, which is a speech quality measure, and the STOI, which is a speech intelligibility measure. To avoid adjusting the step-size of the gradient method for each OSQA, we normalized OSQA scores to uniform the range of the each OSQA score. In this experiments, each OSQA score was normalized so that its maximum and minimum values were 100 and 0 as

The training algorithm was stopped after 10,000 times of executing the whole parameter update process shown in Fig. 4.

IV-A3 Other conditions

It is known that T-F-mask processing causes artificial distortion, so-called musical noise . For all methods, to reduce musical noise, flooring and smoothing were applied to $\hat{G}_{\omega,\tau}$ before T-F-mask processing as

where we used the lower threshold of the T-F mask $G^{\mbox{\scriptsize min}}=0.158$ and smoothing parameter $\beta=0.3$. The frame size of the short-time Fourier transform (STFT) was 512, and the frame was shifted by 256 samples. All the above-mentioned conditions are summarized in Table I.

IV-B Investigation of relationship between number of updates and OSQA score

To investigate whether the DNN source-enhancement function can be trained to increase OSQA scores, we first investigated the relationship between the number of updates and improvement of the OSQA scores. We define “OSQA score improvement” as the difference in the score value from the baseline OSQA score. For the baseline, we use the OSQA score obtained from the observed signal. Since the DNN parameters of the proposed method were initialized by ML-based training, each OSQA score was compared with the OSQA score that had zero updates. Thus, if DNN parameters were successfully trained with the proposed method, the OSQA score improvement would increase in accordance with the number of updates.

Figure 6 shows the OSQA score improvements evaluated on the test dataset. Both OSQA score improvements increased as the number of updates increased for all SNR conditions. These results suggest that the proposed method is effective at increasing arbitrary OSQA scores, such as the PESQ and STOI.

We also investigated the relationship between the number of updates and MSE using the test dataset. Figure 6 shows MSE depending on the number of updates. Under most SNR conditions, MSE did not decrease despite OSQA scores increasing. Table II shows the correlation coefficients between OSQA score improvements and MSE values. There was little correlation between PESQ improvement and MSE, and the correlation between STOI improvement and MSE depended on the input SNR condition. Thus, these results suggest that minimization of MSE does not necessarily maximize OSQA scores.

IV-C Objective evaluation

The source-enhancement performance of the proposed method was compared with those of conventional methods using three objective measurements: the signal-to-distortion ratio (SDR), PESQ, and STOI. The SDR was defined as

and calculated using the “BSS-Eval toolbox .” These measurements were evaluated on the observed signal (OBS), the MMSE- and ML-based DNN training (MMSE and ML), a T-F-mask selection method to increase the PESQ and STOI (C-PESQ and C-STOI), and the proposed method to increase the PESQ and STOI (P-PESQ and P-STOI). To investigate whether the proposed method enables training of a DNN to increase a metric that consists of multiple OSQA scores, we also trained a DNN to increase a mixed-OSQA score (P-MIX). As the first trial, we mixed the PESQ and the STOI. The mixed-OSQA is defined as

Table III lists the evaluation results of each objective measurement on four noise types and four input SNR conditions. The asterisk indicates that the score was significantly higher than both MMSE and ML in a paired one-sided t-test ($\alpha=0.05$). The SDRs tended to be higher when using the conventional MMSE/ML-based objective function than the proposed method under low SNR conditions. The PESQ and STOI of P-PESQ and P-STOI were higher than those of MMSE and ML, respectively. For each method, the PESQ and STOI improved by around 0.1 and 2–5 %, respectively, and significant differences were observed for all noise and SNR conditions. These results suggest that the proposed method was able to train the DNN source-enhancement function to directly increase black-box OSQA scores.

In mixed-OSQA experiments, both PESQ and STOI of P-MIX were higher than those of MMSE and ML under almost all noise and SNR conditions. In the comparison to the results of the mixed-OSQA and single-OSQA (i.e. P-PESQ and P-STOI), P-MIX achieved almost the same or slightly lower PESQ and STOI scores than P-PESQ and P-STOI, respectively. In addition, P-MIX outperformed STOI and PESQ scores than P-PESQ and P-STOI, respectively. These results suggest that the use of the mixed-OSQA would be an effective way to increase multiple-perceptual qualities.

In Table III we also show that the proposed method outperformed the T-F mask selection-based methods in terms of the target OSQA under almost all noise types and SNR conditions. Such favorable experimental results would have been observed because of the flexibility of the T-F mask estimation achieved by the proposed method. In this experiment, the number of the T-F mask template ($=128$) was larger than that used in the previous work ($=32$) . However, since the T-F masks were generated by a combination of the finite number of templates, the patterns of the T-F mask were still limited. These results suggested that by adopting the policy-gradient method to optimize the parameters of a continuous PDF of the T-F mask processing, the flexibility of the T-F mask estimation was improved.

Figure 7 shows examples of the estimated T-F masks and output signal, and Table IV lists its objective scores. The SNR of the observed signal was adjusted to 0 dB using amusement parks noise. Figure 7 shows that the estimated T-F masks reflect the characteristics of each objective function. In comparison to the results of MMSE and ML that reduced the distortion of the target source on average, the T-F mask estimated by P-PESQ strongly reduced the residual noise, even when it distorted the target sound at a middle/high frequency (e.g. Fig. 7 white dotted box), and achieved the best PESQ. In contrast, the T-F mask estimated by P-STOI weakly reduced noise to avoid distorting the target source, even when the noise remained in the non-speech frames (e.g. Fig. 7 white dotted circle), and achieved the best STOI. This may be because the residual noise degrades the sound quality and the distortion of the target source degrades speech intelligibility. The T-F mask estimated by P-MIX involved both characteristics and relaxed the disadvantage of P-PESQ and P-STOI, and both OSQA scores were higher than those of ML and MMSE. Namely, speech distortion at a middle/high frequency was reduced (e.g. Fig. 7 white dotted box) and residual noise in the non-speech frames were reduced (e.g. Fig. 7 white dotted circle).

IV-D Subjective evaluation

To investigate the sound quality of the output signals, subjective speech-quality tests were conducted according to ITU-T P.835 . In the tests, the participants rated three different factors in the samples:

Speech mean-opinion-score (S-MOS): the speech sample was rated 5–not distorted, 4–slightly distorted, 3–somewhat distorted, 2–fairly distorted, or 1–very distorted.

Subjective noise MOS (N-MOS): the background of the sample was 5–not noticeable, 4–slightly noticeable, 3–noticeable but not intrusive, 2–somewhat intrusive, or 1–very intrusive.

Overall MOS (G-MOS): the sound quality of the sample was 5–excellent, 4–good, 3–fair, 2–poor, or 1–bad.

Sixteen participants evaluated the sound quality of the output signals of ML, P-PESQ, and P-STOI. The participants evaluated 20 files for each method; the 20 files consisted of five randomly selected files from the test dataset for each of the four types of noise. The input SNR was 6 dB.

Figure 8 shows the results of the subjective tests. For all factors, P-PESQ achieved a higher score than ML, and statistically significant differences from ML were observed in a paired one-sided $t$-test ($p$-value $=0.05$). The reason for this result suggested that participants may have perceived the degrade of the speech quality from both the speech distortion and the residual noise in speech frame in the output signal of ML. In addition, although there was no statistically significant difference between P-PESQ and P-STOI in terms of S-MOS score, N-MOS score of P-STOI was significantly lower than that of P-PESQ. Thus, G-MOS score of P-STOI was also lower than that of P-PESQ. It would be because P-STOI weakly reduced noise to avoid distorting the target source, even when the noise remained in the non-speech frames as shown in Sec. IV.C.

IV-D2 Speech intelligibility test

We conducted a word-intelligibility test to investigate speech intelligibility. We selected 50 low familiarity words from familiarity-controlled word lists 2003 (FW03) as the test dataset of speech. The selected dataset consisted of Japanese four-mora words whose accent type was Low-High-High-High. The noisy test dataset was created by adding a randomly selected noise at SNR of 6 dB from the noisy dataset, which was used in the objective evaluation. Sixteen participants attempted to write a phonetic transcription for output signals of ML, P-PESQ, and P-STOI. The percentage of correct answers was used as the intelligibility score.

Figure 9 shows the intelligibility score of each method. P-STOI achieved the highest score. In addition, statistically significant differences from ML were observed in an unpaired one-sided $t$-test ($p$-value $=0.05$). From both sound-quality and speech-intelligibility tests, we found that the proposed method could improve the specific hearing quality corresponding to the OSQA score used as the objective function.

V CONCLUSIONS

We proposed a training method for the DNN-based source-enhancement function to increase OSQA scores such as the PESQ. The difficulty is that the gradient of OSQA scores may not be analytically calculated by simply applying the back-propagation algorithm because most OSQA scores are black boxes. To calculate the gradient of the OSQA-based objective function, we formulated a DNN-optimization scheme on the basis of the policy-gradient method. In the experiment, 1) it was revealed that the DNN-based source-enhancement function can be trained using the gradient of the OSQA obtained with the policy-gradient method. In addition, 2) the OSQA score and specific hearing quality corresponding to the OSQA score used as the objective function improved. Therefore, it can be concluded that this method made it possible to use not only analytical objective functions but also black-box functions for the training of the DNN-based source-enhancement function.

Although we focused on maximization of OSQA in this study, the proposed method potentially increases other black-box measurements. In the future, we will aim to adopt the proposed method to increase other black-box objective measures such as the subjective score obtained from a “human-in-the-loop” audio-system and word accuracy of a black-box automatic-speech-recognition system . We found that both the PESQ and STOI could increase simultaneously by mixing multiple OSQA scores as an objective function. In the future, we will also investigate the optimality of the OSQA score and its mixing ratio for the proposed method.

References

-A Deviation of (25)

Then, the gradient of (40) can be calculated using a log-derivative trick as

To normalize the difference in frame length $T^{(i)}$, we multiplied $1/T^{(i)}$ by the original gradient. The log-likelihood function $\ln p(\hat{\bm{S}}^{(i,k)}_{\tau}|\bm{X}^{(i)}_{\tau},\Theta)$ can be expanded as

where $\hat{G}_{\omega,\tau}^{(i)}$ and $(\sigma_{\omega,\tau}^{2})^{(i)}$ can be estimated by forward-propagation of the DNN as (12)–(16), and $\hat{G}_{\omega,\tau}^{(i,k)}$ is given by the sampling algorithm of the proposed method. By using above procedure, $\nabla_{\Theta}\mathcal{J}(\Theta)$ can be calculated by simply applying back-propagation with respect to $\hat{G}_{\omega,\tau}^{(i)}$ and $(\sigma_{\omega,\tau}^{2})^{(i)}$. Please note that since the simulated output signal $\hat{\bm{S}}^{(i,k)}_{\tau}$ deals with the “label data”, the back-propagation algorithm is not applied for $\hat{G}_{\omega,\tau}^{(i,k)}$.