Kazuhiro Kuno

Upper bound of the expected training error of neural network regression for a Gaussian noise sequence

In neural network regression problems, often referred to as additive noise models, NIC (Network Information Criterion) has been proposed as a general model selection criterion to determine the optimal network size with high generalization performance. Although NIC has been derived using asymptotic expansion, it has been pointed out that this technique cannot be applied under the assumption that a target function is in a family of assumed networks and the family is not minimal for representing the target true function, i.e. the overrealizable case, in which NIC reduces to the well-known AIC (Akaike Information Criterion) and others depending on a loss function. Because NIC is the unbiased estimator of generalization error based on training error, it is required to derive the expectations of errors for neural networks for such cases. This paper gives upper bounds of the expectations of training errors with respect to the distribution of training data, which we call the expected training error, for some types of networks under the squared error loss. In the overrealizable case, because the errors are determined by fitting properties of networks to noise components, including in data, the target set of data is taken to be a Gaussian noise sequence. For radial basis function networks and 3-layered neural networks with bell shaped activation function in the hidden layer, the expected training error is bounded above by sigma2* - 2nsigma2*logT/T, where sigma2* is the variance of noise, n is the number of basis functions or the number of hidden units and T is the number of data. Furthermore, for 3-layered neural networks with sigmoidal activation function in the hidden layer, we obtained the upper bound of sigma2* - O(log T/T) when n > 2. If the number of data is large enough, these bounds of the expected training error are smaller than sigma2* - N(n)sigma2*/T as evaluated in NIC, where N(n) is the number of all network parameters.

Regularization learning and early stopping in linear networks

Generally, learning is performed so as to minimize the sum of squared errors between network outputs and training data. Unfortunately, this procedure does not necessarily give us a network with good generalization ability when the number of connection weights are relatively large. In such situation, overfitting to the training data occurs. To overcome this problem: there are several approaches such as regularization learning and early stopping. It has been suggested that these two methods are closely related. In this article, we firstly give an unified interpretation for the relationship between two methods through the analysis of linear networks in the context of statistical regression; i.e. linear regression model. On the other hand, several theoretical works have been done on the optimal regularization parameter and the optimal stopping time. Here, we also consider the problem from the unified viewpoint mentioned above. This analysis enables us to understand the statistical meaning of the optimality. Then, the estimates of the optimal regularization parameter and the optimal stopping time are present and those are examined by simple numerical simulations. Moreover, for the choice of regularization parameter, the relationship between the Bayesian framework and the generalization error minimization framework is discussed.

Statistical analysis of field data of railway noise and vibration collected in an urban area

Abstract The effects of various factors on railway noise and vibration were studied through multiple regression analysis of the field data at 79 sites along 8 railway lines in an urban area. The results show that peak and sound exposure levels of railway noise can be explained fairly well by distance from track, railway structure, train speed, train length and so on. Because of the close relationship between peak and sound exposure levels, the main factors such as distance from track and railway structure give similar contributions to both noise levels. Train speed gives a greater contribution to peak level than to sound exposure level, while train length gives less. Regarding the vibration level, it was found that the contribution of background vibration is greater than that of railway structure and train speed. The results were applied to predict railway noise and vibration, and were also compared with the results of the Shinkansen Super Express railway which have already been reported.

On the problem in model selection of neural network regression in overrealizable scenario

In this article, we analyze the expected training error and the expected generalization error in a special case of overrealizable scenario, in which output data is a Gaussian noise sequence. Firstly, we derived the upper bound of the expected training error of a network, which is independent of input probability distributions. Secondly, based on the first result, we derived the lower bound for the expected generalization error of a network, provided that the inputs are not stochastic. From the first result, it is clear that we should evaluate the degree of overfitting of a network to noise component in data more larger than the evaluation in NIC. From the second result, the expected generalization error, which is directly associated with the model selection criterion, is larger than in NIC. These results suggest that the model selection criterion in overrealizable scenario will be larger than NIC if inputs are not stochastic. Additionally, the results of numerical experiments agree with our theoretical results.

Comparison of noise environment of residences in Nagoya, Japan and in Beijing, China

Abstract A social survey based on the noise environment at residential sites both in Nagoya, Japan and in Beijing, China has been carried out. Noise levels have been measured during one day and night at house sites and a social survey of the reactions of residents to the living environment has also been made. The surveys in both cities indicate that the distribution of noise levels is almost the same, but the reactions of residents to the noise environment are quite different. The analysis and results of the surveys and discussion on the difference between the reactions of residents in the two cities to their noise environments are given in this paper.

Accurate frequency estimation from five samples

Sinusoidal frequency estimation is a problem common in many diverse scientific fields. There are many methods by which the instantaneous frequency of a signal may be calculated. Candidates include determining the shape of the signal’s DFT [l] , using the Hilbert transform [ 21, and using time domain zero-crossing information [ 31. The DFT method is a common way to estimate unknown frequencies. It is easy to understand because of its relation to mathematical Fourier integral theory. It states that if the analyzing signal is real valued and contains a single sinusoid, the DFT forms a single peak figure in the complex plane. From this point of view, two DFT values corresponding to the first and the second values of the peaks in the complex spectrum can be used to determine the unknown sinusoidal frequency by forming the ratio of these real (or imaginary) components. Higher frequency resolution is commonly achieved by increasing sampling density; however, this requires increased computation. In the following section, we propose another high resolution frequency estimation method which uses the results from a small sized DFT.

CHANGES OF THE SHINKANSEN SUPER EXPRESS RAILWAY NOISE SINCE THE OPENING IN 1964

Abstract Slow peak and sound exposure levels L Amax , L AE of the Shinkansen Super Express railway noise measured in 1976, 1981 and 1985 and the physical noise measure prescribed in the environmental quality standards were analyzed by a multivariate statistical method. The results of these surveys were compared. Five factors (distance, train speed, railway structure, train passby and noise barrier) can account for about 61–72% of variance of L Amax and L AE . The contributions of the various factors to the noise levels changed in these surveys. The change in equivalent sound level L eq24 since the opening of the line was also discussed. From the opening to around 1975 L eq24 increased with increasing number of trains per day. Then the level decreased gradually because of the implementation of anti-noise measures, and L eq24 in 1985 seems to be comparable with that in 1965. The present railway noise within 12·5 m from the track, however, exceeds the quality standards in Japan.