Studies on an unsupervised learning model for image denoising problem are underway and Neighbor2Neighbor(NB2NB) is one of the popular methods for the task. In this paper, we analyzed the limitations of the model, and pointed out two main defects. First, some distortion of data occurs when training due to the loss of information during the sampling process. Second, the expressive power of the network is not sufficient for diverse frequency regions. In order to overcome these limitations, a method of reusing discarded pixels in the sampling process as a regularization term and extending the network structure according to the frequency domain was proposed. It was proved quantitatively and qualitatively that the model that applied these methods, F-NB2NB, has the effect of increasing the performance of NB2NB.