지원사업
학술연구/단체지원/교육 등 연구자 활동을 지속하도록 DBpia가 지원하고 있어요.
커뮤니티
연구자들이 자신의 연구와 전문성을 널리 알리고, 새로운 협력의 기회를 만들 수 있는 네트워킹 공간이에요.
논문 기본 정보
- 자료유형
- 학술저널
- 저자정보
- 저널정보
- 한국산업응용수학회 JOURNAL OF THE KOREAN SOCIETY FOR INDUSTRIAL AND APPLIED MATHEMATICS Journal of the Korean Society for Industrial and Applied Mathematics Vol.24 No.4
- 발행연도
- 2020.12
- 수록면
- 331 - 350 (20page)
이용수
초록· 키워드
오류제보하기
Exploding gradient is a widely known problem in training recurrent neural networks. The explosion problem has often been coped with cutting off the gradient norm by some fixed value. However, this strategy, commonly referred to norm clipping, is an ad hoc approach to attenuate the explosion. In this research, we opt to view the problem from a different perspective, the discrete-time optimal control with infinite horizon for a better understanding of the problem. Through this perspective, we fathom the region at which gradient explosion occurs. Based on the analysis, we introduce a gradient-explosion-free algorithm that keeps the training process away from the region. Numerical tests show that this algorithm is at least three times faster than the clipping strategy.
목차
ABSTRACT
1. INTRODUCTION
2. PRELIMINARIES
3. LOCATING THE GRADIENT EXPLOSION
4. GRADIENT-EXPLOSION-FREE TRAINING
5. CONCLUSION
REFERENCES
참고문헌 (18)
참고문헌 신청
Y. Goldberg / 2017 / Neural Network Methods in Natural Language Processing / Morgan & Claypool Publishers
T. Mikolov / 2011 / Extensions of recurrent neural network language model / IEEE International Conference on Acoustics, Speech and Signal Processing :5528~5531
K. Yao / 2013 / Recurrent neural networks for language understanding / Interspeech :2524~2528
I. Sutskever / 2011 / Generating text with recurrent neural networks / International Conference on Machine Learning :1017~1024
A. Graves / 2013 / Speech recognition with deep recurrent neural networks / IEEE International Conference on Acoustics, Speech and Signal Processing :6645~6649
T. Mikolov / 2011 / Extensions of recurrent neural network language model / IEEE International Conference on Acoustics, Speech and Signal Processing :5528~5531
K. Yao / 2013 / Recurrent neural networks for language understanding / Interspeech :2524~2528
I. Sutskever / 2011 / Generating text with recurrent neural networks / International Conference on Machine Learning :1017~1024
A. Graves / 2013 / Speech recognition with deep recurrent neural networks / IEEE International Conference on Acoustics, Speech and Signal Processing :6645~6649
함께 읽어보면 좋을 논문
논문 유사도에 따라 DBpia 가 추천하는 논문입니다. 함께 보면 좋을 연관 논문을 확인해보세요!
-
Accident cases of explosion in the ship : Study on the flammable chemicals and gas concentration
한국분석과학회 학술대회
2016 .05
-
Computational explosion in the frequency estimation of sinusoidal data
CSAM(Communications for Statistical Applications and Methods)
2018 .07
-
Effect of Gradient physical linearity and Static Magnetic Field homogeneity on diffusion weighted image : A phantom study
한국자기학회 학술연구발표회 논문개요집
2021 .11
-
Measurements of the temperature-gradient driven flow
한국진공학회 학술발표회초록집
2016 .08
-
Real-time Response System for the Simulation of Accidental Fires, Explosions and Toxic Releases of Hazardous Pollutants in Chemical Industries
한국대기환경학회 학술대회논문집
2021 .10
-
A new View on the Stability of Eruption Columns in Volcanic Explosions
대한지질학회 학술대회
2022 .10
-
Effect of Gradient Linearity and Static Magnetic Field Homogeneity on Diffusion Weighted Image: A Phantom Study
Journal of Magnetics
2020 .06
-
Dispersion of Benzene Pollutant from an Assumed Explosion in Urban Area
한국기상학회 학술대회 논문집
2015 .04
-
대형 폭발사고로 인한 스모크의 모니터링과 이동경로 예측: 2015년 톈진항 폭발사고를 중심으로
한국대기환경학회지(국문)
2023 .04
-
Spin-orbit torque driven multi-level switching of L10 FePt with Composition Gradient
한국자기학회 학술연구발표회 논문개요집
2022 .11
-
Gradient projection methods for the $n$-coupling problem
대한수학회지
2019 .01
-
QUADRATURE METHOD FOR EQUATIONS WITH NONLINEAR BOUNDARY CONDITIONS ARISING IN A THERMAL EXPLOSION THEORY
East Asian Mathematical Journal
2023 .05
-
LEAST-SQUARE SWITCHING PROCESS FOR ACCURATE AND EFFICIENT GRADIENT ESTIMATION ON UNSTRUCTURED GRID
JOURNAL OF THE KOREAN SOCIETY FOR INDUSTRIAL AND APPLIED MATHEMATICS
2020 .03
-
A memory efficient incremental gradient method for regularized minimization
대한수학회보
2016 .01
-
On Recent 2015 Violent Explosions and Massive Fires Occurring in Tianjin, China and the Associated Atmospheric Impact Observed in Central Korea
한국기상학회 학술대회 논문집
2015 .10
-
A NONLINEAR CONJUGATE GRADIENT METHOD AND ITS GLOBAL CONVERGENCE ANALYSIS
Journal of applied mathematics & informatics
2016 .01
-
Gradient estimates and Harnack inequalites of nonlinear heat equations for the $V$-Laplacian
대한수학회지
2018 .01
-
구면굴절력 가입에 따른 경사AC/A비와 반응AC/A비의 비교 연구
대한시과학회지
2015 .01
-
두 이종 혼합 모형에서의 수정된 경사 하강법
한국데이터정보과학회지
2017 .11
-
Gradient type estimates for linear elliptic systems from composite materials
대한수학회지
2023 .05
이 논문의 저자 정보
최근 본 자료
댓글(0)
0