Self-Scalable Tanh (Stan): Multi-Scale Solutions for Physics-Informed Neural Networks

Raghav Gnanasambandam, Bo Shen, Jihoon Chung, Xubo Yue, Zhenyu Kong

Research output: Contribution to journalArticlepeer-review

Abstract

Differential equations are fundamental in modeling numerous physical systems, including thermal, manufacturing, and meteorological systems. Traditionally, numerical methods often approximate the solutions of complex systems modeled by differential equations. With the advent of modern deep learning, Physics-informed Neural Networks (PINNs) are evolving as a new paradigm for solving differential equations with a pseudo-closed form solution. Unlike numerical methods, the PINNs can solve the differential equations mesh-free, integrate the experimental data, and resolve challenging inverse problems. However, one of the limitations of PINNs is the poor training caused by using the activation functions designed typically for purely data-driven problems. This work proposes a scalable -based activation function for PINNs to improve learning the solutions of differential equations. The proposed Self-scalable (Stan) function is smooth, non-saturating, and has a trainable parameter. It can allow an easy flow of gradients and enable systematic scaling of the input-output mapping during training. Various forward problems to solve differential equations and inverse problems to find the parameters of differential equations demonstrate that the Stan activation function can achieve better training and more accurate predictions than the existing activation functions for PINN in the literature.

Original languageEnglish (US)
Pages (from-to)15588-15603
Number of pages16
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Volume45
Issue number12
DOIs
StatePublished - Dec 1 2023

All Science Journal Classification (ASJC) codes

  • Software
  • Artificial Intelligence
  • Applied Mathematics
  • Computer Vision and Pattern Recognition
  • Computational Theory and Mathematics

Keywords

  • Activation function
  • differential equations
  • inverse problem
  • physics-informed neural networks

Fingerprint

Dive into the research topics of 'Self-Scalable Tanh (Stan): Multi-Scale Solutions for Physics-Informed Neural Networks'. Together they form a unique fingerprint.

Cite this