TY - GEN
T1 - SemiMul
T2 - 21st IEEE International Conference on Machine Learning and Applications, ICMLA 2022
AU - Nezhadi, Ali
AU - Angizi, Shaahin
AU - Roohi, Arman
N1 - Publisher Copyright:
© 2022 IEEE.
PY - 2022
Y1 - 2022
N2 - Multiply-accumulate operation (MAC) is a fundamental component of machine learning tasks, where multiplication (either integer or float multiplication) compared to addition is costly in terms of hardware implementation or power consumption. In this paper, we approximate floating-point multiplication by converting it to integer addition while preserving the test accuracy of shallow and deep neural networks. We mathematically show and prove that our proposed method can be utilized with any floating-point format (e.g., FP8, FP16, FP32, etc.). It is also highly compatible with conventional hardware architectures and can be employed in CPU, GPU, or ASIC accelerators for neural network tasks with minimum hardware cost. Moreover, the proposed method can be utilized in embedded processors without a floating-point unit to perform neural network tasks. We evaluated our method on various datasets such as MNIST, FashionMNIST, SVHN, Cifar-10, and Cifar-100, with both FP16 and FP32 arithmetics. The proposed method preserves the test accuracy and, in some cases, overcomes the overfitting problem and improves the test accuracy.
AB - Multiply-accumulate operation (MAC) is a fundamental component of machine learning tasks, where multiplication (either integer or float multiplication) compared to addition is costly in terms of hardware implementation or power consumption. In this paper, we approximate floating-point multiplication by converting it to integer addition while preserving the test accuracy of shallow and deep neural networks. We mathematically show and prove that our proposed method can be utilized with any floating-point format (e.g., FP8, FP16, FP32, etc.). It is also highly compatible with conventional hardware architectures and can be employed in CPU, GPU, or ASIC accelerators for neural network tasks with minimum hardware cost. Moreover, the proposed method can be utilized in embedded processors without a floating-point unit to perform neural network tasks. We evaluated our method on various datasets such as MNIST, FashionMNIST, SVHN, Cifar-10, and Cifar-100, with both FP16 and FP32 arithmetics. The proposed method preserves the test accuracy and, in some cases, overcomes the overfitting problem and improves the test accuracy.
KW - Approximate Computing
KW - Machine Learning
KW - Neural Networks
UR - http://www.scopus.com/inward/record.url?scp=85152214348&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85152214348&partnerID=8YFLogxK
U2 - 10.1109/ICMLA55696.2022.00139
DO - 10.1109/ICMLA55696.2022.00139
M3 - Conference contribution
AN - SCOPUS:85152214348
T3 - Proceedings - 21st IEEE International Conference on Machine Learning and Applications, ICMLA 2022
SP - 837
EP - 842
BT - Proceedings - 21st IEEE International Conference on Machine Learning and Applications, ICMLA 2022
A2 - Wani, M. Arif
A2 - Kantardzic, Mehmed
A2 - Palade, Vasile
A2 - Neagu, Daniel
A2 - Yang, Longzhi
A2 - Chan, Kit-Yan
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 12 December 2022 through 14 December 2022
ER -