Informacja

Drogi użytkowniku, aplikacja do prawidłowego działania wymaga obsługi JavaScript. Proszę włącz obsługę JavaScript w Twojej przeglądarce.

Wyszukujesz frazę "recurrent neural network" wg kryterium: Temat


Tytuł:
A comparative study of CNN, LSTM, BiLSTM, AND GRU architectures for tool wear prediction in milling processes
Autorzy:
Zegarra, Fabio C.
Vargas-Machuca, Juan
Coronado, Alberto M.
Powiązania:
https://bibliotekanauki.pl/articles/28407329.pdf
Data publikacji:
2023
Wydawca:
Wrocławska Rada Federacji Stowarzyszeń Naukowo-Technicznych
Tematy:
tool wear
feature extraction
preprocessing
recurrent neural network
Opis:
Accurately predicting machine tool wear requires models capable of capturing complex, nonlinear interactions in multivariate time series inputs. Recurrent neural networks (RNNs) are well-suited to this task, owing to their memory mechanisms and capacity to construct highly complex models. In particular, LSTM, BiLSTM, and GRU architectures have shown promise in wear prediction. This study demonstrates that RNNs can automatically extract relevant information from time series data, resulting in highly precise wear models with minimal feature engineering. Notably, this approach avoids the need for excessively large window sizes of data points during model training, which would increase model complexity and processing time. Instead, this study proposes a procedure that achieves low prediction errors with window sizes as small as 100 data points. By employing Bayesian hyperparameter optimization and two preprocessing techniques (detrend and offset), RMSE errors consistently fall below 10. A key difference in this study is the use of boxplots to provide a better representation of result variability, as opposed to solely reporting the best values. The proposed approach matches more complex state of-the-art. methods and offers a powerful tool for wear prediction in engineering applications.
Źródło:
Journal of Machine Engineering; 2023, 23, 4; 122--136
1895-7595
2391-8071
Pojawia się w:
Journal of Machine Engineering
Dostawca treści:
Biblioteka Nauki
Artykuł
Tytuł:
Dynamic network functional comparison via approximate-bisimulation
Autorzy:
Donnarumma, F.
Murano, A.
Prevete, R.
Powiązania:
https://bibliotekanauki.pl/articles/206758.pdf
Data publikacji:
2015
Wydawca:
Polska Akademia Nauk. Instytut Badań Systemowych PAN
Tematy:
continuous time recurrent neural network
dynamic networks
bisimulation
network equivalence
Opis:
It is generally unknown how to formally determine whether different neural networks have a similar behaviour. This question intimately relates to the problem of finding a suitable similarity measure to identify bounds on the input-output response distances of neural networks, which has several interesting theoretical and computational implications. For example, it can allow one to speed up the learning processes by restricting the network parameter space, or to test the robustness of a network with respect to parameter variation. In this paper we develop a procedure that allows for comparing neural structures among them. In particular, we consider dynamic networks composed of neural units, characterised by non-linear differential equations, described in terms of autonomous continuous dynamic systems. The comparison is established by importing and adapting from the formal verification setting the concept of δ−approximate bisimulations techniques for non-linear systems. We have positively tested the proposed approach over continuous time recurrent neural networks (CTRNNs).
Źródło:
Control and Cybernetics; 2015, 44, 1; 99-127
0324-8569
Pojawia się w:
Control and Cybernetics
Dostawca treści:
Biblioteka Nauki
Artykuł
Tytuł:
Basic concepts of dynamic recurrent neural networks development
Autorzy:
Boyko, N.
Pobereyko, P.
Powiązania:
https://bibliotekanauki.pl/articles/410971.pdf
Data publikacji:
2016
Wydawca:
Polska Akademia Nauk. Oddział w Lublinie PAN
Tematy:
recurrent neural network
dynamic system
learning algorithms
reservoir computing
unsteady dynamics
Opis:
In this work formulated relevance, set out an analytical review of existing approaches to the research recurrent neural networks (RNN) and defined precondition appearance a new direction in the field neuroinformatics – reservoir computing. Shows generalized classification neural network (NN) and briefly described main types dynamics and modes RNN. Described topology, structure and features of the model NN with different nonlinear functions and with possible areas of progress. Characterized and systematized wellknown learning methods RNN and conducted their classification by categories. Determined the place RNN with unsteady dynamics of other classes RNN. Deals with the main parameters and terminology, which used to describe models RNN. Briefly described practical implementation recurrent neural networks in different areas natural sciences and humanities, and outlines and systematized main deficiencies and the advantages of using different RNN. The systematization of known recurrent neural networks and methods of their study is performed and on this basis the generalized classification of neural networks was proposed.
Źródło:
ECONTECHMOD : An International Quarterly Journal on Economics of Technology and Modelling Processes; 2016, 5, 2; 63-68
2084-5715
Pojawia się w:
ECONTECHMOD : An International Quarterly Journal on Economics of Technology and Modelling Processes
Dostawca treści:
Biblioteka Nauki
Artykuł
Tytuł:
Detecting objects using Rolling Convolution and Recurrent Neural Network
Autorzy:
Huang, WenQing
Huang, MingZhu
Wang, YaMing
Powiązania:
https://bibliotekanauki.pl/articles/226942.pdf
Data publikacji:
2019
Wydawca:
Polska Akademia Nauk. Czytelnia Czasopism PAN
Tematy:
multi-scale features
global context information
rolling convolution
recurrent neural network
Opis:
At present, most of the existing target detection algorithms use the method of region proposal to search for the target in the image. The most effective regional proposal method usually requires thousands of target prediction areas to achieve high recall rate.This lowers the detection efficiency. Even though recent region proposal network approach have yielded good results by using hundreds of proposals, it still faces the challenge when applied to small objects and precise locations. This is mainly because these approaches use coarse feature. Therefore, we propose a new method for extracting more efficient global features and multi-scale features to provide target detection performance. Given that feature maps under continuous convolution lose the resolution required to detect small objects when obtaining deeper semantic information; hence, we use rolling convolution (RC) to maintain the high resolution of low-level feature maps to explore objects in greater detail, even if there is no structure dedicated to combining the features of multiple convolutional layers. Furthermore, we use a recurrent neural network of multiple gated recurrent units (GRUs) at the top of the convolutional layer to highlight useful global context locations for assisting in the detection of objects. Through experiments in the benchmark data set, our proposed method achieved 78.2% mAP in PASCAL VOC 2007 and 72.3% mAP in PASCAL VOC 2012 dataset. It has been verified through many experiments that this method has reached a more advanced level of detection.
Źródło:
International Journal of Electronics and Telecommunications; 2019, 65, 2; 293-301
2300-1933
Pojawia się w:
International Journal of Electronics and Telecommunications
Dostawca treści:
Biblioteka Nauki
Artykuł
Tytuł:
Character-based recurrent neural networks for morphological relational reasoning
Autorzy:
Mogren, Olof
Johansson, Richard
Powiązania:
https://bibliotekanauki.pl/articles/103847.pdf
Data publikacji:
2019
Wydawca:
Polska Akademia Nauk. Instytut Podstaw Informatyki PAN
Tematy:
morphological analogies
morphological inflection
morphological reinflection
recurrent neural network
character-based modelling
Opis:
We present a model for predicting inflected word forms based on morphological analogies. Previous work includes rule-based algorithms that determine and copy affixes from one word to another, with limited support for varying inflectional patterns. In related tasks such as morphological reinflection, the algorithm is provided with an explicit enumeration of morphological features which may not be available in all cases. In contrast, our model is feature-free: instead of explicitly representing morphological features, the model is given a demo pair that implicitly specifies a morphological relation (such as write: writes specifying infinitive:present). Given this demo relation and a query word (e.g. watch), the model predicts the target word (e.g. watches). To address this task, we devise a character-based recurrent neural network architecture using three separate encoders and one decoder. Our experimental evaluation on five different languages shows that the exact form can be predicted with high accuracy, consistently beating the baseline methods. Particularly, for English the prediction accuracy is 94.85%. The solution is not limited to copying affixes from the demo relation, but generalizes to words with varying inflectional patterns, and can abstract away from the orthographic level to the level of morphological forms.
Źródło:
Journal of Language Modelling; 2019, 7, 1; 139-170
2299-856X
2299-8470
Pojawia się w:
Journal of Language Modelling
Dostawca treści:
Biblioteka Nauki
Artykuł
Tytuł:
An arma type pi-sigma artificial neural network for nonlinear time series forecasting
Autorzy:
Akdeniz, E.
Egrioglu, E.
Bas, E.
Yolcu, U.
Powiązania:
https://bibliotekanauki.pl/articles/91816.pdf
Data publikacji:
2018
Wydawca:
Społeczna Akademia Nauk w Łodzi. Polskie Towarzystwo Sieci Neuronowych
Tematy:
high order artificial neural networks
pi-sigma neural network, forecasting
recurrent neural network
particle swarm optimization (PSO)
Opis:
Real-life time series have complex and non-linear structures. Artificial Neural Networks have been frequently used in the literature to analyze non-linear time series. High order artificial neural networks, in view of other artificial neural network types, are more adaptable to the data because of their expandable model order. In this paper, a new recurrent architecture for Pi-Sigma artificial neural networks is proposed. A learning algorithm based on particle swarm optimization is also used as a tool for the training of the proposed neural network. The proposed new high order artificial neural network is applied to three real life time series data and also a simulation study is performed for Istanbul Stock Exchange data set.
Źródło:
Journal of Artificial Intelligence and Soft Computing Research; 2018, 8, 2; 121-132
2083-2567
2449-6499
Pojawia się w:
Journal of Artificial Intelligence and Soft Computing Research
Dostawca treści:
Biblioteka Nauki
Artykuł
Tytuł:
Monitoring regenerative heat exchanger in steam power plant by making use of the recurrent neural network
Autorzy:
Niksa-Rynkiewicz, Tacjana
Szewczuk-Krypa, Natalia
Witkowska, Anna
Cpałka, Krzysztof
Zalasiński, Marcin
Cader, Andrzej
Powiązania:
https://bibliotekanauki.pl/articles/2031128.pdf
Data publikacji:
2021
Wydawca:
Społeczna Akademia Nauk w Łodzi. Polskie Towarzystwo Sieci Neuronowych
Tematy:
recurrent neural network
intelligent industrial monitoring
Almeida–Pineda recurrent back-propagation
regenerative heat exchanger
steam power plant
Opis:
Artificial Intelligence algorithms are being increasingly used in industrial applications. Their important function is to support operation of diagnostic systems. This paper presents a new approach to the monitoring of a regenerative heat exchanger in a steam power plant, which is based on a specific use of the Recurrent Neural Network (RNN). The proposed approach was tested using real data. This approach can be easily adapted to similar monitoring applications of other industrial dynamic objects.
Źródło:
Journal of Artificial Intelligence and Soft Computing Research; 2021, 11, 2; 143-155
2083-2567
2449-6499
Pojawia się w:
Journal of Artificial Intelligence and Soft Computing Research
Dostawca treści:
Biblioteka Nauki
Artykuł
Tytuł:
Detection and Localization of Audio Event for Home Surveillance Using CRNN
Autorzy:
Suruthhi, V. S.
Smita, V.
Rolant Gini, J.
Ramachandran, K. I.
Powiązania:
https://bibliotekanauki.pl/articles/2055274.pdf
Data publikacji:
2021
Wydawca:
Polska Akademia Nauk. Czytelnia Czasopism PAN
Tematy:
convolutional recurrent neural network
CRNN
gated recurrent unit
GRU
long short-term memory
LSTM
sound event localization and detection
SELD
Opis:
Safety and security have been a prime priority in people’s lives, and having a surveillance system at home keeps people and their property more secured. In this paper, an audio surveillance system has been proposed that does both the detection and localization of the audio or sound events. The combined task of detecting and localizing the audio events is known as Sound Event Localization and Detection (SELD). The SELD in this work is executed through Convolutional Recurrent Neural Network (CRNN) architecture. CRNN is a stacked layer of convolutional neural network (CNN), recurrent neural network (RNN) and fully connected neural network (FNN). The CRNN takes multichannel audio as input, extracts features and does the detection and localization of the input audio events in parallel. The SELD results obtained by CRNN with the gated recurrent unit (GRU) and with long short-term memory (LSTM) unit are compared and discussed in this paper. The SELD results of CRNN with LSTM unit gives 75% F1 score and 82.8% frame recall for one overlapping sound. Therefore, the proposed audio surveillance system that uses LSTM unit produces better detection and overall performance for one overlapping sound.
Źródło:
International Journal of Electronics and Telecommunications; 2021, 67, 4; 735--741
2300-1933
Pojawia się w:
International Journal of Electronics and Telecommunications
Dostawca treści:
Biblioteka Nauki
Artykuł
Tytuł:
Forecasting future values of time series using the lstm network on the example of currencies and WIG20 companies
Prognozowanie przyszłych wartości szeregów czasowych z wykorzystaniem sieci lstm na przykładzie kursów walut i spółek WIG20
Autorzy:
Mróz, Bartosz
Nowicki, Filip
Powiązania:
https://bibliotekanauki.pl/articles/2016294.pdf
Data publikacji:
2020
Wydawca:
Politechnika Bydgoska im. Jana i Jędrzeja Śniadeckich. Wydawnictwo PB
Tematy:
recurrent neural network
RNN
gated recurrent unit
GRU
long short-term memory
LSTM
rekurencyjna sieć neuronowa
blok rekurencyjny
pamięć długookresowa
Opis:
The article presents a comparison of the RNN, GRU and LSTM networks in predicting future values of time series on the example of currencies and listed companies. The stages of creating an application which is a implementation of the analyzed issue were also shown – the selection of networks, technologies, selection of optimal network parameters. Additionally, two conducted experiments were discussed. The first was to predict the next values of WIG20 companies, exchange rates and cryptocurrencies. The second was based on investments in cryptocurrencies guided solely by the predictions of artificial intelligence. This was to check whether the investments guided by the predictions of such a program have a chance of effective earnings. The discussion of the results of the experiment includes an analysis of various interesting phenomena that occurred during its duration and a comprehensive presentation of the relatively high efficiency of the proposed solution, along with all kinds of graphs and comparisons with real data. The difficulties that occurred during the experiments, such as coronavirus or socio-economic events, such as riots in the USA, were also analyzed. Finally, elements were proposed that should be improved or included in future versions of the solution – taking into account world events, market anomalies and the use of supervised learning.
W artykule przedstawiono porównanie sieci RNN, GRU i LSTM w przewidywaniu przyszłych wartości szeregów czasowych na przykładzie walut i spółek giełdowych. Przedstawiono również etapy tworzenia aplikacji będącej realizacją analizowanego zagadnienia – dobór sieci, technologii, dobór optymalnych parametrów sieci. Dodatkowo omówiono dwa przeprowadzone eksperymenty. Pierwszym było przewidywanie kolejnych wartości spółek z WIG20, kursów walut i kryptowalut. Drugi opierał się na inwestycjach w kryptowaluty, kierując się wyłącznie przewidywaniami sztucznej inteligencji. Miało to na celu sprawdzenie, czy inwestowanie na podstawie przewidywania takiego programu pozwala na efektywne zarobki. Omówienie wyników eksperymentu obejmuje analizę różnych ciekawych zjawisk, które wystąpiły w czasie jego trwania oraz kompleksowe przedstawienie relatywnie wysokiej skuteczności proponowanego rozwiązania wraz z wszelkiego rodzaju wykresami i porównaniami z rzeczywistymi danymi. Analizowano również trudności, które wystąpiły podczas eksperymentów, takie jak koronawirus, wydarzenia społeczno-gospodarcze czy zamieszki w USA. Na koniec zaproponowano elementy, które należałoby ulepszyć lub uwzględnić w przyszłych wersjach rozwiązania, uwzględniając wydarzenia na świecie, anomalie rynkowe oraz wykorzystanie uczenia się nadzorowanego.
Źródło:
Zeszyty Naukowe. Telekomunikacja i Elektronika / Uniwersytet Technologiczno-Przyrodniczy w Bydgoszczy; 2020, 24; 13-30
1899-0088
Pojawia się w:
Zeszyty Naukowe. Telekomunikacja i Elektronika / Uniwersytet Technologiczno-Przyrodniczy w Bydgoszczy
Dostawca treści:
Biblioteka Nauki
Artykuł
Tytuł:
A Study of Correlation between Fishing Activity and AIS Data by Deep Learning
Autorzy:
Shen, K. Y.
Chu, Y. J.
Chang, S. J.
Chang, S. M.
Powiązania:
https://bibliotekanauki.pl/articles/1841621.pdf
Data publikacji:
2020
Wydawca:
Uniwersytet Morski w Gdyni. Wydział Nawigacyjny
Tematy:
AIS Data
deep learning framework
learning methods
Recurrent Neural Network
(RNN)
Automatic Identification System
(AIS)
fishing operation
Opis:
Previous researches on the prediction of fishing activities mainly rely on the speed over ground (SOG) as the referential attribute to determine whether the vessel is navigating or in fishing operation. Since more and more fishing vessels install Automatic Identification System (AIS) either voluntarily or under regulatory requirement, data collected from AIS in real time provide more attributes than SOG which may be utilized to improve the prediction. To be specific, the ships' trajectory patterns and the changes in course become available and should be considered. This paper aims to improve the accuracy in the identification of fishing activities. First, we do feature extraction from the AIS data of coastal waters around Taiwan and build a Recurrent Neural Network (RNN) model. Then, the activity data of fishing vessels are divided into fishing and non-fishing. Finally, based on the testing by feeding various fishing activity data, we can identify the fishing status automatically.
Źródło:
TransNav : International Journal on Marine Navigation and Safety of Sea Transportation; 2020, 14, 3; 527-531
2083-6473
2083-6481
Pojawia się w:
TransNav : International Journal on Marine Navigation and Safety of Sea Transportation
Dostawca treści:
Biblioteka Nauki
Artykuł
Tytuł:
Recurrent neural identification and control of a continuous bioprocess via first and second order learning
Autorzy:
Baruch, I.
Mariaca-Gaspar, C. R.
Powiązania:
https://bibliotekanauki.pl/articles/385133.pdf
Data publikacji:
2010
Wydawca:
Sieć Badawcza Łukasiewicz - Przemysłowy Instytut Automatyki i Pomiarów
Tematy:
backpropagation learning
direct adaptive neural control
indirect adaptive sliding mode control
Kalman filter recurrent neural network identifier
Levenberg-Marquardt learning
Opis:
This paper applies a new Kalman Filter Recurrent Neural Network (KFRNN) topology and a recursive Levenberg-Mar quardt (L-M) learning algorithm capable to estimate para meters and states of highly nonlinear unknown plant in noisy environment. The proposed KFRNN identifier, learned by the Backpropagation and L-M learning algorithm, was incorporated in a direct and indirect adaptive neural con trol schemes. The proposed control schemes were applied for real-time recurrent neural identification and control of a continuous stirred tank bioreactor model, where fast convergence, noise filtering and low mean squared error of reference tracking were achieved.
Źródło:
Journal of Automation Mobile Robotics and Intelligent Systems; 2010, 4, 4; 37-52
1897-8649
2080-2145
Pojawia się w:
Journal of Automation Mobile Robotics and Intelligent Systems
Dostawca treści:
Biblioteka Nauki
Artykuł
Tytuł:
Local stability conditions for discrete-time cascade locally recurrent neural networks
Autorzy:
Patan, K.
Powiązania:
https://bibliotekanauki.pl/articles/907772.pdf
Data publikacji:
2010
Wydawca:
Uniwersytet Zielonogórski. Oficyna Wydawnicza
Tematy:
sieć lokalnie rekurencyjna
stabilność
stabilizacja
uczenie się
optymalizacja ograniczona
locally recurrent neural network
stability
stabilization
learning
constrained optimization
Opis:
The paper deals with a specific kind of discrete-time recurrent neural network designed with dynamic neuron models. Dynamics are reproduced within each single neuron, hence the network considered is a locally recurrent globally feedforward. A crucial problem with neural networks of the dynamic type is stability as well as stabilization in learning problems. The paper formulates local stability conditions for the analysed class of neural networks using Lyapunov's first method. Moreover, a stabilization problem is defined and solved as a constrained optimization task. In order to tackle this problem, a gradient projection method is adopted. The efficiency and usefulness of the proposed approach are justified by using a number of experiments.
Źródło:
International Journal of Applied Mathematics and Computer Science; 2010, 20, 1; 23-34
1641-876X
2083-8492
Pojawia się w:
International Journal of Applied Mathematics and Computer Science
Dostawca treści:
Biblioteka Nauki
Artykuł
Tytuł:
Comparison of optimization algorithms of connectionist temporal classifier for speech recognition system
Porównanie algorytmów optymalizacji klasyfikatora czasowego do systemu rozpoznawania mowy
Autorzy:
Amirgaliyev, Yedilkhan
Darkhan, Kuanyshbay
Shoiynbek, Aisultan
Powiązania:
https://bibliotekanauki.pl/articles/408796.pdf
Data publikacji:
2019
Wydawca:
Politechnika Lubelska. Wydawnictwo Politechniki Lubelskiej
Tematy:
recurrent neural network
search method
acoustic
systems modeling language
rekurencyjna sieć neuronowa
metoda wyszukiwania
akustyka
język modelowania systemów
Opis:
This paper evaluates and compares the performances of three well-known optimization algorithms (Adagrad, Adam, Momentum) for faster training the neural network of CTC algorithm for speech recognition. For CTC algorithms recurrent neural network has been used, specifically Long- Short-Term memory. LSTM is effective and often used model. Data has been downloaded from VCTK corpus of Edinburgh University. The results of optimization algorithms have been evaluated by the Label error rate and CTC loss.
W artykule dokonano oceny i porównania wydajności trzech znanych algorytmów optymalizacyjnych (Adagrad, Adam, Momentum) w celu przyspieszenia treningu sieci neuronowej algorytmu CTC do rozpoznawania mowy. Dla algorytmów CTC wykorzystano rekurencyjną sieć neuronową, w szczególności LSTM, która jest efektywnym i często używanym modelem. Dane zostały pobrane z wydziału VCTK Uniwersytetu w Edynburgu. Wyniki algorytmów optymalizacyjnych zostały ocenione na podstawie wskaźników Label error i CTC loss.
Źródło:
Informatyka, Automatyka, Pomiary w Gospodarce i Ochronie Środowiska; 2019, 9, 3; 54-57
2083-0157
2391-6761
Pojawia się w:
Informatyka, Automatyka, Pomiary w Gospodarce i Ochronie Środowiska
Dostawca treści:
Biblioteka Nauki
Artykuł
Tytuł:
Robust zeroing neural networks with two novel power-versatile activation functions for solving dynamic Sylvester equation
Autorzy:
Zhou, Peng
Tan, Mingtao
Powiązania:
https://bibliotekanauki.pl/articles/2173674.pdf
Data publikacji:
2022
Wydawca:
Polska Akademia Nauk. Czytelnia Czasopism PAN
Tematy:
recurrent neural network
RNN
zeroing neural network
ZNN
robust zeroing neural network
RZNN
fixed-time convergence
rekurencyjna sieć neuronowa
zerowanie sieci neuronowej
konwergencja w ustalonym czasie
Opis:
In this work, two robust zeroing neural network (RZNN) models are presented for online fast solving of the dynamic Sylvester equation (DSE), by introducing two novel power-versatile activation functions (PVAF), respectively. Differing from most of the zeroing neural network (ZNN) models activated by recently reported activation functions (AF), both of the presented PVAF-based RZNN models can achieve predefined time convergence in noise and disturbance polluted environment. Compared with the exponential and finite-time convergent ZNN models, the most important improvement of the proposed RZNN models is their fixed-time convergence. Their effectiveness and stability are analyzed in theory and demonstrated through numerical and experimental examples.
Źródło:
Bulletin of the Polish Academy of Sciences. Technical Sciences; 2022, 70, 3; art. no. e141307
0239-7528
Pojawia się w:
Bulletin of the Polish Academy of Sciences. Technical Sciences
Dostawca treści:
Biblioteka Nauki
Artykuł
Tytuł:
An optimized parallel implementation of non-iteratively trained recurrent neural networks
Autorzy:
El Zini, Julia
Rizk, Yara
Awad, Mariette
Powiązania:
https://bibliotekanauki.pl/articles/2031147.pdf
Data publikacji:
2021
Wydawca:
Społeczna Akademia Nauk w Łodzi. Polskie Towarzystwo Sieci Neuronowych
Tematy:
GPU implementation
parallelization
Recurrent Neural Network
RNN
Long-short Term Memory
LSTM
Gated Recurrent Unit
GRU
Extreme Learning Machines
ELM
non-iterative training
Opis:
Recurrent neural networks (RNN) have been successfully applied to various sequential decision-making tasks, natural language processing applications, and time-series predictions. Such networks are usually trained through back-propagation through time (BPTT) which is prohibitively expensive, especially when the length of the time dependencies and the number of hidden neurons increase. To reduce the training time, extreme learning machines (ELMs) have been recently applied to RNN training, reaching a 99% speedup on some applications. Due to its non-iterative nature, ELM training, when parallelized, has the potential to reach higher speedups than BPTT. In this work, we present Opt-PR-ELM, an optimized parallel RNN training algorithm based on ELM that takes advantage of the GPU shared memory and of parallel QR factorization algorithms to efficiently reach optimal solutions. The theoretical analysis of the proposed algorithm is presented on six RNN architectures, including LSTM and GRU, and its performance is empirically tested on ten time-series prediction applications. Opt- PR-ELM is shown to reach up to 461 times speedup over its sequential counterpart and to require up to 20x less time to train than parallel BPTT. Such high speedups over new generation CPUs are extremely crucial in real-time applications and IoT environments.
Źródło:
Journal of Artificial Intelligence and Soft Computing Research; 2021, 11, 1; 33-50
2083-2567
2449-6499
Pojawia się w:
Journal of Artificial Intelligence and Soft Computing Research
Dostawca treści:
Biblioteka Nauki
Artykuł

Ta witryna wykorzystuje pliki cookies do przechowywania informacji na Twoim komputerze. Pliki cookies stosujemy w celu świadczenia usług na najwyższym poziomie, w tym w sposób dostosowany do indywidualnych potrzeb. Korzystanie z witryny bez zmiany ustawień dotyczących cookies oznacza, że będą one zamieszczane w Twoim komputerze. W każdym momencie możesz dokonać zmiany ustawień dotyczących cookies