key: cord-0115865-7fdabsn1
authors: Ronaghi, Farnoush; Salimibeni, Mohammad; Naderkhani, Farnoosh; Mohammadi, Arash
title: COVID19-HPSMP: COVID-19 Adopted Hybrid and Parallel Deep Information Fusion Framework for Stock Price Movement Prediction
date: 2021-01-02
journal: nan
DOI: nan
sha: dc866d17fbd3f03b20a9686f543a465a638302d5
doc_id: 115865
cord_uid: 7fdabsn1

The novel of coronavirus (COVID-19) has suddenly and abruptly changed the world as we knew at the start of the 3rd decade of the 21st century. Particularly, COVID-19 pandemic has negatively affected financial econometrics and stock markets across the globe. Artificial Intelligence (AI) and Machine Learning (ML)-based prediction models, especially Deep Neural Network (DNN) architectures, have the potential to act as a key enabling factor to reduce the adverse effects of the COVID-19 pandemic and future possible ones on financial markets. In this regard, first, a unique COVID-19 related PRIce MOvement prediction (COVID19 PRIMO) dataset is introduced in this paper, which incorporates effects of social media trends related to COVID-19 on stock market price movements. Afterwards, a novel hybrid and parallel DNN-based framework is proposed that integrates different and diversified learning architectures. Referred to as the COVID-19 adopted Hybrid and Parallel deep fusion framework for Stock price Movement Prediction (COVID19-HPSMP), innovative fusion strategies are used to combine scattered social media news related to COVID-19 with historical mark data. The proposed COVID19-HPSMP consists of two parallel paths (hence hybrid), one based on Convolutional Neural Network (CNN) with Local/Global Attention modules, and one integrated CNN and Bi-directional Long Short term Memory (BLSTM) path. The two parallel paths are followed by a multilayer fusion layer acting as a fusion centre that combines localized features. Performance evaluations are performed based on the introduced COVID19 PRIMO dataset illustrating superior performance of the proposed framework.

The novel of coronavirus has suddenly and abruptly changed the world as we knew at the end of the 2 nd decade of the 21 st century. The global COVID-19 pandemic caused market volatility (Mazue, 2020; Baek, 2020) rocketing upward around the world. In particular, the pandemic has negatively triggered several sectors including but not limited to stock markets, global supply chains, labor markets, and consumption behaviors. Disruptions of such sectors, especially the stock markets (Bustos, 2020; Al-Awadhi, 2020; Ansari, 2020) , can adversely affect the global economy. The United States volatility levels in the mid-March of 2020 are similar to those last seen during October 1987; after 1929 to 1939 during in 2008 . In September 2008 Jones Industrial Average fell 777.68 points in intraday trading. During the recent pandemic, in the latter part of March 2020, volatility began to retreat and, by late April, fell sharply but remained well above pre-pandemic levels. It is expected that the emerging markets, the ones with restricted resources to cope with negative impacts of the COVID-19, more substantially feel the COVID-19 pressure due to having slower economic growth and not having sufficient capital inflows.

In these sad and unfortunate pandemic times, Artificial Intelligence (AI) and Machine Learning (ML)-based (Radojicic, 2020; Rezaei, 2020; Hoseinzade, 2019; Chong, 2019; Seong, 2021; Zhang, 2021 ) stock market movement prediction solutions can potentially prevent the pandemic crisis that negatively affecting the market across the world causing unexpected havocs.

Literature Review: Stock market movement prediction is a key and challenging problem in financial econometrics as such has attracted extensive recent research focus (Frankel, 1995; Ronaghi, 2020; Mohammadi, 2017; Edwards, 2007; Bollen, 2011; Jiang, 2020; Hu, 2019; Koshiyama, 2020; Schumaker, 2009) . It is widely acknowledged that investors need high-quality data to make informed and accurate decisions. Particularly, in times of market crisis, specifically during the recent COVID-19 pandemic, investors need advanced Big-Data Analytic and Information Technologies (Choudrie, 2021) to acquire timely and accurate data. Using high-quality data, investors can perform fast analysis and decision making in the market volatility and react quickly to the fast changing conditions. Any positive or negative news related to the stock market crisis can have a ripple effect on the investors' decision-making process within the stock markets. During the pandemic area, typically, stock movement prediction becomes significantly challenging as stock markets tend to face high fluctuations. Consequently, it is of paramount importance to develop in-novative and advanced processing and learning solutions to accurately predict stock movements for achieving maximum potential profit. This has resulted in a recent surge of interest in ML/AI-based prediction techniques (Hu, 2019; Anik, 2020) and fusion of multi-modal information sources. In the context of stock price movement prediction, historical stock prices are typically fused with information obtained from media news. For the latter, in addition to the conventional news platforms, recently, extensive interest is shown towards utilization of Internet-based news resources, such as social media for development of ML/AI predictive models. The manuscript focuses on this topic and examines the role of COVID-19 related social media news on behavior of Dow Jones market.

Recent advancements and developments in the field of ML and AI, in particular, Deep Neural Networks (DNNs), have motivated different research works to incorporate such advanced modeling techniques for prediction and forecasting tasks in stock markets (Tetlock, 2007) . In particular, there has been a recent surge of interest in information fusion (Narkhede, 2021) and sentiment analysis (Choudrie, 2021) based on stock market data (Patel, 2015; Gite, 2021) and their effects on stock markets. On the one hand, Reference (Patel, 2015) focused on predicting direction of movement of stock and stock price index. This study compared different ML models including Artificial Neural Network (ANN), Support Vector Machine (SVM), Random Forests (RF), and naive-Bayes.

Two different approaches are developed to feed ML models. The first approach involves computation of ten technical parameters using stock trading data, while the second approach focuses on representing these technical parameters as trend deterministic data. Experiments are performed on the Indian stock markets and based on a dataset consisting of 10 years of historical data from 2003 to 2012 of two stocks (Reliance Industries and Infosys Ltd.) and two stock price indices (CNX Nifty and S&P Bombay Stock Exchange (BSE) Sensex). Experimental results show that prediction performance improves when technical parameters are represented as trend deterministic data.

Similarly, Reference (Patel, 2015) focused on predicting future values of stock market index via a two-stage fusion approach. Support Vector Regression (SVR) is used in the first stage, while ANN, RF, or SVR are used in the second stage resulting in SV R-AN N , SV R-RF and SV R-SV R fusion prediction models. The prediction performance of these hybrid models is compared with the stand-alone models illustrating the power of hybrid modeling. On the other hand, Reference (Gite, 2021) focused on predicting stock prices from sentiment analysis with use of ML/DL approaches.

News headlines have a tremendous effect on the buying and selling patterns as traders easily get influenced by what they read. In this work, News headlines are extracted from Pulse and historical prices are obtained from Yahoo Finance. The proposed DL models, i.e., LSTM and LSTM-CNN model, performed reasonably well illustrating potentials of DL for sentiment analysis.

DNN-based solutions are data-driven techniques that learn the underlying dynamics of the stock price movements through processing of a large amount of data. DNN-based methodologies are, typically, data hungry and will not perform well in the absence of a large and diversified set of data resources. Availability of public news media, Internet-based news channels, and social media can pave the way to better train DNN models and further increase utilization of AI within stock markets. This research field, however, is still in its infancy due to its high dependence on the reliability and quality of the information available through Internet-based news channels and social media resources (Hu, 2019) . Furthermore, such data sources can not be directly used for prediction tasks (Luss, 2015) due to the highly correlated nature of stock market price movements. To tackle the aforementioned issues, there is an unmet and timely quest to develop and design: (i) Hybrid processing/learning models based on different and diversified learning architectures to capture underlying correlations and variabilities of the data sources, and; (ii) Smart fusion strategies to combine scattered social media news with historical mark data. The main objective of the proposed DNN-based predictive model is to construct a new information fusion framework to analyze and interpret ever-changing trends during the COVID-19 pandemic area.

Contributions: The paper makes the following contributions. First, a unique and real COVID-19 related PRIce MOvement prediction (COVID19 PRIMO) dataset (PRIMO, 2021) 1 " is constructed to incorporate effects of internet-based and social media trends related to COVID-19 on stock market price movements. The main component of the constructed COVID19 PRIMO dataset is based on Twitter messages. It is well known that news and media move stock prices (Fama, 1998; Huang, 2020; Yun, 2019) . Nowadays, information reaches out to the public via different news platforms ranging from newspaper, radio and television to social media and Internet-based venues. In this area, social media, especially Twitter, is a popular and widely used platform to share personalized opinion on different topics. Twitter is also used extensively by politicians who potentially have high impact on stock price movements. Based on a survey on Statista (Clement, 1 The COVID19 PRIMO dataset is accessible through the following page: https://github.com/MSBeni/COVID19 PRIMO#COVID19-PRIMO. 2020), from the first quarter of 2017 to 2020, Twitter had 186 million active users worldwide.

Based on the constructed COVID19 PRIMO dataset, the paper proposes a data-driven (DNNbased) COVID-19 adopted Hybrid and Parallel deep fusion framework for Stock price Movement Prediction (COVID19-HPSMP) that uses information fusion to combine COVID-19 related Twitter data with extended horizon market historical data. More specifically, in contrary to the existing data-driven movement prediction models, where a single DL model is used (Ronaghi, 2020) , the proposed COVID19-HPSMP is a hybrid framework with two parallel paths, i.e., one based on Convolutional Neural Network (CNN) with Local/Global Attention modules, and one integrated CNN and Bi-directional Long Short term Memory (BLSTM). The former path is incorporated within the COVID19-HPSMP framework to extract temporal features, while the latter path is used to extract spatial features. The two parallel paths are followed by a multilayer fusion layer acting as a fusion centre that combines localized features extracted in each of the two parallel paths. The COVID19 PRIMO dataset is used to evaluate the performance of the proposed COVID19-HPSMP framework, which illustrates its superior performance compared to its stand-alone (non-hybrid) counterparts.

The remainder of the paper is organized as follows: Section 2 introduces the COVID19 PRIMO dataset and formulates the stock movement prediction task. The COVID19-HPSMP hybrid framework is presented in Section 3. The implementation study and results are presented in Section 4.

Finally, Section 5 concludes the paper.

In this section, first, the COVID19 PRIMO dataset is introduced, which is constructed based on the Dow Jones stock market index and its associated Twitter messages for the period of 01/01/2016 to 30/07/2020. The focus is on the problem of stock price movement prediction as close observation of market movements can reveal presence of a significant amount of trading targets with minor movement ratios. More specifically, the paper focuses on investigating effects of COVID-19 pandemic on stock price movement prediction. In this paper, stock movement prediction is modeled as a two-class classification problem based on the adjusted closing price of the underlying stocks.

The adjusted closing price is commonly utilized to compute the associated stock dividends and earnings (Xie, 2013) . Furthermore, the adjusted closing price is beneficial to learn and predict fluctuations in the stock market (Li, 2014; Rekabsaz, 2017) .

We have prepared a new dataset for the aforementioned prediction problem, which can facilitate analysis and evaluation of potential impacts of a pandemic on stock market and can provide priceless insights to combat future possible pandemic. The constructed COVID19 PRIMO dataset consists of two components, i.e., historical prices and Twitter messages. The first component, historical data, is obtained from Dow Jones stock market. With the ticker of DJI. Dow Jones is a stock market index that measures the performance of 30 large companies like Apple, Boeing, and Microsoft. Historical stock market prices are obtained from the Yahoo finance. For this task, we used the Yahoo finance library in Python 2 to collect the data from the Yahoo API. For some of the stocks, we also used Alpha Vantage APIs 3 . The data is prepared based on three different temporal resolutions, i.e., daily; weekly, and; monthly. The daily prices are used in our model described later for the task of stock movement prediction.

Capitalizing on the facts identified in Section 1, for the news component of the COVID-19 price movement prediction dataset, we focused on Twitter. Fig. 1 shows the block diagram of the approach followed to collect and analyze Twitter messages. Web scraping from the Twitter search engine is utilized to build the Twitter dataset. The official API of the Twitter has some limitations that restricts the extent of text that can be extracted. Additionally, the official API of the Twitter cuts the tweets at times, which in turn results in items with missing data. We have developed a localized API to address the aforementioned issues. The localized API uses Twitter search engine and directly collects the required dataset from Twitter. We set up our data collection platform based on scraping the twitter website. The twitter web scraping returns the Tweet text content with a range of useful attributes, for example, T weet − ID, Tweet Created at, Retweet, Text, Favorite Count, Hashtag Text, User ID, Followers Count, Friends Count, Statuses Count, User Created at, and Location. To collect informative public Tweets, we added a constraint to our implementation to collect tweets retweeted more than once. Many other unnecessary attributes regarding a tweet were also removed from the data gathering session to focus on the essential information such as date, tweet text, and number of retweets. Fig. 2 illustrates an illustrative example of raw tweets collected by web scraping.

A critical challenge is scraping the raw content of Twitter data. Such a process takes extensive time and needs manual and cumbersome pre-processing procedures. We retrieved Dow Jones and COV ID − 19 tweets by querying symbols $DJI, #DOW , Covid19, #Covid19, #Covid − 19 and CoronaV irus. Additionally, the corresponding data associated with historical prices are collected. The constructed dataset includes related tweets from 01/01/2016 to 30/07/2020 for the Dow Jones stock index. Not every day is considered as a trading day, i.e., weekends and holidays are not among the trading dates and ought to be out of the analysis scope. To better organize and use the input, we subtract the number of days in a year from the number of weekends, the number of half trading days, and the number of market holidays. More specifically, our dataframe is created by combining historical prices and Tweet corpora and matching them to the trading days. Consequently, we considered 1, 152 trading days from January 2016 to July 2020 to build our dataset. The COVID19 PRIMO dataset is then divided into a training set from January 2016 to January 2020, and a validation set from January 2020 to February 2020. Data from 01/03/2020 to 30/07/2020 is kept to be used for test purposes.

In this sub-section, we visualize the existing relations between different parameters of the COVID19 PRIMO dataset, particularly that of COVID-19 tweets with other parameters. As stated above, the introduced dataset is constructed based on the Dow Jones stock market index and its associated Twitter messages for the period of 01/01/2016 to 30/07/2020. The COVID-19 pandemic crisis covers a fraction of the data represented in the COVID19 PRIMO dataset but plays an essential role in predicting the pandemic's effects on the stock market movements. The COVID-19 related tweets appeared in 2020 (from February to July), specifically starting to show up from the end of February. The dates containing the COVID-19 are stamped with True, making it possible to consider its distribution in the whole frame. 

In this section, we describe the constituent components used in the development of the proposed COVID19-HPSMP framework. As stated previously, the main architecture of the COVID19-HPSMP is developed based on DNNs. The prominent advantage of DNNs is their ability to extract meaningful patterns from raw data through multiple non-linear transformations and approximation of complex non-linear functions (Al-Dulaimi, 2019). More specifically, the proposed COVID19-HPSMP is a data-driven (deep learning model) designed based on hybrid or multiple-model strategies.

The COVID19-HPSMP framework extracts and interprets the available news corpus via temporal attention modeling based two key principles, i.e., "Diverse Influence" and "Sequential Context Dependency". To achieve these objectives, the COVID19-HPSMP is designed as a hybrid multi-modal fusion framework that integrates information obtained from stock market historical data and social media (Twitter data). The proposed hybrid framework consists of three paths, two parallel paths, i.e., the CNN Local/Glocal path, and; the CNN-LSTM path, together with a fusion path. In what follows, we present each of the three constituent paths of the proposed COVID19-HPSMP framework.

The first parallel path of the proposed COVID19-HPSMP framework is a CNN-based local/global attention model designed to capture and extract spatial features from the input data.

More specifically, the CNN-based path consists of Local and Global attention layers, which are described in details below.

Intuitively speaking, a word embedding model produces representations for each word in the Twitter corpus. Let us denote the l th Tweet among the available set of N T Tweets with T (l) ,

W number of words. The embedding can be thought of as a linear operator (function) that takes as an input a one-hot vector e (l)

. Note that, here N W denotes the number of words in the overall vocabulary. The embedding then maps the one-hot vector into a dense feature vector x is obtained based on a trainable weighting matrix (to be learned) of the embedding layer as follows

( 1) The embedding layer's output together with price values are provided as a concatenated input to the Local Attention Layer (LAL). The LAL focuses on the words, which are more informative within a localized window. More specifically, let the l th Tweet be represented by N 

where • denotes the Hadamard product (element-wise multiplication), σ· is the sigmoid activation function, and

where superscript T denotes transpose operator. The attention score s (l)

i is used as a weight for the words to form localized word embedding as followsx

i . A higher attention score can be interpreted as higher importance associated with that specific word than the others.

The weighted sequences then go through a Convolutional layer with a kernel size of 15, which is designed to avoid overfitting. A Max-Pooling layer is then implemented after the convolution one to creates position invariance over larger local regions and down-sample the input. Addition of the Max-Pooling layer also leads to a faster convergence rate by selecting superior invariant features, which in turn improves generalization performance.

The output of the LAL is the provided as input to a Global Attention layer (GAL). This scoring process of the GAL is similar in nature to that of the LAL (Eqs. (2)-(3)). However, the attention score, now denoted by s (l,GA) i , is computed through the entire input, i.e.,

where W

By applying global attention, the effect of uninformative words will be diminished, and the global semantic meaning will be captured more precisely through the CNN path. This completes description of the CNN Local/Global path of the proposed COVID19-HPSMPf ramework. Next, we present the CNN-BLSTM path.

The second parallel path of the proposed COVID19-HPSMP is a hybrid CNN and BLSTM attention model, referred to as the CNN-BLSTM path. Similar to the CNN Local/Global path, in the first step, the Twitter messages are provided as input to a "Word Embedding layer". As stated previously, a pre-trained unsupervised Glove model (Pennington, 2014) is used as the word embedding layer within the COVID19-HPSMP framework. Afterwards, the corpus and prices are encoded by a CNN layer to extract general contextual features. An attention layer is assigned across all the vectors to calculate the weighted corpus. At the next step, a second CNN layer is implemented to capture and learn more fine-tuned features. The first CNN layer has 50 number of filters with a window size of 25. The second CNN layer has 100 filters with a window size of 25.

The first Attention layer is used to capture essential and unique features to provide insight into the vector of the data including tweets and prices. The second Attention layer acts on each vector and calculates the weighted mean of these encoded corpus vectors to represent the overall sequential context information. A global Max-Pooling layer is then applied to capture the essential features and reduce the framework's complexity. Global Max-Pooling is similar to the regular version but with pool size equals to the size of the input. At the next stage, an attention-based BLSTM layer is designed to remember what has been previously learned to better understand the input. The attention-based BLSTM layer is described next.

To encode temporal information based on the available set of news corpus and financial timeseries data, BLSTM is incorporated within the COVID19-HPSMP hybrid framework. Learning based on financial time-series data is a sequence learning task for which BLSTMs are considered as the state-of-the-art DNN architectures. The LSTM architecture is initially developed by Hochreiter and Schmidhuber (Hochreiter, 1997) to address the vanishing and exploding gradient problem of conventional Recurrent Neural Networks (RNNs). Since then, LSTM models have gained significant popularity owing to their extensions, advancements and successful applications in different domains.

Generally speaking, LSTM is a memory-based architecture that uses different gating functions and a memory state to manage process if information through time (Di, 2018) . LSTM works based on the following update model at each time step (denoted by t)

where

vectors, and; tan(·) represents element-wise hyperbolic tangent activation function. Furthermore, h t and h t−1 represent the current and previous hidden states, respectively. In the context of the proposed COVID19-HPSMP and to encode the temporal layer, we adopt Bidirectional version of the LSTM (BDLSTM) to feed the i th word embedding. BLSTM can access both the preceding and succeeding contexts. It separates the hidden layer into two parts, forward state sequence and backward state sequence based on an iterative process.

The As a final note, we would like to briefly focus on trading from financial perspective to better link the COVID19 PRIMO (presented in Section 2) and the COVID19-HPSMP framework (discussed in the section). The COVID19 PRIMO provides means to trade based on 5-day in-advance predictions (provided by the COVID19-HPSMP). Generally speaking, a common reason for purchasing, conventionally used by traders and stockholders, is to only buy a stock when its prediction is increasing as this provides the chance to sell at a higher price in future and gain profit. This future to sell a stock, varies for different investors, e.g., some consider a short period trade, some are more eager to mid-term trades, and others focus on long-term trades. A sentimental engine that can infer the market's sense and predict its movements is a significant asset. In other words, the "BUY" and "SELL" decisions can be made more successfully when an accurate movement prediction model is available. For use of COVID19-HPSMP as a trading engine over the COVID19 PRIMO, we consider the following two trading approaches:

• The 50-50 Trading Approach: Whenever the movement prediction value is less than 0.5 (i.e., the model senses a plunge in the market's future trends) the "SELL" signal will be sent to the trader. Alternatively, when the movement prediction value is greater than 0.5 (i.e., the model predicts a boost in the market price) it will generate a "BUY" signal. We refer to this algorithm as the 50-50 approach. The traders have a chance to buy stocks at a lower price before the price goes up and sell them at a higher price. Based on these signals, provided by the COVID19-HPSMP framework, the traders have insightful predictions for the next five days. In other words, based on the COVID19 PRIMO, 5-day in advance predictions can be computed allowing decisions to buy a stock for their portfolio (when predictions show that market will face a "surge") or vise versa.

• The 60-40 Trading Approach: As an alternative approach, we consider a BUY and SELL with a HOLD scenario, by defining new thresholds over the market movement predictions.

Based on this new approach, if the movement prediction value is less than 0.4 we consider a decrease in the market price. For the values greater than 0.6, the increase in the market price is predicted. For the values between 0.4 and 0.6, the HOLD signal will be offered. We refer to this model as the 60-40 approach.

Experimental results and comparisons are presented in this section to evaluate the proposed hybrid COVID19-HPSMP framework for the task of stock movement prediction. As stated previously, the problem at hand is a classification one with the following expected outputs: (i) On one Table 1 : Accuracy, Sensitivity, and Specificity comparisons between the proposed COVID19-HPSMP framework, those of thee stand-alone models, and accuracies of algorithms from References (Hu, 2019; Xu, 2018) .

The COVID19 hand, within a 5 days prediction horizon, if the adjusted stock price of a specific day is more than that of the previous day, the output of that specific day would be 1. Then, the sum of the output values is computed over the 5 days horizon and if the sum is greater than a pre-defined threshold of 3, we consider the final output for that 5 day horizon to be 1, denoting a rise, and; (ii) On the other hand, when the adjusted stock price associated with a specific day is less than its previous day, value 0 is assigned as the output of that specific day. When the number of such 0 output values within the 5 days window is more than 3, we consider the final output to be 0, representing the fall prediction/state.

To perform the evaluations, the available Twitter news corpora is tokenized and words occurring less than 5 times are removed to construct the vocabulary. It is worth noting that removing words with limited usage will reduce the associated memory cost of the DNN models. As stated above, These three implemented models are trained with Adam optimizer (Kingma, 2014) with a learning rate of 0.001. To reduce the training times of the implemented models, Batch Normalization (Ioffe, 2015) is utilized to normalize the underlying layers. Furthermore, to avoid overfitting issues and improve the overall robustness of the implementations, 0.5 dropout is used within the fully connected layers. Finally, the computational graphs of the implemented models are constructed via Tensorflow (Abadi, 2016) to fine tune different hyper-parameters.

In this sub-section, we represent different experimental results to evaluate the performance of the proposed COVID19-HPSMP framework for the stock movement prediction. The accuracy of the proposed models areas follows: 64.65% for the stand-alone CNN-based local/global; 62.06% for the stand-alone CNN-LSTM, and 66.48% for the hybrid attention model, i.e., the COVID19-HPSMP framework. The accuracy of all three implemented models are shown in Fig. 8(a) . The accuracy is a fraction of correct predictions to the total number of predictions. The loss function associated with the evaluated models is illustrated Fig. 8(b) . Loss function demonstrates the distinction between the output of the model and the target value in order to show the probability of misclassification.

We demonstrate the accuracy of the baseline models in Table 1 comparing performance of the three Table 1 . In particular, we should point out that the higher the numerical value of sensitivity, the less likely the stock movement model returns FP results. The results show that the proposed hybrid model provides considerable improvement in terms of sensitivity, compared to its stand-alone counterparts.

(a) (b) Figure 11 : (a) Buy/Sale Return based on the 50-50 approach, i.e., whenever the movement prediction value is less than 0.5 the "SELL" signal will be sent to the trader. Alternatively, when the model predicts a boost in the market price (movement prediction value is greater than 0.5) "BUY" signal will be generated. (b) Buy/Sale Return based on 60-40 approach, i.e., when the movement prediction value is less than 0.4 we consider a decrease in the market price sending the "SELL" signal. When the prediction values are greater than 0.6, the "BUY" signal will be sent.

For the values between 0.4 and 0.6, the HOLD signal will be offered. Fig. 10 illustrates confusion matrices (Bhatt, 2021) associated to the proposed COVID19-HPSMP framework and its two stand-alone components (implemented individually). As it can be observed, the hybrid model (the COVID19-HPSMP) outperforms its counterparts. It is worth mentioning that the achieved accuracy of 66.48% is significant, although in absolute terms it seems to be low.

First, please note that average accuracies achieved in the literature for the task of price movement prediction is around 50%. Second, these lower accuracies are obtained based on a much wider window of information compared to the limited duration of the introduced COVID19 PRIMO dataset.

The limited duration of the dataset is due to recent emergence of the COVID-19 pandemic.

Stock Trading Experiments: In this part, to evaluate effects of the classification prediction mechanism on the investors' return, we conduct a back-testing experiment by simulating stock trading for March 2020 to the end of July 2020, which is during the COVID-19 financial crisis period. Investors' return (Zhang, 2020) is computed as follows

with r t = p t p t−1 − 1, and w t [0, 1]

where R t denotes the final return of the day, and r t is the return of the day based on the adjusted close price. In the proposed COVID19-HPSMP framework, term w t represents the baseline prediction of the market's movement. For evaluation purposes, we consider the following three trading mechanisms:

• First Trading Mechanism: We consider a scenario with 100 stock shares of a specific company on 01/03/2020. In this regard, first, we perform a selective test over six prominent companies from different sectors, and then conduct a Monte Carlo (MC) simulation. In both cases and to better simulate a real-world trading scenario, we consider a transaction cost of 0.3% for each trading. We have calculated risk-adjusted return based on the Sharpe Ratio (SR) approach. In summary, the final profit and SR of our COVID19-HPSMP engine is 6, 369 and 1.07%, respectively, for the 50-50 approach. The final profit and SR for the second trading mechanism (60-40) is 8, 813 and 2.49%, respectively. Generally speaking, SRs greater than one are preferred with higher the SR value, the better the risk to return scenario for the investors. Furthermore, we consider that the stock shares will be available to buy and sell at the moment they are requested. For the first scenario, the following six prominent companies are selected: Apple, W altDesney, IBM , V isa, Caterpillar, and Coca − Cola. Fig. 11(a) shows the return obtained based on the COVID19-HPSMP framework and the 50-50 trading approach, compared to the market return. Fig. 11(b) illustrates the results of this 60-40 trading algorithm. As can be seen, the returns obtained based on the 60-40 approach are higher than the return values gained from the market, standalone CNN Local/Global

Model, and standalone CNN-LSTM Model. The average market return for the 60-40 approach is 0.497 which shows improvements over the 50-50 algorithm. In Fig. 11 , the "BUY" and "SELL" signals are also shown with green and red triangles, respectively. As can be seen, the COVID19-HPSMP framework closely follows the historical data showing the real market return. Furthermore, the COVID19-HPSMP engine, in most scenarios, correctly predicts the trend of the market in the future and produces timely "BUY" and "SELL" signals. We should point out that the profit is calculated as the difference between the Adjusted Close price of Sell and Buy signals after deducting the transaction cost and risk-adjusted return. Further details on the per-stock returns are provided in Table 1 for the 50-50 trading approach, and Table 2 for the 60-40 trading approach. In brief, the final profit and Sharpe Ratio (SP) of the proposed COVID19-HPSMP engine based on the 50-50 approach is 6, 369 and 1.07% respectively.

The 60-40 trading approach provides total profit of 8, 813 with SR of 2.49%, which means that, overall, the proposed model performs considerably well. The COVID19-HPSMP model makes the highest profit for Visa Company, which is 10, 606$ and the lowest profit for Walt Disney Company is 3, 712$. Based on the results, we construct a portfolio to measure the overall profitability of the six selected stocks. It is assumed that 100 shares of each stock are bought, and the trade is made based on the proposed COVID19-HPSMP buy/sell signals. The final profit of our COVID19-HPSMP engine is 6, 369$, which means that, overall, our model performs considerably well. In a second scenario, MC simulation of 100 runs is performed considering the profit of all the companies in the Dow Jones. In each MC run, six stocks are randomly selected and the procedure outlined above for the first scenario is applied. As can be seen in Fig. 12(b) , based on the MC results, the proposed method offers a positive return over this COVID-19 crisis period and can provide the users with a profit despite being in a pandemic area.

With regards to Figs. 11 and 12, we would like to mention that the market return is computed based on daily trading while the return calculated by the COVID19-HPSMP Framework is based on Buy/Sell signals that the proposed model has produced. In other words, the results shown in the aforementioned figures include the market daily return (r t = pt p t−1 − 1), which corresponds to using a daily buy/sell signal. While at times daily trading results in higher return, the proposed model achieved average return of 0.453, which is significantly higher than the average daily market return of 0.004. It is interesting to observe that the COVID19-HPSMP Framework performs better as the model sees more market data. For instance, due to the COVID-19 crisis, abnormal market fluctuations started at the middle of March 2020 and continued until April 2020. Some false predictions in those periods are due to these fluctuations since our model has not seen these type of market reactions during its training phase. After learning from these initial abnormal market fluctuations, the model gradually performs better in predicting the market trend. For example, it can be observed that there is a sharp drop in daily returns in mid of June 2020 where the proposed model's adaptive nature (actively learning) has resulted in achieving positive reruns during that period.

• Second Trading Mechanism: In the first trading mechanism, we considered an equal weight for all the stocks in our portfolio. Here, we construct a portfolio based on market price weighting (Plyakha, 2017) as follows

where w p i represents the weight of each stock based on its market price, and p i denotes the market price (we assume that market prices are adjusted close price of the stock on the first day of March). Weights are applied based on each stock's market price on the date that the transaction is going to be placed. Further details on market price weighting is provided in Table 1 . To evaluate the performance of our model Expected return (E(R)) and standard deviation of the return (Std(R)) are computed. The results show that Visa has a profit of 10, 606$ with standard deviation of 0.156, which means that visa stock is riskier than others.

While the Walt Disney has lower profit and lower risk than the other stocks. Overall, the sum From a finance perspective, the risk also needs to be presented as a model can lead to more returns but higher risk. The risk may be defined as the variability of returns that are expected from a given investment. The greater the variability, the riskier the stock.

Investment in the Treasury bond is risk-free because of the governmental guarantees. In contrast, investment in equity shares is risky because of the uncertainty on return. Some shares are riskier than others, and even in years when the overall stock market goes up, many individual shares decline in their price. Therefore, investing in one share is extremely risky.

Investment risk, then, is related to the probability of actually earning a low or negative return.

Therefore, the probability of low earning or negative return is higher in riskier investments.

Consequently, it is necessary to measure the degree of risk associated with the investment. For this purpose, we consider the scenario where 100 shares in the Dow Jones market are bought on 01/03/2020 and held until the end of July. As a measure of risk, we calculated Sharpe Ratio (SR). The cumulative return and sharpe ratio of our baseline models based on two the trading approaches are illustrated in Tables 5 and 6. The proposed COVID19-HPSMP based on the 60-40 trading approach obtains the highest cumulative return and SR compared to its counterparts. It is worth mentioning that a higher SR shows superiority of a portfolio compared to its counterparts.

As a final experiment, we evaluate the performance based on the following three trading shares in the Dow Jones market are bought on 01/03/2020 based on buy, sell and hold scenarios (the so called 60-40 approach); (ii) Hold Strategy 1: Second, we consider the "BUY"

and "HOLD" strategy where 100 shares in the Dow Jones market are bought and held for a long period regardless of the fluctuations in the market. For this purpose, we bought the stock on the day that the COVID19-HPSMP framework predicted a "BUY" signal and "HOLD"

it until the end of the period. Based on the signal generated by our model, we buy the stock on 2020-03-19, and sell it on 2020-07-29 when the models predicted a "SELL" signal, and;

(iii) Hold Strategy 2: Finally, we consider the scenario where 100 shares in the Dow Jones market are bought on 01/03/2020 and held until the end of July. The results are shown in Table 4 illustrating that the proposed COVID19-HPSMP framework achieves higher return across different strategies compared to its counterparts.

Comparisons with Trading Indicators: In this part, we compare the COVID19-HPSMP engine the Moving Average Convergence/Divergence (MACD) (Vaidya, 2020) , which are among the most widely used technical indicators by market technicians to identify short, medium, and long-term price movements. First we focus on the SAM approach, which is also known as moving mean or rolling mean in statistics. The SMA is one of the oldest and most straightforward strategies for analysing signals and trade based on the results. In this approach, the moving average of the price or return is calculated to estimate the trends in the market price. The SMA is calculated by creating a series of averages of the market data's time-ordered subsets. While different window times can be used, a 5-day moving average is calculated for comparison with the proposed COVID19-HPSMP framework. Fig. 11 (b) compares returns obtained from the COVID19-HPSMP framework with those obtained from the SMA approach and that of the market. Fig. 11 (b) demonstrates that our COVID19-HPSMP framework outperforms the SMA in prediction of the market's trend. A second indicator utilized for evaluation purposes is the MACD method, which is another important trading indicator wildly used by traditional technical analysis and algorithmic trading. By leveraging the close price's historical value, MACD can be calculated as a collection of three time-series: (i) The MACD value series, which is the difference between the shorter period (fast) and the longer period (slow) of the exponential moving average (EMA); (ii) The MACD Signal, which is the EMA of the MACD value series, and; (iii) A divergence series, which is the difference between Items (i) and

(ii). Three parameters a, b, c are used to calculate the MACD, where a is related to the fast EMA;

b relates to slow EMA, and; c is related to EMA. To compare the proposed framework with the M ACD indicator, we used a = 12, b = 26, and c = 9, which are the commonly used values for these parameters. The BUY/SELL signals generated based on the MACD indicator is shown in Fig. 13 . The MACD provides five different Buy/Sell signal as shown in Fig. 13 , which is much less than the Buy/Sell signals provided by the COVID19-HPSMP framework. The cumulative return of different models are illustrated in Tables 5 and 6. Based on the results shown in Tables 5 and 6, the COVID19-HPSMP framework outperforms the other two indicators in terms of the cumulative return.

Significance Tests: As a final experiment, we evaluate the hypothesis that predictions made by the COVID19-HPSMP framework are more profitable than those obtained via a standalone CNN Local/Global model (potentially illustrating benefits of a hybrid design), and the Moving Average Convergence/Divergence (MACD) method, a common trading indicator (potentially illustrating benefits in comparison to a commonly utilized indicator). In this regard, we have conducted statistical significance analysis of predictive models given their rate of return through a null hypothesis test. In the first test, the null hypothesis is to use the stand-alone CNN Local/Global engine for investment, and the alternative hypothesis is to use the COVID19-HPSMP framework as a recommendation engine for investment. The price movement prediction in stock market is assumed to be aleatory (null is not rejected) until it is proven the results of a recommendation engine (in this case, the COVID19-HPSMP framework) has no relationship with the results of a random trader (null is rejected). To define a confidence threshold, a cut point of α=2.145 corresponding to 95% of confidence is established. The resulting t-value is equal to 3.0526, which is associated with p-value of 0.004303 showing a significant result as p < 0.05. This suggests that the null hypothesis is not accepted, therefore, predictions made by COVID19-HPSMP are more profitable.

Similarly, in a second experiment, we check if predictions of the COVID19-HPSMP framework is more profitable than those of the MACD indicator. The computed t-value is 2.8433 (i.e., p-value of 0.00651), which means that the COVID19-HPSMP framework reaches better results compared to the MACD, which is a common trading indicator.

As a final note, we would like to mention that the variance between the actual return from a given portfolio and the expected return over a given period of time is referred to as an "Abnormal Return". A positive abnormal return refers to the scenario that the actual return is higher than its expected return. Abnormal return can be achieved in stock market, however, the volatile nature of the market is hard to earn abnormal return using only the financial indicators, i.e., the stock data, as the market is dependent on a variety of social, political, and economical factors. There are three forms of efficient markets (i.e., weak, semi-strong, and strong), defined based on the nature of information utilized to determine asset prices. In the weak form, asset prices referring to all past prices and trading volume information fully reflect all the market data. In the semi-strong form, asset prices reflect all publicly known and available information. In the strong form, asset prices fully reflect all information, including public (e.g., news) and private information. We consider semi-strong form of efficiency for the markets, in which prices reflect the news data and financial indicators. The proposed COVID19-HPSMP framework incorporates effects of social media trends related to COVID-19 on stock movement prediction. The proposed framework uses information fusion to combine COVID-19 related Twitter data with historical data to trade 5-day in-advance in order to predict the movement of the market. Our model can predict the stock market's fluctuations with more than 66% accuracy, which will hopefully be a metric to earn abnormal return.

Motivated by abrupt, sudden, and negative effects of COVID-19 pandemic on stock markets, first, the paper introduced a unique COVID-19 related PRIce MOvement prediction (COVID19 PRIMO) dataset. The constructed dataset incorporates effects of social media trends related to COVID-19 on stock market price movements. Based on the constructed COVID19 PRIMO dataset, the paper then proposed a novel data-driven (DNN-based) COVID-19 adopted Hybrid and Parallel deep fusion framework for Stock price Movement Prediction (COVID19-HPSMP). The proposed framework uses information fusion to combine COVID-19 related Twitter data with extended horizon market historical data. More specifically, in contrary to the existing data-driven stock price movement prediction models, where a single DNN model is used, the COVID19-HPSMP framework is a hybrid model consisting of two parallel paths (i.e., the CNN Local/Glocal path, and; the CNN-LSTM path) and a fusion path that combines localized features. Each of the two parallel paths is a unique attention module extracting different attention related features. The rationale behind such a hybrid and parallel structure is the significance of the attention network and the intuition that extracting different attention-related features would improve the overall performance of the model.

The proposed COVID19-HPSMP architecture can predict the stock price movements during the pandemic crisis to forecast sudden sharp movements (fall or rise) in the stock market. Based on the results of the COVID19-HPSMP architecture, we can predict the stock market's fluctuations with more than 66% accuracy, which will hopefully be a metric to be more prepared for the unexpected havocs.

Tensorflow: Large-scale Machine Learning on Heterogeneous Distributed Systems

Death and Contagious Infectious Diseases: Impact of the COVID-19 Virus on Stock Market Returns

A Multimodal and Hybrid Deep Neural Network Model for Remaining Useful Life Estimation

An Intelligent Technique for Stock Market Prediction

SutteARIMA: Short-term Forecasting Method, a Case: Covid-19 and Stock Market in Spain

Sutte Indicator: A Technical Indicator in Stock Market

COVID-19 and Stock Market Volatility: An Industry Level Analysis

Stock Price Change Prediction Using News Text Mining

Cervical Cancer Detection in Papsmear whole slide Images using convNet with Transfer Learning and Progressive Resizing

Twitter Mood as a Stock Market Predictor

Stock Market Movement Forecast: A Systematic Review

Multitask Learning. Machine Learning

Deep Learning Networks for Stock Market Analysis and Prediction: Methodology, Data Representations, and Case Studies

Machine Learning Techniques and Older Adults Processing of online Information and Misinformation: A COVID-19 Study

Number of Monetizable Daily Active Twitter Users (mDAU) Worldwide from 1st Quarter 2017 to 1st Quarter 2020

Deep Learning Essentials: your Hands-on Guide to the Fundamentals of Deep Learning and Neural Network Modeling

Technical Analysis of Stock Trends

Market Efficiency, Long-term Returns, and Behavioral Finance

Financial Markets And Monetary Policy

Explainable Stock Prices Prediction from Financial News Articles using Sentiment Analysis

Long Short-Term Memory

CNNpred: CNN-based Stock Market Prediction using a Diverse Set of Variables

Listening to Chaotic Whispers: A Deep Learning Framework for News-oriented Stock Trend Prediction

Using Social Media Mining Technology to Improve Stock Price Forecast Accuracy

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Applications of Deep Learning in Stock Market Prediction: Recent Progress

Adam: A Method for Stochastic Optimization

Algorithms in Future Capital Markets

News Impact on Stock Price Return via Sentiment Analysis. Knowledge-Based Systems

NLTK: The Natural Language Toolkit

Predicting Abnormal Returns from News Using Text Classification

COVID-19 and the March 2020 Stock Market Crash. Evidence from S&P1500. Finance Research Letters

Interactive Gaussian-sum Filtering for Estimating Systematic Risk in Financial Econometrics

Gas Detection and Identification Using Multimodal Artificial Intelligence based Sensor Fusion

Predicting Stock and Stock Price Index Movement using Trend Deterministic Data Preparation and Machine Learning Techniques

The Impact of Stock Market Price Fourier Transform Analysis on the Gated Recurrent Unit Classifier Model

Volatility Prediction Using Financial Disclosures Sentiments with Word Embedding-based IR Models

Glove: Glove Vector of Word Representation

Why Does an Equal-Weighted Portfolio Outperform Value-and Price-Weighted Portfolios

Stock Price Prediction using Deep Learning and Frequency Decomposition

NDSMPF: A Noisy Deep Neural Network Fusion Framework for Stock Price Movement Prediction

Textual Analysis of Stock Market Prediction Using Breaking Financial News: The AZFin Text System

Interpretable Convolutional Neural Networks with Dual Local and Global Attention for Review Rating Prediction

Predicting Stock Movements based on Financial News with Segmentation. Expert Systems with Applications

Giving Content to Investor Sentiment: The Role of Media in The Stock Market

Moving Average Convergence-Divergence (MACD) Trading Rule: An Application in Nepalese Stock Market(NEPSE)

Semantic Frames to Predict Stock Price Movement

Stock Movement Prediction from Tweets and Historical Prices

Stock Prices Prediction using the Title of Newspaper Articles with Korean Natural Language Processing

The Role of Investor Attention in Predicting Stock Prices: The Long Short-Term Memory Networks Perspective

Deep Learning for Portfolio Optimization