admin
August 15, 2022
12:31 pm

Abstract

We study the price impact of order book events – limit orders, market orders and cancelations – using the NYSE TAQ data for 50 U.S. stocks. We show that, over short time intervals, price changes are mainly driven by the order flow imbalance, defined as the imbalance between supply and demand at the best bid and ask prices. Our study reveals a linear relation between order flow imbalance and price changes, with a slope inversely proportional to the market depth. These results are shown to be robust to seasonality effects, and stable across time scales and across stocks. We argue that this linear price impact model, together with a scaling argument, implies the empirically observed “square-root” relation between price changes and trading volume. However, the relation between price changes and trade volume is found to be noisy and less robust than the one based on order flow imbalance.

1. Introduction

The availability of high-frequency records of trades and quotes has stimulated an extensive empirical and theoretical literature on the relation between order flow, liquidity and price movements in order-driven markets. A particularly important issue for applications is the impact of orders on prices: the optimal liquidation of a large block of shares, given a fixed time horizon, crucially involves assumptions on price impact (see Bertsimas and Lo [6], Almgren and Chriss [2], Obizhaeva and Wang [35]).

Various models for price impact have been proposed in the literature but there is little agreement on how to model it [7]. In the empirical literature, price impact has been described by various authors as linear, non-linear, square root, virtual, mechanical, temporary, instantaneous, permanent or transient. The only consensus seems to be the intuitive notion that imbalance between supply and demand moves prices.

The empirical literature on price impact has primarily focused on trades. One approach is to study the impact of “parent orders” gradually executed over time using proprietary data (see Engle et. al [14], Almgren et. al [3]). Alternatively, empirical studies on public data [16, 18, 20, 28, 29, 43, 38, 39] have investigated the relation between the direction and sizes of trades and price changes and typically conclude that the price impact of trades is an increasing, concave (“square root”) function of their size. This focus on trades leaves out the information in quotes, which provide a more detailed picture of price formation [15], and raises a natural question: is volume of trades truly the best explanatory variable for price movements in markets where many quote events can happen between two trades?

Understanding the price impact of orders is also important from a theoretical perspective, in the context of optimal order execution. Huberman and Stanzl [25] show that there are arbitrage opportunities if the effect of trades on prices is permanent and the impact is non-linear; Gatheral [19] extends this analysis by showing that if the price impact function is non-linear, impact needs to decay in a particular way to exclude arbitrage. Bouchaud et al. [9] associated the decay of price impact of trades with limit orders, arguing that there is a “delicate interplay between two opposite tendencies: strongly correlated market orders that lead to super-diffusion (or persistence), and mean reverting limit orders that lead to sub-diffusion (or anti-persistence)”. This insight implies that looking solely at trades, without including the effect of limit orders amounts to ignoring an important part of the price formation mechanism.

There is ample evidence that limit orders play an important role in determining price dynamics. Arriving limit orders significantly reduce the impact of trades [44] and the concave shape of the price impact function changes depending on the contemporaneous limit order arrivals [41]. The outstanding limit orders (also known as market depth) significantly affect the impact of an individual trade ([30]) and low depth is associated with large price changes [45, 17]. Hasbrouck and Seppi [22] use depth as one of the factors that determine price impact. The emphasis in these studies remains, however, on trades and there are few empirical studies that focus on limit orders from the outset. Notable exceptions are Engle & Lunde [15], Hautsch and Huang [23] who perform an impulse-response analysis of limit and market orders, Hopman [24] who analyzes the impact of different order categories over 30 minute intervals and Bouchaud et al. [12] who examine the impact of market orders, limit orders and cancelations at the level of individual events.

1.1 Summary

We conduct in this study an empirical investigation of the impact of order book events –market orders, limit orders and cancelations– on equity prices. Although previous studies give a relatively complex description of this impact, we argue that, in fact, their impact on price dynamics may be modeled parsimoniously through a single variable, the order flow imbalance (OFI), which represents the net order flow at the bid and ask and tracks changes in the size of the bid and ask queues by

increasing every time the bid size increases, the ask size decreases or the bid/ask prices increase
decreases every time the bid size decreases, the ask size increases or the bid/ask prices decrease.

Interestingly, this variable treats a market sell and a cancel buy of the same size as equivalent, since they have the same effect on the size of the bid queue. We find that this aggregate variable explains mid-price changes over short time scales in a linear fashion, for a large sample of stocks, with an average R²of 65%. The resulting price impact model relates prices, trades, limit orders and cancelations in a simple way: it is linear, requires the estimation of a single parameter and it is robust across stocks and across timescales.

The slope of this relation, which we call the price impact coefficient, exhibits intraday seasonality in line with known intraday patterns observed in spreads, market depth and price volatility [1, 4, 31, 34] which have been explained in terms of intraday shifts in information asymmetry [33] or informativeness of trades [21]. Motivated by a stylized model of the order book, we relate the intraday changes in the price impact coefficient to variations in market depth and show that price impact is inversely proportional to the depth of the order book. This allows us to explain intraday patterns in price impact and price volatility using only observable quantities – the order flow imbalance and the market depth, as opposed to unobservable parameters previously invoked in the literature, such as information asymmetry or informativeness of trades.

The intuition that “it takes volume to move prices”, though widely confirmed by empirical studies [27], is not easy to explain theoretically (see [37, Chapter 6.2]). In Section 4, we show that our price impact model, together with a scaling argument, leads to an apparent “square root” relation between price changes and trade volume, similar to some findings in the empirical literature [11, 40]. However, we argue that this relation is not robust and is a statistical artifact due to the aggregation of data.

1.2 Outline

The article is structured as follows. In Section 2, motivated by a stylized model of the order book, we specify a parsimonious model that links stock price changes, order flow imbalance and market depth. Section 3 describes the trades and quotes data and estimation results for our model. There, we also show how intraday patterns in depth and order flow imbalance generate intraday patterns in price impact and price volatility. In Section 4 we discuss the role of trading volume as an explanatory variable and show that order flow imbalance is more effective in explaining price moves than variables based on trades. We also derive a scaling relation between order flow imbalance and traded volume and show how the “square-root” price impact of volume follows from our model. We present our conclusions in Section 5.

2. A model for the price impact of orders

2.1 Variables

We focus on ‘Level I order book’: the limit orders sitting at the best bid and ask. Every observation of the bid and the ask consists of the bid price P^B, the size q^Bof the bid queue (in number of shares), the ask price P^Aand the size q^Aof the ask queue (in number of shares):

where P_kis the mid-quote price at time t_kand δ is the tick size (equal to 1 cent in our data).

2.2 A stylized model of the order book

Consider first a stylized model of the order book in which

the number of shares at each price level beyond the best bid/ask is equal to D.
limit orders arrivals and cancelations occur only at the best bid/ask.

We will show that under these assumptions a linear relation holds between order flow imbalance and price changes. Consider three scenarios, when only market buy orders, limit buy orders or limit sell cancels happen over some time interval [t,t + ∆t]:

Market sell orders remove M^sshares from the bid.

Market sell orders remove M^sshares from the bid, while limit buy orders add L^bshares to the bid.

Market sell orders and limit buy cancels remove M^s+ C^bshares from the bid, while limit buy orders add L^bshares to the bid.

The three variables M^b, C^sand L^sfor the ask can be defined analogously. Under the above assumptions, the impact of order book events at the bid (ask) side of the book is additive and only depends on their net effect on the bid (ask) queue size:

∆P^b= d(L^b− C^b− M^s)/De

Similarly, for the ask:

∆P^a= −d(L^s− C^s− M^b)/De

These relations are remarkably simple – they involve no parameters and incorporate the effects of all order book events on bid and ask prices. Although the following analysis can be carried for the bid and the ask prices separately, we take their average (the mid-price) to simplify the analysis:

2.3 Model specification

In reality, order books have complex dynamics and the relation (1) will only hold in a statistical sense. For example, limit orders and cancelations occur at all levels of the order book. The distribution of depth across price levels often has humps, gaps and is itself a separate object of study [39, 46]. Moreover, the depth is subject to important intraday fluctuations. Finally, there may be hidden orders in the book which are not reported in the data [5]. With these considerations, we suggest the following relation:

where c,λ are constants and ν_iis a noise term. Note that the stylized model exposed above corresponds to λ = 1.

The specification (2-3) may be regarded as a model of the instantaneous price impact over a short time interval [t_k₋₁,t_k]. An order, submitted at τ ∈ [t_k₋₁,t_k], has a contribution e_τand joins the aggregate order flow imbalance OFI_k. If the order goes in the same direction as the majority of the orders (sgn(e_τ) = sgn(OFI_k)), it reinforces the concurrent order flow imbalance and can affect the price. If the order goes against the concurrent order flow imbalance (sgn(e_τ) = −sgn(OFI_k)), it is compensated by other orders and may have an instantaneous impact of zero. In our model all events (including trades) have a linear price impact, equal to β on average. Their realized impact, however, depends on the rest of the orders that arrive during the same time interval.

The idea that the concurrent limit order activity can make a difference in terms of trades’ impact was demonstrated by Stephens et al. [41], where authors show that the shape of the price impact function essentially depends on the contemporaneous limit order activity. Our approach can also be related to the model proposed by Bouchaud et al. [12]. where order book events have a linear impact on prices, which depends on their signs and types^[2]. The major difference of our models lies in the aggregation across time and events. As argued in [12], order book events have complicated auto- and cross-correlation structures on the timescale of individual events, which typically vanish after 10 seconds. In our data the autocorrelations at a timescale of 10 seconds are small and quickly vanish as well (ACF plots for a representative stock are shown on Figure 1). Finally, Hasbrouck and Seppi [22] propose a model similar to (2, 3) for explaining the price impact of trades. Although their focus is on trades, they also allow the price impact coefficient to depend on contemporaneous liquidity factors and change through time.

However, the linear equation (2) is quite different from models of price impact that consider only the size of trades [18, 20, 29, 43, 38, 39]. Instead of modeling price impact of trades as a (nonlinear) function of trade size, we show that the price impact of all events (including trades) is a linear function of their size after events are aggregated into a single imbalance variable. In Section 4 we will argue that, first, the effect of trades on prices is adequately captured by the order flow imbalance and, second, that if one leaves out all events except trades, the relation 2 leads to an apparent concave relation between price changes and trade volume.

The next section provides an overview of the estimation results for our model.

Figure 1: ACF of the mid-price changes ∆P_k, the order flow imbalance OFI_kand the 5% significance bounds for the Schlumberger stock (SLB).

3. Estimation and results

3.1 The trades and quotes (TAQ) data

Our data set consists of one calendar month (April, 2010) of trades and quotes (TAQ) data for 50 stocks. The stocks were selected by a random number generator from the S&P 500 constituents. The S&P 500 composition for that month was obtained from Compustat and the data for individual stocks was obtained from the TAQ consolidated quotes and TAQ consolidated trades databases. The data were obtained through Wharton Research Data Services (WRDS).

Consolidated quotes contains all changes in queue sizes at the best bid and ask. For each stock, a data update consists of a timestamp (rounded to the nearest second), bid price, bid size, ask price, ask size and exchange flag. Consolidated trades (or market orders) consist of a timestamp, a price and a size. These two data sets are often referred to as Level 1 data, as opposed to Level 2 data, which also includes quote updates deeper in the book.

Our reason for using TAQ data rather than Level 2 order book data, is that it is far more accessible, yet contains all events in the top order book (best bid and ask updates). We demonstrate that Level 1 TAQ data can be successfully used to study limit orders and we hope that more empirical studies of that subject will follow. We note that the ratio of the number of quote updates to the number trades is roughly 40 to 1 in our data. Many empirical studies have focused exclusively on trades rather than quotes, but the sheer ratio in the size of these data sets is a good indicator that more information may be conveyed by the quotes than by trades.

Using a procedure described in detail in the appendix, we aggregate all quote updates to estimate the National Best Bid and Offer sizes and prices (NBBO) at each quote update. Instead of aggregating all exchanges in this fashion, one may also simply filter by the exchange flag and study one exchange at the time. Focussing on one exchange at a time yields similar results.

We use a uniform grid in time {t₀,…,t_N} with a timescale t_k−t_k₋₁≡ ∆t = 10 seconds to compute the price changes and the order flow imbalances. To test the robustness of our findings to the choice of the basic timescale, we repeated our calculations on a subsample of stocks for different values of ∆t, ranging from 10 quote updates (usually less than half of a second in our data) up to 10 minutes. The fit of our model generally increases with ∆t, but the rest of the results stays the same. Time aggregation serves two purposes: first, it alleviates the issue of data discreteness and second, it mitigates the errors due to the trade matching algorithm (described in the Appendix).

3.2 Empirical findings

We assume that the price impact coefficient β is constant over each half-hour interval [T_i,T_i₊₁] and estimate the model by ordinary least squares regression in each half-hour subsample for each stock:

Figure 2 presents a scatter plot of ∆P_kagainst OFI_kfor one of the half-hour subsamples. Table 2 reports regression outputs, averaged across time for each stock. This table provides strong evidence of a linear relation between order flow imbalance and price changes. The goodness of fit is surprisingly high for all of the stocks, suggesting that the model (2) performs well regardless of stock-specific features^[3]. In addition to the high quality of fits, the regression coefficient β_iis virtually always statistically significant (at a 95% level of the z-test), while the intercept is mostly insignificant. Figure 3 represents the histogram of excess kurtosis values of the residuals ˆ_kacross subsamples: the relatively low level of kurtosis shows that the residuals are not predominantly associated with large price changes. Since the regression residuals demonstrate heteroscedasticity, we used White’s heteroscedasticity-consistent standard errors for the z-test. To check for higher order/nonlinear dependence we add a quadratic term ˆγ_Q,iOFI_k|OFI_k| to the regression. The increase in R², from 65% to 68% on average, is barely noticeable and the coefficient ˆγ_Q,iis statistically insignificant in most samples.

Figure 2: Scatter plot of ∆P_kagainst OFI_kfor the Schlumberger stock (SLB), 04/01/2010 11:30-12:00pm.

Figure 3: Distribution of excess kurtosis of the residuals ˆ_kacross stocks and time.

Table 1. Descriptive statistics

Name	Ticker	Price	Daily volume,	Number of best quote	Number of trades	Average Spread,	Maximum spread,	Best quote depth,
			shares	updates		cents	cents	shares
Advanced Micro Devices	AMD	9.61	20872996	417204	6687	1	1	1035
Apollo Group	APOL	62.92	1949337	172942	4095	2	5	15
American Express	AXP	45.21	8678723	559701	7748	1	24	79
Autozone	AZO	179.03	243197	43682	1081	9	35	7
Bank of America	BAC	18.43	164550168	1529395	15008	1	1	3208
Becton Dickinson	BDX	78.07	1130362	61029	2968	2	5	15
Bank of New York Mellon	BK	31.77	6310701	285619	5518	1	1	122
Boston Scientific	BSX	7.13	25746787	309441	6768	1	1	2965
Peabody Energy corp	BTU	47.14	5210642	298616	7267	1	3	29
Caterpillar	CAT	67.20	6664891	392499	8224	1	2	38
Chubb	CB	52.22	1951618	149010	3601	1	2	43
Carnival	CCL	40.16	4275911	215427	5503	1	2	53
Cincinnati Financial	CINF	29.41	688914	51373	1528	1	2	42
CME Group	CME	322.83	418955	38504	1412	31	103	5
Coach	COH	41.91	3126469	176795	4458	1	2	41
ConocoPhillips	COP	56.09	9644544	426614	8621	1	2	84
Coventry Health Care	CVH	24.16	1157022	79305	2213	1	2	38
Denbury Resources	DNR	17.88	5737740	263173	4643	1	1	186
Devon Energy	DVN	66.98	3260982	177006	5805	2	4	18
Equifax	EFX	35.34	799505	62957	1945	1	3	39
Eaton	ETN	78.53	1757136	67989	3580	2	6	13
Fiserv	FISV	52.56	1038311	58304	2208	1	3	20
Hasbro	HAS	39.48	1322037	86040	2672	1	2	34
HCP	HCP	32.63	2872521	213045	4357	1	2	48
Starwood Hotels	HOT	50.59	3164807	150252	5106	2	4	22
Kohl’s	KSS	56.88	3064821	128196	4936	1	3	27
L-3 Communications	LLL	94.64	670937	72818	2141	2	6	9
Lockheed Martin	LMT	84.14	1416072	88254	3333	2	5	15
Macy’s	M	23.40	8324639	491756	6469	1	1	176
Marriott	MAR	34.45	5014098	238190	5499	1	2	65
McAfee	MFE	40.04	2469324	109073	3561	1	2	40
McGraw-Hill	MHP	34.90	1954576	102389	3261	1	2	42
Medco Health Solutions	MHS	63.22	2798098	109382	4680	1	3	25
Merck	MRK	36.03	13930842	448748	7997	1	1	231
Marathon Oil	MRO	32.33	5035354	341408	5522	1	1	143
MeadWestvaco	MWV	26.96	1035547	92825	2312	1	3	37
Newmont Mining	NEM	53.43	5673718	435295	7717	1	2	38
Omnicom	OMC	41.17	3357585	150800	4359	1	2	65
MetroPCS Communications	PCS	7.53	4424560	107967	2901	1	1	523
Pultegroup	PHM	11.80	6834683	262420	4604	1	1	319
PerkinElmer	PKI	23.98	1268774	78114	2127	1	2	72
Ryder System	R	44.01	631889	47422	2085	2	5	11
Reynolds American	RAI	54.44	773387	56236	2076	1	4	22
Schlumberger	SLB	67.94	9476060	440839	10286	1	2	39
Teco Energy	TE	16.52	1070815	70318	1807	1	1	148
Time Warner Cable	TWC	53.21	1770234	88286	3554	2	3	22
Whirlpool	WHR	97.73	1424264	134152	3348	4	9	10
Windstream	WIN	11.03	2508830	104887	2937	1	1	798
Watson Pharmaceuticals	WPI	42.51	895967	63094	2024	1	3	29
XTO Energy	XTO	48.13	7219436	612804	5040	1	7	225
Grand mean		51.75	7512376	223232	4552	2	6	227

Table 1 presents the average mid-price, daily transaction volume, daily number of best quote updates, daily number of trades, spread and the depth at the best bid and ask for 50 randomly chosen U.S. stocks. All values are calculated from the filtered data, that consists of 21 trading day during April, 2010.

Table 2. Relation between price changes and order flow imbalance.

Ticker	Average results	Hypothesis testing
αˆ	t(αˆ)	βˆ	t(β^ˆ)	γˆ_Q	t(γˆ_Q)	R2	{α 6= 0}	{β 6= 0}	{γ_Q6= 0}
AMD	-0.0032	-0.17	0.0008	9.96	1.4E-07	0.68	64%	0%	98%	22%
APOL	0.0038	0.10	0.0555	10.32	-2.2E-04	-1.17	63%	12%	91%	4%
AXP	0.0019	0.08	0.0082	13.87	-3.8E-06	-0.88	69%	11%	100%	5%
AZO	0.0101	0.33	0.1619	6.39	-9.3E-04	-0.89	47%	23%	97%	3%
BAC	-0.0018	-0.09	0.0002	18.36	1.9E-09	0.01	79%	1%	100%	8%
BDX	-0.0008	-0.06	0.0536	10.08	-1.1E-04	-0.38	63%	9%	100%	8%
BK	-0.0078	-0.19	0.0069	14.97	-4.0E-06	-0.57	74%	3%	100%	6%
BSX	0.0000	-0.01	0.0003	6.12	7.8E-08	1.14	58%	0%	81%	22%
BTU	0.0048	0.12	0.0242	14.51	-3.5E-05	-1.26	72%	11%	100%	3%
CAT	0.0147	0.23	0.0194	14.85	-1.9E-05	-1.13	71%	12%	99%	3%
CB	-0.0086	-0.07	0.0191	11.97	-3.5E-07	0.00	64%	5%	100%	8%
CCL	-0.0067	-0.18	0.0140	13.88	-1.2E-05	-0.64	70%	3%	99%	7%
CINF	-0.0030	-0.02	0.0260	10.73	-7.0E-06	0.27	70%	1%	98%	16%
CME	0.0506	0.05	0.6262	4.98	-7.2E-03	-0.99	35%	15%	94%	2%
COH	-0.0221	-0.45	0.0179	12.75	-1.7E-05	-0.77	69%	2%	100%	3%
COP	-0.0008	0.06	0.0084	12.50	-5.8E-06	-1.17	68%	10%	100%	3%
CVH	-0.0034	-0.06	0.0217	10.83	7.6E-06	0.20	65%	3%	99%	10%
DNR	-0.0008	-0.04	0.0045	12.76	-1.3E-07	0.19	69%	1%	99%	13%
DVN	0.0112	0.18	0.0370	11.48	-1.0E-04	-1.59	65%	17%	97%	0%
EFX	-0.0032	-0.04	0.0222	8.71	6.4E-05	0.64	56%	1%	98%	18%
ETN	-0.0076	0.05	0.0712	10.51	-2.3E-04	-1.14	65%	14%	98%	1%
FISV	-0.0002	0.06	0.0397	10.42	-2.3E-05	-0.19	63%	4%	100%	8%
HAS	-0.0031	-0.02	0.0222	11.45	4.7E-06	0.21	67%	3%	100%	16%
HCP	-0.0078	-0.17	0.0150	13.60	-1.4E-05	-0.46	67%	2%	100%	6%
HOT	-0.0012	0.05	0.0345	12.64	-7.2E-05	-1.21	68%	10%	99%	2%
KSS	-0.0030	-0.04	0.0317	13.82	-5.4E-05	-0.80	71%	10%	98%	3%
LLL	0.0160	0.32	0.1000	11.76	-3.8E-04	-0.75	67%	14%	96%	3%
LMT	0.0006	0.00	0.0520	13.58	-1.2E-04	-0.98	72%	14%	100%	1%
M	-0.0010	0.04	0.0043	15.82	8.8E-08	0.13	75%	0%	100%	12%
MAR	-0.0039	-0.02	0.0121	14.61	-4.1E-06	-0.23	71%	3%	100%	4%
MFE	0.0087	0.16	0.0205	12.72	-3.8E-05	-0.38	68%	7%	100%	7%
MHP	-0.0073	-0.13	0.0211	11.62	5.8E-06	0.14	68%	2%	99%	11%
MHS	-0.0055	-0.16	0.0334	11.70	-8.3E-05	-1.10	66%	9%	99%	3%
MRK	-0.0065	-0.20	0.0032	12.53	-5.4E-07	-0.38	69%	1%	100%	8%
MRO	0.0018	0.07	0.0058	13.67	-3.6E-07	0.22	69%	5%	100%	13%
MWV	-0.0011	0.01	0.0205	11.79	-1.7E-05	-0.25	68%	3%	100%	7%
NEM	-0.0102	-0.22	0.0170	13.81	-1.9E-05	-1.36	71%	8%	100%	2%
OMC	-0.0099	-0.28	0.0144	11.88	-4.5E-06	-0.01	65%	2%	99%	13%
PCS	-0.0006	-0.03	0.0015	5.21	1.8E-06	1.01	53%	0%	79%	24%
PHM	0.0006	0.03	0.0027	10.33	8.4E-07	0.55	66%	1%	98%	21%
PKI	-0.0004	-0.03	0.0102	7.25	4.1E-05	1.10	53%	2%	94%	29%
R	0.0006	0.03	0.0667	10.14	3.7E-05	0.01	63%	8%	98%	10%
RAI	-0.0070	-0.10	0.0396	10.40	2.6E-05	0.01	66%	5%	100%	11%
SLB	-0.0077	-0.15	0.0198	16.76	-1.8E-05	-1.15	76%	7%	100%	1%
TE	0.0011	0.05	0.0049	6.66	1.4E-05	1.45	54%	2%	86%	30%
TWC	-0.0130	-0.13	0.0384	11.80	-5.6E-05	-0.44	64%	8%	99%	5%
WHR	0.0628	0.63	0.1278	10.26	-3.3E-04	-0.80	65%	22%	97%	4%
WIN	-0.0004	-0.03	0.0009	3.12	1.5E-06	0.76	44%	1%	60%	15%
WPI	-0.0090	-0.21	0.0270	10.47	2.9E-05	0.28	66%	3%	98%	14%
XTO	-0.0088	-0.18	0.0029	13.28	2.7E-07	0.30	65%	0%	100%	18%
Average	0.0002	-0.02	0.0398	11.47	-2.0E-04	-0.28	65%	6%	97%	9%

Table 2 presents a cross-section of results (averaged across time) for the regressions:

where ∆P_kare the 10-second mid-price changes and OFI_kare the contemporaneous order flow imbalances. These regressions were estimated using 273 half-hour subsamples (indexed by i) for each stock and their outputs were averaged across subsamples. Each subsample typically contains about 180 observations (indexed by k). The t-statistics were computed using White’s standard errors. For brevity, we report the R², the average ˆα_iand the average β^ˆ_ionly for the first regression (with a single OFI_kterm). There is almost no difference between averages of estimates β^ˆ_iand β^ˆ_Qiand the R²in two regressions. The last three columns report the percentage of samples where the coefficient(s) passed the z-test at the 5%

significance level.

Both regressions are estimated using ordinary least squares. The results are presented in Table 3: the quality of these fits convincingly demonstrates that the instantaneous price impact (measured by β^ˆ_i) is inversely related to market depth. There are three stocks with bad fits (namely APOL, AZO and CME) and we note that they also have wide spreads and low values of depth. It is possible that for these stocks other factors, such as the presence of hidden orders and depth beyond the best price levels the order book may dominate the instantaneous price impact. The intercept αˆ_L,iis highly statistically significant (being an estimate of parameter c) and α_M,iˆ , which is included to absorb the means, is mostly insignificant. Since the residuals of these regressions appear to be autocorrelated, the t-statistics and confidence intervals in Table 3 are computed with Newey-West standard errors. Coinciding with our intuition for (1), estimates λ^ˆare very close to 1 across stocks and the hypothesis {λ = 1} cannot be rejected for 35 out of 50 stocks. The restricted model (with λ = 1) also demonstrates a good quality of fit, making this a good approximation. However, the coefficient ˆc is generally different from in (1). Lower values of ˆc mean that mid-prices are (on average) more resilient to the incoming orders than indicated by AD_i(which is only a rough measure of market depth). In summary, λ = 1 appears to be a good approximation for most of the stocks and only the constant c needs to be calibrated to the data. The general case of regression (5) is illustrated on Figure 4 by a scatter plot for a representative stock.

Figure 4: Log-log scatter plot of the price impact coefficient estimate β^ˆ_iagainst average market depth AD_ifor the Schlumberger stock (SLB).

Table 3. Relation between the price impact coefficient and market depth.

Ticker	Parameter estimates	5% confidence intervals		Fit measures
cˆ	λˆ	t(cˆ)	t(λ^ˆ)	cˆ_l	cˆ_u	λˆ_l	λˆu	R2	corr[β,^ˆβ^ˆˆ]²	corr[β,ˆ βˆˆ∗]2
AMD	0.23	0.94	27.74	23.11	0.22	0.25	0.86	1.02	78%	86%	86%
APOL	0.27	0.36	4.43	1.05	0.15	0.39	-0.32	1.04	2%	30%	31%
AXP	0.14	0.83	13.95	26.48	0.12	0.16	0.77	0.89	84%	76%	76%
AZO	0.39	0.67	5.48	5.10	0.25	0.53	0.41	0.92	13%	17%	16%
BAC	0.27	0.96	25.27	19.74	0.25	0.29	0.90	1.03	76%	87%	87%
BDX	0.38	1.04	22.83	18.64	0.35	0.41	0.93	1.15	71%	68%	68%
BK	0.21	0.92	17.52	54.54	0.19	0.24	0.88	0.95	93%	91%	90%
BSX	0.35	0.98	14.98	24.55	0.31	0.40	0.90	1.05	73%	81%	81%
BTU	0.42	1.12	40.90	36.77	0.40	0.44	1.06	1.18	87%	83%	83%
CAT	0.29	0.96	21.70	16.87	0.27	0.32	0.85	1.07	87%	83%	83%
CB	0.32	1.02	27.08	49.61	0.30	0.34	0.98	1.06	92%	89%	89%
CCL	0.26	0.96	24.36	37.55	0.24	0.29	0.91	1.01	87%	83%	83%
CINF	0.31	0.97	20.05	47.39	0.28	0.34	0.93	1.01	92%	88%	88%
CME	1.27	0.50	2.55	1.99	0.29	2.24	0.01	0.99	2%	4%	3%
COH	0.37	1.05	15.29	36.65	0.32	0.43	0.98	1.12	77%	75%	75%
COP	0.13	0.80	8.52	15.95	0.10	0.16	0.70	0.89	75%	66%	66%
CVH	0.32	1.03	26.50	37.51	0.29	0.34	0.98	1.08	89%	89%	89%
DNR	0.23	0.96	32.44	40.90	0.22	0.24	0.92	1.01	91%	89%	89%
DVN	0.26	0.91	13.50	16.66	0.22	0.30	0.80	1.02	45%	56%	56%
EFX	0.30	0.99	20.16	26.13	0.27	0.33	0.92	1.07	84%	79%	79%
ETN	0.45	1.07	11.51	17.34	0.38	0.53	0.95	1.19	60%	56%	56%
FISV	0.34	1.01	23.35	30.70	0.31	0.36	0.94	1.07	84%	77%	77%
HAS	0.32	1.00	26.36	46.00	0.30	0.34	0.96	1.05	89%	83%	83%
HCP	0.19	0.89	22.93	51.27	0.17	0.21	0.86	0.93	94%	90%	90%
HOT	0.44	1.12	19.53	26.59	0.40	0.48	1.04	1.20	82%	80%	79%
KSS	0.39	1.05	24.40	33.17	0.36	0.42	0.99	1.11	85%	78%	78%
LLL	0.43	1.01	13.21	14.45	0.37	0.50	0.87	1.14	51%	58%	58%
LMT	0.50	1.14	7.31	13.49	0.37	0.64	0.98	1.31	60%	52%	52%
M	0.19	0.90	37.41	57.39	0.18	0.20	0.87	0.93	94%	92%	92%
MAR	0.28	0.98	22.58	50.20	0.25	0.30	0.94	1.02	92%	88%	88%
MFE	0.31	1.01	20.28	46.20	0.28	0.34	0.96	1.05	91%	86%	86%
MHP	0.27	0.94	19.60	33.62	0.24	0.30	0.89	1.00	82%	74%	74%
MHS	0.53	1.16	17.03	34.25	0.47	0.59	1.10	1.23	85%	81%	80%
MRK	0.13	0.81	18.07	32.20	0.11	0.14	0.76	0.86	87%	81%	81%
MRO	0.23	0.94	35.54	49.68	0.21	0.24	0.91	0.98	94%	93%	93%
MWV	0.32	1.05	28.07	37.81	0.30	0.34	1.00	1.10	90%	85%	85%
NEM	0.26	0.98	18.79	25.97	0.23	0.28	0.91	1.05	81%	77%	77%
OMC	0.30	0.96	29.47	17.76	0.28	0.32	0.85	1.06	83%	85%	85%
PCS	0.30	1.02	21.27	18.73	0.27	0.33	0.90	1.14	53%	82%	82%
PHM	0.28	0.98	36.43	35.12	0.26	0.29	0.93	1.04	86%	90%	90%
PKI	0.30	1.07	26.59	38.35	0.28	0.32	1.00	1.13	82%	88%	87%
R	0.37	1.02	18.51	15.76	0.33	0.41	0.90	1.15	57%	58%	58%
RAI	0.35	1.03	24.94	40.46	0.32	0.38	0.98	1.08	86%	76%	76%
SLB	0.35	1.06	18.98	40.60	0.31	0.38	1.01	1.12	91%	88%	88%
TE	0.21	1.00	16.18	24.28	0.18	0.24	0.92	1.09	70%	86%	86%
TWC	0.37	1.04	17.70	15.96	0.33	0.42	0.91	1.16	72%	79%	79%
WHR	0.78	1.18	9.24	11.54	0.61	0.94	0.98	1.38	44%	43%	42%
WIN	5.81	1.60	16.09	11.70	5.11	6.52	1.33	1.87	28%	71%	71%
WPI	0.27	0.92	19.33	28.99	0.24	0.30	0.86	0.98	78%	76%	76%
XTO	0.31	1.04	30.85	39.51	0.29	0.33	0.98	1.09	89%	91%	91%
Grand mean	0.45	0.98	20.74	29.53	0.38	0.52	0.88	1.08	74%	75%	75%

Table 3 presents the results of regressions:

where β^ˆ_iis the price impact coefficient for the i-th half-hour subsample and AD_iis the average market depth for that subsample. These regressions were estimated for each of the 50 stocks, using 273 estimates of β^ˆ_ifor that stock, obtained from (4). The second regression uses estimates λ^ˆobtained from the first regression. The t-statistics and the confidence intervals were computed using Newey-West standard errors. Confidence intervals are built with normal critical values. The last three columns provide three alternative fit measures – the R²of the linear regression (5), the squared correlation

3.3 Intraday patterns

The link that we established between the price impact and the market depth has an important implication. Since the market depth follows a predictable pattern of intraday seasonality ([1], [31]), the price impact coefficient must also have a predictable intraday pattern. To demonstrate it, we averaged β^ˆ_ifor each stock and each half-hour interval across days, resulting in the intraday seasonality pattern for that stock, normalized these values by the average β^ˆ_iof that stock and averaged the normalized seasonality patterns across stocks. The same procedure was repeated for AD_iand the results are shown on Figure 5.

Figure 5: Intraday patterns in the price impact coefficient β^ˆ_iand the average depth AD_i.

Near the market open, depth is two times lower than it is on average, indicating that the order book is relatively shallow. In a shallow market, the incoming orders can easily affect the mid-price and the price impact coefficient is two times higher near the market open than on average. Moreover, price impact is five times higher at the market open compared to the market close.

The intraday pattern in price impact can be used to explain the intraday patterns in price volatility, observed by many researchers ([1], [4], [21], [33]). Similarly to the price impact coefficient and the market depth, we computed the intraday patterns in variances of ∆P_kand OFI_k, using half-hour subsamples (indexed by i). Taking the variance on both sides of equation (2) demonstrates the link between var[∆P_k]_i, var[OFI_k]_iand β_i:

(7)

The average patterns are plotted on Figure 6. Notice that the price volatility has a sharp peak near the market open, while the volatility of order flow imbalance has a peak near the market close. This peak is, however, offset by a low price impact, which gradually declines throughout the day. For the i-th half-hour interval, the equation (7) implies that var[∆P_k]_i≈ β^ˆ_i²var[OFI_k]_iwhich is demonstrated on Figure 6^[4].

Figure 6: Intraday seasonality in variances var[∆P_k]_i, var[OFI_k]_i, the price impact coefficient β^ˆ_iand the expression β_i²var[OFI_k]_i.

The intraday pattern in price variance was explained by Madhavan et al. [33] in terms of a structural model. They argued that the volatility is higher in the morning because of the higher inflow of both public and private information. Similarly, Hasbrouck [21] argued that the peak of price volatility at market open is mostly due to higher intensity of public information. Both studies agree that the impact of trades is larger in the morning. Our model contributes to this discussion by explaining the peak of price volatility using tangible quantities, rather than unobservable parameters. We also argue that the price impact of trades and the information asymmetry may be, in fact, two sides of the same coin.

First, we associate the higher volatility of order flow imbalance at market the open and close with a higher rate of trading, that is, higher inflow of public and private information. Second, if the bid-ask spread is small (it is mostly equal to 1 cent in our data), limit order traders may avoid being “picked off” only by lowering the number of submitted orders, reducing the depth. Therefore, if limit order traders are aware of information asymmetry in the morning, the low depth may simply indicate this asymmetry. In our model, low depth also implies a higher price impact, making the information advantages harder to realize at the market open.

4. Price impact of trades

4.1 Trade imbalance vs order flow imbalance

The previous section discussed the linear relation between price changes and OFI_k– our measure of supply/demand imbalance. However, little has been said about trade imbalances, which are widely used in the academic literature [10, 20, 22, 24, 29, 38] and in practice [43]. The aim of this section is to compare the price impact of trades and order flow imbalance and show that the (nonlinear) price impact of trade volume may be derived from our linear model for the price impact of order flow.

For convenience we will call ‘buy trade’ a transaction initiated by a market buy order and ‘sell trade’ a transaction initiated by a market sell order. We define the trade imbalance during a time interval [t_k₋₁,t_k] as the difference between volumes of buy and sell trades during that interval:

N(t_k) N(t_k)

TIk = X bn − X sn,

n=N(t_k−₁)+1 n=N(t_k−₁)+1

Here, b_nis the size of a buyer-initiated trade that occurs at the n-th quote; b_n= 0 if no buy trade occurs at that quote. Similarly, s_nis the size of a sell trade that occurs at the n-th quote or zero. The procedure that matches trades with quotes and classifies them as buys or sells is described in the Appendix.

To compare the explanatory power of trade and order flow imbalances with respect to price changes, we perform the following regressions

The regressions are estimated separately for every half-hour subsample of data (indexed by i). If the effect of trades is included in the order flow imbalance, the coefficients θ^ˆ_T,iin (8c) must be indistinguishable from zero. We note that regressions (8a-8c) contain only the linear terms, because we found no evidence of non-linear price impacts in our data (for neither OFI_knor TI_k). The average results of these regressions are presented in Panel A of Table 4. Clearly, when OFI_kand TI_kare taken individually, each of them has a statistically significant influence on price changes. Comparing the two we observe that OFI_kexplains price changes better than TI_k– the average R²for order flow imbalance is 65% compared to 32% for the trade imbalance. When two variables are used together to explain price changes, the dependence on trade imbalance becomes questionable. The average t-statistic of TI_kdecreases by a factor of four and the coefficients θ^ˆ_T,iare statistically significant in only 31% of subsamples. However, the dependence on OFI_kremains convincingly strong. Our findings show that:

The order flow imbalance OFI_kexplains price movements better than the imbalance of trades.
The effect of trade imbalance is adequately included in OFI_k, a more general measure of supply/demand imbalance.

Table 4. Comparison of order flow imbalance and trade imbalance.

Shape

Description automatically generated with medium confidence

Table 4 presents the average results of regressions:

where ∆P_kare the 10-second mid-price changes (Panel A) or changes in trade prices between L trades (Panel B), OFI_kare the contemporaneous order flow imbalances and TI_kare the contemporaneous trade imbalances. For Panel A, these regressions were estimated using 273 half-hour subsamples (indexed by i) for each stock and their outputs were averaged across subsamples. Each subsample typically contains about 180 observations (indexed by k). For Panel B, data was pooled across half-hour subsamples, resulting in 13 subsamples for each stock.

The t-statistics were computed using White’s standard errors. For each of three regressions, Table 4 reports the average R², the average tstatistic of the coefficient(s), the percentage of samples where the coefficient(s) passed the z-test at the 5% significance level and the F-statistic of the regression. The outputs for Panel B were averaged across stocks.

Interestingly, we found that the relation between ∆_LP_k^tand OFI_kis concave in some samples, and similarly for ∆_LP_k^tand TI_k. We estimated regressions (8a) and (8b) for transaction price changes with additional quadratic terms OFI_k|OFI_k| (respectively, TI_k|TI_k|) and found that they are significant in nearly half of the samples with t-statistics of -2.8 on average (-2.3 for TI_k|TI_k|). Sampling data at special times (trade times) may introduce biases to the right side of the regression. One possible explanation is that traders submit their orders when they expect their impact to be minimal, leading to a concave (sublinear) impact. Supporting this idea of sampling biases, we found that when mid-prices are sampled at trade times, the price impact of OFI_kis again concave in some samples. On another hand, when we regressed last trade prices sampled at 1-minute frequency on OFI_k, we observed the concave price impact once again. This suggests that using either trade times or trade prices may lead to non-linear price impact. However the quadratic term in our regressions is insignificant in about half of the samples and marginally significant in the the other half of the data.

The relation between price changes and volume is empirically confirmed by many authors (see [27] for a review). Recently, traded volume became an important metric for order execution algorithms – these algorithms often attempt to match a certain percentage of the total traded volume to reduce the price impact. However, it remains unclear whether the traded volume truly determines the magnitude of price moves and whether it is a good metric for price impact. Casting doubt on this assertion, Jones et al. [26] showed that the relation between the daily volatility and the daily volume is essentially due to the number of trades and not the volume per se (also see [?] for the discussion).

We extend this result in two ways. First, we show that even when prices are driven by order flow imbalance, an apparent (concave) dependence on traded volume may emerge as an artifact due to data aggregation. Second, we empirically confirm that the price-volume relation is an indirect one – it becomes statistically insignificant after accounting for the order flow imbalance.

5. Conclusion

We have introduced order flow imbalance, a variable that cumulates the sizes of order book events, treating the contributions of market, limit and cancel orders equally, and provided empirical and theoretical evidence for a linear relation between high-frequency price changes and order flow imbalance for individual stocks. We have shown that this linear model is robust across stocks and the impact coefficient is inversely proportional to market depth. These relations suggest that prices respond to changes in the supply and demand for shares at the best quotes, and that the impact coefficient fluctuates with the amount of liquidity provision, or depth, in the market. Moreover, we have demonstrated that order flow imbalance is a stronger driver of high-frequency price changes than standard measures of trade imbalance. Trades seem to carry little to no information about price changes after the simultaneous order flow imbalance is taken into account. If trades do not help to explain price changes after controlling for the order flow imbalance, it is highly possible that the relation between price changes and traded volume simply capture the noisy scaling relation between these variables.

Overall, these findings seem to give an intuitive picture of the price impact of order book events, which is somewhat simpler than the one conveyed by previous studies.

Table 5. Comparison of traded volume and order flow imbalance.

Ticker	Avg Hˆ	Stdev Hˆ	Order flow imbalance			Traded volume				Both covariates
R2	t(β^ˆ_O)	β_O6= 0	F	R2	t(β^ˆ_V)	β_V6= 0	F	R2	t(φ^ˆ_O)	t(φ^ˆ_V)	φ_O6= 0	φ_V6= 0	F
AMD	0.06	0.08	63%	10.3	99%	356	14%	4.5	83%	34	63%	9.4	1.1	99%	35%	182
APOL	0.24	0.08	53%	8.3	90%	258	25%	6.8	99%	63	57%	6.9	2.9	89%	84%	144
AXP	0.16	0.08	55%	10.5	100%	249	20%	6.6	100%	48	57%	9.0	2.8	100%	81%	133
AZO	0.43	0.22	39%	5.5	96%	131	32%	5.3	100%	93	50%	4.3	3.6	94%	96%	98
BAC	0.09	0.08	73%	16.3	100%	560	24%	5.6	83%	61	74%	13.9	1.2	96%	35%	285
BDX	0.26	0.10	55%	8.4	99%	261	27%	6.3	100%	71	58%	6.7	2.9	98%	84%	147
BK	0.11	0.07	68%	13.1	100%	437	19%	6.6	97%	46	68%	11.5	2.0	99%	58%	225
BSX	-0.17	2.41	68%	8.4	100%	486	14%	3.3	95%	33	69%	8.0	0.1	97%	12%	246
BTU	0.24	0.07	58%	10.5	99%	283	23%	6.8	99%	57	60%	8.9	2.4	99%	78%	151
CAT	0.22	0.07	56%	10.4	98%	250	19%	6.0	98%	44	57%	8.9	2.1	98%	63%	131
CB	0.19	0.09	56%	10.1	99%	261	23%	6.4	99%	58	58%	8.2	2.6	99%	74%	141
CCL	0.14	0.07	60%	11.3	100%	309	19%	6.6	99%	45	62%	9.9	2.4	99%	74%	162
CINF	0.13	0.12	67%	10.6	99%	505	30%	6.1	98%	85	69%	8.7	2.0	99%	55%	268
CME	0.49	0.24	28%	4.1	94%	78	30%	4.8	99%	83	42%	3.2	3.6	86%	94%	71
COH	0.19	0.07	60%	10.4	99%	299	22%	6.5	99%	52	61%	8.9	2.2	98%	69%	157
COP	0.16	0.07	56%	9.8	100%	277	20%	6.0	96%	49	58%	8.4	2.4	100%	70%	145
CVH	0.18	0.10	62%	10.2	100%	352	27%	5.9	99%	72	64%	8.2	2.2	100%	70%	189
DNR	0.08	0.07	64%	12.0	99%	376	17%	6.3	95%	38	65%	10.7	1.8	99%	55%	193
DVN	0.26	0.07	52%	8.6	93%	236	24%	6.7	100%	59	55%	7.1	2.9	91%	81%	131
EFX	0.20	0.11	52%	8.1	99%	241	26%	5.4	99%	69	56%	6.4	2.7	97%	75%	137
ETN	0.26	0.10	55%	8.2	97%	252	27%	6.4	99%	70	58%	6.8	2.9	96%	83%	142
FISV	0.19	0.11	57%	9.1	100%	284	25%	5.9	100%	65	59%	7.3	2.2	99%	66%	153
HAS	0.20	0.09	61%	10.1	100%	328	26%	6.2	100%	67	63%	8.2	2.3	100%	73%	175
HCP	0.14	0.07	57%	11.1	100%	268	21%	7.0	99%	50	59%	9.3	2.7	100%	79%	143
HOT	0.23	0.08	57%	9.7	98%	263	24%	6.9	100%	60	60%	8.2	3.0	98%	85%	145
KSS	0.24	0.08	60%	10.8	97%	318	25%	6.6	99%	61	62%	9.0	2.4	97%	74%	169
LLL	0.33	0.12	58%	9.4	94%	323	34%	6.9	100%	101	63%	7.1	3.0	91%	86%	188
LMT	0.28	0.09	61%	10.7	99%	327	31%	7.3	100%	85	64%	8.4	2.9	99%	84%	182
M	0.11	0.07	69%	13.9	100%	463	20%	6.3	99%	46	69%	12.2	2.0	100%	60%	238
MAR	0.15	0.07	61%	12.3	100%	324	21%	6.9	99%	50	62%	10.4	2.4	100%	71%	170
MFE	0.16	0.09	60%	10.9	99%	318	24%	7.1	98%	62	62%	8.8	2.5	99%	71%	170
MHP	0.20	0.10	62%	10.2	99%	377	25%	5.9	99%	62	64%	8.5	1.9	99%	55%	199
MHS	0.23	0.08	56%	9.2	99%	258	24%	6.6	100%	58	58%	7.7	2.6	98%	77%	139
MRK	0.10	0.07	62%	11.0	100%	330	17%	5.4	99%	40	63%	9.8	1.8	100%	55%	170
MRO	0.09	0.06	61%	11.8	100%	333	16%	6.3	95%	36	63%	10.6	2.0	100%	54%	172
MWV	0.18	0.10	62%	10.3	100%	330	28%	6.7	100%	75	64%	8.2	2.4	100%	74%	180
NEM	0.20	0.07	56%	9.9	99%	253	20%	6.1	99%	47	58%	8.6	2.5	99%	75%	135
OMC	0.15	0.09	57%	10.1	99%	286	20%	6.4	98%	48	59%	8.6	2.4	98%	73%	151
PCS	0.11	0.18	62%	7.1	96%	411	18%	3.7	97%	54	63%	6.5	0.7	93%	20%	214
PHM	0.07	0.08	64%	10.2	100%	384	15%	5.5	90%	34	65%	9.4	1.2	99%	40%	195
PKI	0.11	0.11	55%	7.8	99%	266	20%	4.8	97%	47	57%	6.7	1.8	98%	53%	141
R	0.27	0.11	56%	8.6	98%	259	28%	6.0	100%	74	59%	6.9	2.9	97%	85%	147
RAI	0.25	0.10	61%	9.2	99%	334	28%	5.7	100%	73	63%	7.6	2.4	99%	71%	182
SLB	0.24	0.07	62%	12.0	99%	330	19%	5.5	98%	46	63%	10.6	1.7	99%	51%	171
TE	0.09	1.69	60%	8.0	98%	371	18%	4.4	85%	48	61%	7.2	1.3	98%	39%	196
TWC	0.25	0.10	55%	9.7	99%	253	27%	6.6	100%	73	58%	7.6	3.0	99%	81%	142
WHR	0.34	0.11	56%	8.2	97%	272	29%	6.3	100%	78	59%	6.6	2.9	95%	86%	156
WIN	0.06	0.26	48%	3.9	79%	340	10%	2.8	50%	34	49%	3.7	0.6	79%	29%	179
WPI	0.22	0.10	61%	9.6	98%	361	28%	5.8	100%	75	64%	7.7	2.2	98%	71%	196
XTO	0.08	0.06	53%	10.9	100%	238	15%	6.5	100%	32	55%	9.6	2.7	100%	78%	125
Grand mean	0.18	0.18	58%	9.8	98%	313	23%	6.0	97%	58	61%	8.3	2.3	97%	67%	168

Table 5 presents the average results of regressions:

, ,

where ∆P_kare the 10-second mid-price changes, OFI_kare the contemporaneous order flow imbalances and V OL_kare the contemporaneous trade volumes. The exponents Hˆ_iwere estimated in each subsample beforehand using a logarithmic regression: log|∆P_k| = logθ^ˆ¯_i+ H^ˆ_ilogV OL_k+ logξ^ˆ_k. These regressions were estimated using 273 half-hour subsamples (indexed by i) for each stock and their outputs were averaged across subsamples. Each subsample typically contains about 180 observations (indexed by k). The t-statistics were computed using White’s standard errors. For each of three regressions, Table 5 reports the average R², the average t-statistic of the coefficient(s), the percentage of samples where the coefficient(s) passed the z-sest at the 5% significance level and the F-statistic of the regression.

SOLUTION:

{

“cells”: [

{

“cell_type”: “markdown”,

“source”: [

“<h1 style=\”color:purple\”>Author</h1>\n”,

“\n”,

” \n”,

“- Name: \n”,

“- Student ID: \n”

“metadata”: {

“id”: “jgavCdIkFqci”

}

{

“cell_type”: “markdown”,

“source”: [

“## Introduction”

“metadata”: {

“id”: “9_lbTOJHFvti”

}

{

“cell_type”: “markdown”,

“source”: [

“Many studies have been conducted to describe the relationship between the trade activity(volume) and the price change. Ask (Sell) and Bid (Buy) orders on the order book might signal the direction of market movement. To model the possible pattern between the order imbalance (between ask and bid sides) and price change, we will build new predictors and check the performance of multiple linear regression in paper trading (backtesting with historical data). \n”,

“\n”

“metadata”: {

“id”: “zxpfm7a9Ft78”

}

{

“cell_type”: “markdown”,

“source”: [

“## Data”

“metadata”: {

“id”: “w1lbOpn_FzaX”

}

{

“cell_type”: “code”,

“execution_count”: 1,

“metadata”: {

“id”: “Tjr4Fzs5FhHI”

“outputs”: [],

“source”: [

“import pandas as pd\n”,

“import numpy as np\n”,

“import os, math, pathlib\n”,

“from os import listdir\n”,

“from multiprocessing import Process\n”,

“import warnings\n”,

“warnings.filterwarnings(‘ignore’)”

]

{

“cell_type”: “code”,

“execution_count”: 2,

“metadata”: {

“id”: “4apEYH9BFhHL”

“outputs”: [],

“source”: [

“df=pd.read_csv(‘/content/Data.csv’)”

]

{

“cell_type”: “code”,

“execution_count”: 3,

“metadata”: {

“colab”: {

“base_uri”: “https://localhost:8080/”,

“height”: 206

“id”: “Mszj398SFhHL”,

“outputId”: “793f5c92-3e01-47e3-f40a-93d93c974040”

“outputs”: [

{

“output_type”: “execute_result”,

“data”: {

“text/plain”: [

” Unnamed: 0 AskPrice BidPrice AskVolume BidVolume\n”,

“0 2020-12-21 09:00:00.500 4445.0 4419.0 467.0 87.0\n”,

“1 2020-12-21 09:00:00.500 4445.0 4419.0 467.0 87.0\n”,

“2 2020-12-21 09:00:00.500 4445.0 4419.0 467.0 87.0\n”,

“3 2020-12-21 09:00:00.500 4445.0 4419.0 467.0 87.0\n”,

“4 2020-12-21 09:00:00.500 4445.0 4419.0 467.0 87.0”

“text/html”: [

“\n”,

” <div id=\”df-0e5ae9fa-2a73-45a5-9a3d-722fbe8b47df\”>\n”,

” <div class=\”colab-df-container\”>\n”,

” <div>\n”,

“<style scoped>\n”,

” .dataframe tbody tr th:only-of-type {\n”,

” vertical-align: middle;\n”,

” }\n”,

“\n”,

” .dataframe tbody tr th {\n”,

” vertical-align: top;\n”,

” }\n”,

“\n”,

” .dataframe thead th {\n”,

” text-align: right;\n”,

” }\n”,

“</style>\n”,

“<table border=\”1\” class=\”dataframe\”>\n”,

” <thead>\n”,

” <tr style=\”text-align: right;\”>\n”,

” <th></th>\n”,

” <th>Unnamed: 0</th>\n”,

” <th>AskPrice</th>\n”,

” <th>BidPrice</th>\n”,

” <th>AskVolume</th>\n”,

” <th>BidVolume</th>\n”,

” </tr>\n”,

” </thead>\n”,

” <tbody>\n”,

” <tr>\n”,

” <th>0</th>\n”,

” <td>2020-12-21 09:00:00.500</td>\n”,

” <td>4445.0</td>\n”,

” <td>4419.0</td>\n”,

” <td>467.0</td>\n”,

” <td>87.0</td>\n”,

” </tr>\n”,

” <tr>\n”,

” <th>1</th>\n”,

” <td>2020-12-21 09:00:00.500</td>\n”,

” <td>4445.0</td>\n”,

” <td>4419.0</td>\n”,

” <td>467.0</td>\n”,

” <td>87.0</td>\n”,

” </tr>\n”,

” <tr>\n”,

” <th>2</th>\n”,

” <td>2020-12-21 09:00:00.500</td>\n”,

” <td>4445.0</td>\n”,

” <td>4419.0</td>\n”,

” <td>467.0</td>\n”,

” <td>87.0</td>\n”,

” </tr>\n”,

” <tr>\n”,

” <th>3</th>\n”,

” <td>2020-12-21 09:00:00.500</td>\n”,

” <td>4445.0</td>\n”,

” <td>4419.0</td>\n”,

” <td>467.0</td>\n”,

” <td>87.0</td>\n”,

” </tr>\n”,

” <tr>\n”,

” <th>4</th>\n”,

” <td>2020-12-21 09:00:00.500</td>\n”,

” <td>4445.0</td>\n”,

” <td>4419.0</td>\n”,

” <td>467.0</td>\n”,

” <td>87.0</td>\n”,

” </tr>\n”,

” </tbody>\n”,

“</table>\n”,

“</div>\n”,

” <button class=\”colab-df-convert\” onclick=\”convertToInteractive(‘df-0e5ae9fa-2a73-45a5-9a3d-722fbe8b47df’)\”\n”,

” title=\”Convert this dataframe to an interactive table.\”\n”,

” style=\”display:none;\”>\n”,

” \n”,

” <svg xmlns=\”http://www.w3.org/2000/svg\” height=\”24px\”viewBox=\”0 0 24 24\”\n”,

” width=\”24px\”>\n”,

” <path d=\”M0 0h24v24H0V0z\” fill=\”none\”/>\n”,

” <path d=\”M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\”/><path d=\”M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\”/>\n”,

” </svg>\n”,

” </button>\n”,

” \n”,

” <style>\n”,

” .colab-df-container {\n”,

” display:flex;\n”,

” flex-wrap:wrap;\n”,

” gap: 12px;\n”,

” }\n”,

“\n”,

” .colab-df-convert {\n”,

” background-color: #E8F0FE;\n”,

” border: none;\n”,

” border-radius: 50%;\n”,

” cursor: pointer;\n”,

” display: none;\n”,

” fill: #1967D2;\n”,

” height: 32px;\n”,

” padding: 0 0 0 0;\n”,

” width: 32px;\n”,

” }\n”,

“\n”,

” .colab-df-convert:hover {\n”,

” background-color: #E2EBFA;\n”,

” box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n”,

” fill: #174EA6;\n”,

” }\n”,

“\n”,

” [theme=dark] .colab-df-convert {\n”,

” background-color: #3B4455;\n”,

” fill: #D2E3FC;\n”,

” }\n”,

“\n”,

” [theme=dark] .colab-df-convert:hover {\n”,

” background-color: #434B5C;\n”,

” box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n”,

” filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n”,

” fill: #FFFFFF;\n”,

” }\n”,

” </style>\n”,

“\n”,

” <script>\n”,

” const buttonEl =\n”,

” document.querySelector(‘#df-0e5ae9fa-2a73-45a5-9a3d-722fbe8b47df button.colab-df-convert’);\n”,

” buttonEl.style.display =\n”,

” google.colab.kernel.accessAllowed ? ‘block’ : ‘none’;\n”,

“\n”,

” async function convertToInteractive(key) {\n”,

” const element = document.querySelector(‘#df-0e5ae9fa-2a73-45a5-9a3d-722fbe8b47df’);\n”,

” const dataTable =\n”,

” await google.colab.kernel.invokeFunction(‘convertToInteractive’,\n”,

” [key], {});\n”,

” if (!dataTable) return;\n”,

“\n”,

” const docLinkHtml = ‘Like what you see? Visit the ‘ +\n”,

” ‘<a target=\”_blank\” href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>’\n”,

” + ‘ to learn more about interactive tables.’;\n”,

” element.innerHTML = ”;\n”,

” dataTable[‘output_type’] = ‘display_data’;\n”,

” await google.colab.output.renderOutput(dataTable, element);\n”,

” const docLink = document.createElement(‘div’);\n”,

” docLink.innerHTML = docLinkHtml;\n”,

” element.appendChild(docLink);\n”,

” }\n”,

” </script>\n”,

” </div>\n”,

” “

]

“metadata”: {},

“execution_count”: 3

}

“source”: [

“df.head()”

]

{

“cell_type”: “code”,

“execution_count”: 4,

“metadata”: {

“colab”: {

“base_uri”: “https://localhost:8080/”,

“height”: 206

“id”: “VvczMB-kFhHN”,

“outputId”: “ccc3d0b8-9205-4cfa-d308-8720cf5cfb02”

“outputs”: [

{

“output_type”: “execute_result”,

“data”: {

“text/plain”: [

” Unnamed: 0 AskPrice BidPrice AskVolume BidVolume\n”,

“3805830 2021-01-05 14:59:58.000 4398.0 4397.0 120.0 201.0\n”,

“3805831 2021-01-05 14:59:58.500 4398.0 4397.0 92.0 164.0\n”,

“3805832 2021-01-05 14:59:59.000 4398.0 4397.0 84.0 35.0\n”,

“3805833 2021-01-05 14:59:59.500 4398.0 4397.0 47.0 46.0\n”,

“3805834 2021-01-05 15:00:00.000 4398.0 4397.0 84.0 30.0”

“text/html”: [

“\n”,

” <div id=\”df-79832f5e-73cd-4498-8ec4-7eeeb4f1038e\”>\n”,

” <div class=\”colab-df-container\”>\n”,

” <div>\n”,

“<style scoped>\n”,

” .dataframe tbody tr th:only-of-type {\n”,

” vertical-align: middle;\n”,

” }\n”,

“\n”,

” .dataframe tbody tr th {\n”,

” vertical-align: top;\n”,

” }\n”,

“\n”,

” .dataframe thead th {\n”,

” text-align: right;\n”,

” }\n”,

“</style>\n”,

“<table border=\”1\” class=\”dataframe\”>\n”,

” <thead>\n”,

” <tr style=\”text-align: right;\”>\n”,

” <th></th>\n”,

” <th>Unnamed: 0</th>\n”,

” <th>AskPrice</th>\n”,

” <th>BidPrice</th>\n”,

” <th>AskVolume</th>\n”,

” <th>BidVolume</th>\n”,

” </tr>\n”,

” </thead>\n”,

” <tbody>\n”,

” <tr>\n”,

” <th>3805830</th>\n”,

” <td>2021-01-05 14:59:58.000</td>\n”,

” <td>4398.0</td>\n”,

” <td>4397.0</td>\n”,

” <td>120.0</td>\n”,

” <td>201.0</td>\n”,

” </tr>\n”,

” <tr>\n”,

” <th>3805831</th>\n”,

” <td>2021-01-05 14:59:58.500</td>\n”,

” <td>4398.0</td>\n”,

” <td>4397.0</td>\n”,

” <td>92.0</td>\n”,

” <td>164.0</td>\n”,

” </tr>\n”,

” <tr>\n”,

” <th>3805832</th>\n”,

” <td>2021-01-05 14:59:59.000</td>\n”,

” <td>4398.0</td>\n”,

” <td>4397.0</td>\n”,

” <td>84.0</td>\n”,

” <td>35.0</td>\n”,

” </tr>\n”,

” <tr>\n”,

” <th>3805833</th>\n”,

” <td>2021-01-05 14:59:59.500</td>\n”,

” <td>4398.0</td>\n”,

” <td>4397.0</td>\n”,

” <td>47.0</td>\n”,

” <td>46.0</td>\n”,

” </tr>\n”,

” <tr>\n”,

” <th>3805834</th>\n”,

” <td>2021-01-05 15:00:00.000</td>\n”,

” <td>4398.0</td>\n”,

” <td>4397.0</td>\n”,

” <td>84.0</td>\n”,

” <td>30.0</td>\n”,

” </tr>\n”,

” </tbody>\n”,

“</table>\n”,

“</div>\n”,

” <button class=\”colab-df-convert\” onclick=\”convertToInteractive(‘df-79832f5e-73cd-4498-8ec4-7eeeb4f1038e’)\”\n”,

” title=\”Convert this dataframe to an interactive table.\”\n”,

” style=\”display:none;\”>\n”,

” \n”,

” <svg xmlns=\”http://www.w3.org/2000/svg\” height=\”24px\”viewBox=\”0 0 24 24\”\n”,

” width=\”24px\”>\n”,

” <path d=\”M0 0h24v24H0V0z\” fill=\”none\”/>\n”,

” </svg>\n”,

” </button>\n”,

” \n”,

” <style>\n”,

” .colab-df-container {\n”,

” display:flex;\n”,

” flex-wrap:wrap;\n”,

” gap: 12px;\n”,

” }\n”,

“\n”,

” .colab-df-convert {\n”,

” background-color: #E8F0FE;\n”,

” border: none;\n”,

” border-radius: 50%;\n”,

” cursor: pointer;\n”,

” display: none;\n”,

” fill: #1967D2;\n”,

” height: 32px;\n”,

” padding: 0 0 0 0;\n”,

” width: 32px;\n”,

” }\n”,

“\n”,

” .colab-df-convert:hover {\n”,

” background-color: #E2EBFA;\n”,

” box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n”,

” fill: #174EA6;\n”,

” }\n”,

“\n”,

” [theme=dark] .colab-df-convert {\n”,

” background-color: #3B4455;\n”,

” fill: #D2E3FC;\n”,

” }\n”,

“\n”,

” [theme=dark] .colab-df-convert:hover {\n”,

” background-color: #434B5C;\n”,

” box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n”,

” filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n”,

” fill: #FFFFFF;\n”,

” }\n”,

” </style>\n”,

“\n”,

” <script>\n”,

” const buttonEl =\n”,

” document.querySelector(‘#df-79832f5e-73cd-4498-8ec4-7eeeb4f1038e button.colab-df-convert’);\n”,

” buttonEl.style.display =\n”,

” google.colab.kernel.accessAllowed ? ‘block’ : ‘none’;\n”,

“\n”,

” async function convertToInteractive(key) {\n”,

” const element = document.querySelector(‘#df-79832f5e-73cd-4498-8ec4-7eeeb4f1038e’);\n”,

” const dataTable =\n”,

” await google.colab.kernel.invokeFunction(‘convertToInteractive’,\n”,

” [key], {});\n”,

” if (!dataTable) return;\n”,

“\n”,

” const docLinkHtml = ‘Like what you see? Visit the ‘ +\n”,

” ‘<a target=\”_blank\” href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>’\n”,

” + ‘ to learn more about interactive tables.’;\n”,

” element.innerHTML = ”;\n”,

” dataTable[‘output_type’] = ‘display_data’;\n”,

” await google.colab.output.renderOutput(dataTable, element);\n”,

” const docLink = document.createElement(‘div’);\n”,

” docLink.innerHTML = docLinkHtml;\n”,

” element.appendChild(docLink);\n”,

” }\n”,

” </script>\n”,

” </div>\n”,

” “

]

“metadata”: {},

“execution_count”: 4

}

“source”: [

“df.tail()”

]

{

“cell_type”: “code”,

“execution_count”: 6,

“metadata”: {

“id”: “O_zMcI0UFhHO”

“outputs”: [],

“source”: [

“df1=df.reset_index()[‘BidVolume’]”

]

{

“cell_type”: “code”,

“execution_count”: 7,

“metadata”: {

“colab”: {

“base_uri”: “https://localhost:8080/”

“id”: “b1RYWva7FhHO”,

“outputId”: “01a8133a-4731-475a-8248-eb98de9a4e2a”

“outputs”: [

{

“output_type”: “execute_result”,

“data”: {

“text/plain”: [

“0 87.0\n”,

“1 87.0\n”,

“2 87.0\n”,

“3 87.0\n”,

“4 87.0\n”,

” … \n”,

“3805830 201.0\n”,

“3805831 164.0\n”,

“3805832 35.0\n”,

“3805833 46.0\n”,

“3805834 30.0\n”,

“Name: BidVolume, Length: 3805835, dtype: float64”

]

“metadata”: {},

“execution_count”: 7

}

“source”: [

“df1”

]

{

“cell_type”: “code”,

“execution_count”: 8,

“metadata”: {

“colab”: {

“base_uri”: “https://localhost:8080/”,

“height”: 295

“id”: “lzR6i915FhHP”,

“outputId”: “c8e8efbb-cc08-4ae2-dc24-b63cfd6b4498”

“outputs”: [

{

“output_type”: “execute_result”,

“data”: {

“text/plain”: [

“[<matplotlib.lines.Line2D at 0x7fd552aa5110>]”

]

“metadata”: {},

“execution_count”: 8

{

“output_type”: “display_data”,

“data”: {

“text/plain”: [

“<Figure size 432×288 with 1 Axes>”

“image/png”: “”

“metadata”: {

“needs_background”: “light”

}

“source”: [

“import matplotlib.pyplot as plt\n”,

“plt.plot(df1)”

]

{

“cell_type”: “code”,

“execution_count”: 9,

“metadata”: {

“id”: “J04-nL5FFhHP”

“outputs”: [],

“source”: [

“### LSTM are sensitive to the scale of the data. so we apply MinMax scaler “

]

{

“cell_type”: “code”,

“execution_count”: 10,

“metadata”: {

“id”: “doC4t0lpFhHQ”

“outputs”: [],

“source”: [

“import numpy as np”

]

{

“cell_type”: “code”,

“execution_count”: 11,

“metadata”: {

“colab”: {

“base_uri”: “https://localhost:8080/”

“id”: “ask8cTjjFhHQ”,

“outputId”: “d352140b-6600-47c2-8501-fa988be019d8”

“outputs”: [

{

“output_type”: “execute_result”,

“data”: {

“text/plain”: [

“0 87.0\n”,

“1 87.0\n”,

“2 87.0\n”,

“3 87.0\n”,

“4 87.0\n”,

” … \n”,

“3805830 201.0\n”,

“3805831 164.0\n”,

“3805832 35.0\n”,

“3805833 46.0\n”,

“3805834 30.0\n”,

“Name: BidVolume, Length: 3805835, dtype: float64”

]

“metadata”: {},

“execution_count”: 11

}

“source”: [

“df1”

]

{

“cell_type”: “code”,

“execution_count”: 12,

“metadata”: {

“id”: “TLTg9wdVFhHQ”

“outputs”: [],

“source”: [

“from sklearn.preprocessing import MinMaxScaler\n”,

“scaler=MinMaxScaler(feature_range=(0,1))\n”,

“df1=scaler.fit_transform(np.array(df1).reshape(-1,1))”

]

{

“cell_type”: “code”,

“execution_count”: 13,

“metadata”: {

“colab”: {

“base_uri”: “https://localhost:8080/”

“id”: “AVv0VpaeFhHR”,

“outputId”: “b27b64c2-5276-49ab-f797-58d5bcadef4f”

“outputs”: [

{

“output_type”: “stream”,

“name”: “stdout”,

“text”: [

“[[0.0018373 ]\n”,

” [0.0018373 ]\n”,

” …\n”,

” [0.00073915]\n”,

” [0.00097145]\n”,

” [0.00063355]]\n”

]

}

“source”: [

“print(df1)”

]

{

“cell_type”: “code”,

“execution_count”: 14,

“metadata”: {

“id”: “D-ipDoBSFhHR”

“outputs”: [],

“source”: [

“##splitting dataset into train and test split\n”,

“training_size=int(len(df1)*0.65)\n”,

“test_size=len(df1)-training_size\n”,

“train_data,test_data=df1[0:training_size,:],df1[training_size:len(df1),:1]”

]

{

“cell_type”: “code”,

“execution_count”: 15,

“metadata”: {

“colab”: {

“base_uri”: “https://localhost:8080/”

“id”: “f5LrV5sLFhHR”,

“outputId”: “02460f05-4492-45cd-d941-139594b6b9c8”

“outputs”: [

{

“output_type”: “execute_result”,

“data”: {

“text/plain”: [

“(2473792, 1332043)”

]

“metadata”: {},

“execution_count”: 15

}

“source”: [

“training_size,test_size”

]

{

“cell_type”: “code”,

“execution_count”: 16,

“metadata”: {

“colab”: {

“base_uri”: “https://localhost:8080/”

“id”: “2iRbi6JKFhHS”,

“outputId”: “b33be162-db61-4772-ef47-a38c2da6a17b”

“outputs”: [

{

“output_type”: “execute_result”,

“data”: {

“text/plain”: [

“array([[0.0018373 ],\n”,

” [0.0018373 ],\n”,

” …,\n”,

” [0.00069691],\n”,

” [0.00069691]])”

]

“metadata”: {},

“execution_count”: 16

}

“source”: [

“train_data”

]

{

“cell_type”: “code”,

“execution_count”: 17,

“metadata”: {

“id”: “J8JBYEXpFhHS”

“outputs”: [],

“source”: [

“import numpy\n”,

“# convert an array of values into a dataset matrix\n”,

“def create_dataset(dataset, time_step=1):\n”,

“\tdataX, dataY = [], []\n”,

“\tfor i in range(len(dataset)-time_step-1):\n”,

“\t\ta = dataset[i:(i+time_step), 0] ###i=0, 0,1,2,3—–99 100 \n”,

“\t\tdataX.append(a)\n”,

“\t\tdataY.append(dataset[i + time_step, 0])\n”,

“\treturn numpy.array(dataX), numpy.array(dataY)”

]

{

“cell_type”: “code”,

“execution_count”: 18,

“metadata”: {

“id”: “ZjQtYk8BFhHT”

“outputs”: [],

“source”: [

“# reshape into X=t,t+1,t+2,t+3 and Y=t+4\n”,

“time_step = 100\n”,

“X_train, y_train = create_dataset(train_data, time_step)\n”,

“X_test, ytest = create_dataset(test_data, time_step)”

]

{

“cell_type”: “code”,

“execution_count”: 19,

“metadata”: {

“colab”: {

“base_uri”: “https://localhost:8080/”

“id”: “L1Swq4icFhHT”,

“outputId”: “9c9ebf25-0c4e-4bff-81a4-d8cd399d15ff”

“outputs”: [

{

“output_type”: “stream”,

“name”: “stdout”,

“text”: [

“(2473691, 100)\n”,

“(2473691,)\n”

]

{

“output_type”: “execute_result”,

“data”: {

“text/plain”: [

“(None, None)”

]

“metadata”: {},

“execution_count”: 19

}

“source”: [

“print(X_train.shape), print(y_train.shape)”

]

{

“cell_type”: “code”,

“execution_count”: 20,

“metadata”: {

“colab”: {

“base_uri”: “https://localhost:8080/”

“id”: “ZswVkPi8FhHT”,

“outputId”: “8a62c21e-f0d6-4522-ab13-85d8108b1660”

“outputs”: [

{

“output_type”: “stream”,

“name”: “stdout”,

“text”: [

“(1331942, 100)\n”,

“(1331942,)\n”

]

{

“output_type”: “execute_result”,

“data”: {

“text/plain”: [

“(None, None)”

]

“metadata”: {},

“execution_count”: 20

}

“source”: [

“print(X_test.shape), print(ytest.shape)”

]

{

“cell_type”: “code”,

“execution_count”: 21,

“metadata”: {

“id”: “U6UM-8xqFhHU”

“outputs”: [],

“source”: [

“# reshape input to be [samples, time steps, features] which is required for LSTM\n”,

“X_train =X_train.reshape(X_train.shape[0],X_train.shape[1] , 1)\n”,

“X_test = X_test.reshape(X_test.shape[0],X_test.shape[1] , 1)”

]

{

“cell_type”: “code”,

“execution_count”: 22,

“metadata”: {

“id”: “DgZz9Bs5FhHU”

“outputs”: [],

“source”: [

“### Create the Stacked LSTM model\n”,

“from tensorflow.keras.models import Sequential\n”,

“from tensorflow.keras.layers import Dense\n”,

“from tensorflow.keras.layers import LSTM”

]

{

“cell_type”: “code”,

“execution_count”: 23,

“metadata”: {

“id”: “Lk-9tSZOFhHU”

“outputs”: [],

“source”: [

“model=Sequential()\n”,

“model.add(LSTM(50,return_sequences=True,input_shape=(100,1)))\n”,

“model.add(LSTM(50,return_sequences=True))\n”,

“model.add(LSTM(50))\n”,

“model.add(Dense(1))\n”,

“model.compile(loss=’mean_squared_error’,optimizer=’adam’)\n”

]

{

“cell_type”: “code”,

“execution_count”: 24,

“metadata”: {

“colab”: {

“base_uri”: “https://localhost:8080/”

“id”: “NSmQcT3WFhHU”,

“outputId”: “81dfb315-15ac-4082-aa78-c894890d4ce7”

“outputs”: [

{

“output_type”: “stream”,

“name”: “stdout”,

“text”: [

“Model: \”sequential\”\n”,

“_________________________________________________________________\n”,

” Layer (type) Output Shape Param # \n”,

“=================================================================\n”,

” lstm (LSTM) (None, 100, 50) 10400 \n”,

” \n”,

” lstm_1 (LSTM) (None, 100, 50) 20200 \n”,

” \n”,

” lstm_2 (LSTM) (None, 50) 20200 \n”,

” \n”,

” dense (Dense) (None, 1) 51 \n”,

” \n”,

“=================================================================\n”,

“Total params: 50,851\n”,

“Trainable params: 50,851\n”,

“Non-trainable params: 0\n”,

“_________________________________________________________________\n”

]

}

“source”: [

“model.summary()”

]

{

“cell_type”: “code”,

“execution_count”: 25,

“metadata”: {

“colab”: {

“base_uri”: “https://localhost:8080/”

“id”: “TQf0GLgaFhHV”,

“outputId”: “b03339b1-ad32-480b-9f5a-6fb0fe40f505”

“outputs”: [

{

“output_type”: “stream”,

“name”: “stdout”,

“text”: [

“Model: \”sequential\”\n”,

“_________________________________________________________________\n”,

” Layer (type) Output Shape Param # \n”,

“=================================================================\n”,

” lstm (LSTM) (None, 100, 50) 10400 \n”,

” \n”,

” lstm_1 (LSTM) (None, 100, 50) 20200 \n”,

” \n”,

” lstm_2 (LSTM) (None, 50) 20200 \n”,

” \n”,

” dense (Dense) (None, 1) 51 \n”,

” \n”,

“=================================================================\n”,

“Total params: 50,851\n”,

“Trainable params: 50,851\n”,

“Non-trainable params: 0\n”,

“_________________________________________________________________\n”

]

}

“source”: [

“model.summary()”

]

{

“cell_type”: “code”,

“execution_count”: 25,

“metadata”: {

“id”: “uRPkrZayFhHV”

“outputs”: [],

“source”: [

“”

]

{

“cell_type”: “code”,

“execution_count”: 28,

“metadata”: {

“colab”: {

“base_uri”: “https://localhost:8080/”

“id”: “XFNKu-fDFhHV”,

“outputId”: “362d7d94-ceb9-4271-c6ce-35b5d20eb57e”

“outputs”: [

{

“output_type”: “execute_result”,

“data”: {

“text/plain”: [

“<keras.callbacks.History at 0x7fd4c7fc8f50>”

]

“metadata”: {},

“execution_count”: 28

}

“source”: [

“model.fit(X_train,y_train,validation_data=(X_test,ytest),epochs=10,batch_size=64,verbose=1)”

]

{

“cell_type”: “code”,

“execution_count”: 29,

“metadata”: {

“id”: “MP2F59k9FhHV”

“outputs”: [],

“source”: [

“import tensorflow as tf”

]

{

“cell_type”: “code”,

“execution_count”: 30,

“metadata”: {

“colab”: {

“base_uri”: “https://localhost:8080/”,

“height”: 35

“id”: “IKxCtK6JFhHV”,

“outputId”: “38984df0-9153-4024-e9c4-0cd1e57d6931”

“outputs”: [

{

“output_type”: “execute_result”,

“data”: {

“text/plain”: [

“‘2.8.0′”

“application/vnd.google.colaboratory.intrinsic+json”: {

“type”: “string”

}

“metadata”: {},

“execution_count”: 30

}

“source”: [

“tf.__version__”

]

{

“cell_type”: “code”,

“source”: [

“### Lets Do the prediction and check performance metrics\n”,

“train_predict=model.predict(X_train)\n”,

“test_predict=model.predict(X_test)”

“metadata”: {

“id”: “ZYXRhHqihkQr”

“execution_count”: null,

“outputs”: []

{

“cell_type”: “code”,

“execution_count”: null,

“metadata”: {

“id”: “MM0AckqgFhHW”

“outputs”: [],

“source”: [

“##Transformback to original form\n”,

“train_predict=scaler.inverse_transform(train_predict)\n”,

“test_predict=scaler.inverse_transform(test_predict)”

]

{

“cell_type”: “code”,

“execution_count”: null,

“metadata”: {

“id”: “rU8PFC1vFhHW”

“outputs”: [],

“source”: [

“### Calculate RMSE performance metrics\n”,

“import math\n”,

“from sklearn.metrics import mean_squared_error\n”,

“math.sqrt(mean_squared_error(y_train,train_predict))”

]

{

“cell_type”: “code”,

“execution_count”: null,

“metadata”: {

“colab”: {

“base_uri”: “https://localhost:8080/”

“id”: “GBrz0nPzFhHX”,

“outputId”: “d6a5cc48-655e-4138-a617-717034db6306”

“outputs”: [

{

“output_type”: “execute_result”,

“data”: {

“text/plain”: [

“237.19528904864237”

]

“metadata”: {},

“execution_count”: 34

}

“source”: [

“### Test Data RMSE\n”,

“math.sqrt(mean_squared_error(ytest,test_predict))”

]

{

“cell_type”: “code”,

“execution_count”: null,

“metadata”: {

“colab”: {

“base_uri”: “https://localhost:8080/”,

“height”: 265

“id”: “XZcCzkisFhHX”,

“outputId”: “40802fcc-ee69-451f-a6de-0a729129feda”

“outputs”: [

{

“output_type”: “display_data”,

“data”: {

“text/plain”: [

“<Figure size 432×288 with 1 Axes>”

“image/png”: “”

“metadata”: {

“needs_background”: “light”

}

“source”: [

“### Plotting \n”,

“# shift train predictions for plotting\n”,

“look_back=100\n”,

“trainPredictPlot = numpy.empty_like(df1)\n”,

“trainPredictPlot[:, :] = np.nan\n”,

“trainPredictPlot[look_back:len(train_predict)+look_back, :] = train_predict\n”,

“# shift test predictions for plotting\n”,

“testPredictPlot = numpy.empty_like(df1)\n”,

“testPredictPlot[:, :] = numpy.nan\n”,

“testPredictPlot[len(train_predict)+(look_back*2)+1:len(df1)-1, :] = test_predict\n”,

“# plot baseline and predictions\n”,

“plt.plot(scaler.inverse_transform(df1))\n”,

“plt.plot(trainPredictPlot)\n”,

“plt.plot(testPredictPlot)\n”,

“plt.show()”

]

{

“cell_type”: “code”,

“execution_count”: null,

“metadata”: {

“colab”: {

“base_uri”: “https://localhost:8080/”

“id”: “2-A3QifDFhHX”,

“outputId”: “c3f8897c-6851-49d8-a2e2-a2ddd190888d”

“outputs”: [

{

“output_type”: “execute_result”,

“data”: {

“text/plain”: [

“441”

]

“metadata”: {},

“execution_count”: 36

}

“source”: [

“len(test_data)”

]

{

“cell_type”: “code”,

“execution_count”: null,

“metadata”: {

“colab”: {

“base_uri”: “https://localhost:8080/”

“id”: “ZcMbhA5NFhHX”,

“outputId”: “58899205-2fa7-4acd-ec3f-06d96b5daff8”

“outputs”: [

{

“output_type”: “execute_result”,

“data”: {

“text/plain”: [

“(1, 100)”

]

“metadata”: {},

“execution_count”: 37

}

“source”: [

“x_input=test_data[341:].reshape(1,-1)\n”,

“x_input.shape\n”

]

{

“cell_type”: “code”,

“execution_count”: null,

“metadata”: {

“id”: “0vXOejOZFhHY”

“outputs”: [],

“source”: [

“”

]

{

“cell_type”: “code”,

“execution_count”: null,

“metadata”: {

“id”: “VawQjpVmFhHY”

“outputs”: [],

“source”: [

“”

]

{

“cell_type”: “code”,

“execution_count”: null,

“metadata”: {

“id”: “21IxQfiXFhHY”

“outputs”: [],

“source”: [

“temp_input=list(x_input)\n”,

“temp_input=temp_input[0].tolist()”

]

{

“cell_type”: “code”,

“execution_count”: null,

“metadata”: {

“id”: “S0ivcO4TFhHY”,

“outputId”: “6974538f-ba4a-45fd-de74-e7fd165bf911”

“outputs”: [

{

“data”: {

“text/plain”: [

“[0.8583551465000423,\n”,

” 0.8866418981676942,\n”,

” 0.8743139407244789,\n”,

” 0.8843198513890065,\n”,

” 0.8783669678290975,\n”,

” 0.8986321033521913,\n”,

” 0.925821160179009,\n”,

” 0.9287764924427933,\n”,

” 0.9567677108840666,\n”,

” 0.9386979650426415,\n”,

” 0.933040614709111,\n”,

” 0.9495060373216249,\n”,

” 0.9642404796082076,\n”,

” 0.9551211686228154,\n”,

” 0.9598919192772104,\n”,

” 0.9663514312251966,\n”,

” 0.9624672802499368,\n”,

” 0.9229502659799038,\n”,

” 0.9598497002448705,\n”,

” 0.9879253567508233,\n”,

” 0.985941062230854,\n”,

” 0.9253145317909315,\n”,

” 0.9217259140420504,\n”,

” 0.964747107996285,\n”,

” 0.9757240564046274,\n”,

” 0.9915984125643842,\n”,

” 0.9697289538123788,\n”,

” 0.9761462467280253,\n”,

” 0.9679557544541082,\n”,

” 1.0000000000000002,\n”,

” 0.9901629654648318,\n”,

” 0.9905007177235499,\n”,

” 0.9653803934813816,\n”,

” 0.9848855864223593,\n”,

” 0.9708688676855528,\n”,

” 0.9402600692392133,\n”,

” 0.8774803681499621,\n”,

” 0.8348391454867856,\n”,

” 0.8541332432660644,\n”,

” 0.7733682344000676,\n”,

” 0.7726927298826314,\n”,

” 0.8801401671873683,\n”,

” 0.8400743054969182,\n”,

” 0.8967322468969012,\n”,

” 0.8552731571392387,\n”,

” 0.8388499535590646,\n”,

” 0.7423372456303303,\n”,

” 0.8232711306256861,\n”,

” 0.7814320695769654,\n”,

” 0.6665963016127672,\n”,

” 0.7921557037912694,\n”,

” 0.6411804441442204,\n”,

” 0.6861437135860848,\n”,

” 0.6600101325677616,\n”,

” 0.6520307354555435,\n”,

” 0.5864223591995272,\n”,

” 0.5658616904500551,\n”,

” 0.660896732246897,\n”,

” 0.6551549438486872,\n”,

” 0.7097019336316812,\n”,

” 0.664527569028118,\n”,

” 0.6943764248923416,\n”,

” 0.692181035210673,\n”,

” 0.6356919699400492,\n”,

” 0.6526640209406402,\n”,

” 0.637802921557038,\n”,

” 0.7267162036646122,\n”,

” 0.7138816178333194,\n”,

” 0.7419150553069325,\n”,

” 0.7500211095161702,\n”,

” 0.7722283205268936,\n”,

” 0.8304905851557884,\n”,

” 0.8194291986827664,\n”,

” 0.8289706999915563,\n”,

” 0.8125474964113824,\n”,

” 0.7877649244279323,\n”,

” 0.7516254327450818,\n”,

” 0.7842607447437306,\n”,

” 0.7797433082833742,\n”,

” 0.8132652199611587,\n”,

” 0.8141096006079542,\n”,

” 0.7947310647639958,\n”,

” 0.8333614793548934,\n”,

” 0.8589884319851391,\n”,

” 0.8390188296884238,\n”,

” 0.8562864139153934,\n”,

” 0.8748627881448958,\n”,

” 0.887824031073208,\n”,

” 0.9009541501308793,\n”,

” 0.9279321117959978,\n”,

” 0.9485349995778098,\n”,

” 0.9333361479354896,\n”,

” 0.9174617917757326,\n”,

” 0.925441188887951,\n”,

” 0.9177151059697712,\n”,

” 0.9483239044161109,\n”,

” 0.9406400405302711,\n”,

” 0.9663514312251966,\n”,

” 0.9563033015283293,\n”,

” 0.964915984125644]”

]

“execution_count”: 440,

“metadata”: {},

“output_type”: “execute_result”

}

“source”: [

“temp_input”

]

{

“cell_type”: “code”,

“execution_count”: null,

“metadata”: {

“id”: “ngNu6HSvFhHY”,

“outputId”: “4c04f07f-a817-4a61-8421-cab32c6e5324”

“outputs”: [

{

“name”: “stdout”,

“output_type”: “stream”,

“text”: [

“[0.94413203]\n”,

“101\n”,

“1 day input [0.8866419 0.87431394 0.88431985 0.87836697 0.8986321 0.92582116\n”,

” 0.92877649 0.95676771 0.93869797 0.93304061 0.94950604 0.96424048\n”,

” 0.95512117 0.95989192 0.96635143 0.96246728 0.92295027 0.9598497\n”,

” 0.98792536 0.98594106 0.92531453 0.92172591 0.96474711 0.97572406\n”,

” 0.99159841 0.96972895 0.97614625 0.96795575 1. 0.99016297\n”,

” 0.99050072 0.96538039 0.98488559 0.97086887 0.94026007 0.87748037\n”,

” 0.83483915 0.85413324 0.77336823 0.77269273 0.88014017 0.84007431\n”,

” 0.89673225 0.85527316 0.83884995 0.74233725 0.82327113 0.78143207\n”,

” 0.6665963 0.7921557 0.64118044 0.68614371 0.66001013 0.65203074\n”,

” 0.58642236 0.56586169 0.66089673 0.65515494 0.70970193 0.66452757\n”,

” 0.69437642 0.69218104 0.63569197 0.65266402 0.63780292 0.7267162\n”,

” 0.71388162 0.74191506 0.75002111 0.77222832 0.83049059 0.8194292\n”,

” 0.8289707 0.8125475 0.78776492 0.75162543 0.78426074 0.77974331\n”,

” 0.81326522 0.8141096 0.79473106 0.83336148 0.85898843 0.83901883\n”,

” 0.85628641 0.87486279 0.88782403 0.90095415 0.92793211 0.948535\n”,

” 0.93333615 0.91746179 0.92544119 0.91771511 0.9483239 0.94064004\n”,

” 0.96635143 0.9563033 0.96491598 0.94413203]\n”,

“1 day output [[0.9379593]]\n”,

“2 day input [0.87431394 0.88431985 0.87836697 0.8986321 0.92582116 0.92877649\n”,

” 0.95676771 0.93869797 0.93304061 0.94950604 0.96424048 0.95512117\n”,

” 0.95989192 0.96635143 0.96246728 0.92295027 0.9598497 0.98792536\n”,

” 0.98594106 0.92531453 0.92172591 0.96474711 0.97572406 0.99159841\n”,

” 0.96972895 0.97614625 0.96795575 1. 0.99016297 0.99050072\n”,

” 0.96538039 0.98488559 0.97086887 0.94026007 0.87748037 0.83483915\n”,

” 0.85413324 0.77336823 0.77269273 0.88014017 0.84007431 0.89673225\n”,

” 0.85527316 0.83884995 0.74233725 0.82327113 0.78143207 0.6665963\n”,

” 0.7921557 0.64118044 0.68614371 0.66001013 0.65203074 0.58642236\n”,

” 0.56586169 0.66089673 0.65515494 0.70970193 0.66452757 0.69437642\n”,

” 0.69218104 0.63569197 0.65266402 0.63780292 0.7267162 0.71388162\n”,

” 0.74191506 0.75002111 0.77222832 0.83049059 0.8194292 0.8289707\n”,

” 0.8125475 0.78776492 0.75162543 0.78426074 0.77974331 0.81326522\n”,

” 0.8141096 0.79473106 0.83336148 0.85898843 0.83901883 0.85628641\n”,

” 0.87486279 0.88782403 0.90095415 0.92793211 0.948535 0.93333615\n”,

” 0.91746179 0.92544119 0.91771511 0.9483239 0.94064004 0.96635143\n”,

” 0.9563033 0.96491598 0.94413203 0.93795931]\n”,

“2 day output [[0.9286534]]\n”,

“3 day input [0.88431985 0.87836697 0.8986321 0.92582116 0.92877649 0.95676771\n”,

” 0.93869797 0.93304061 0.94950604 0.96424048 0.95512117 0.95989192\n”,

” 0.96635143 0.96246728 0.92295027 0.9598497 0.98792536 0.98594106\n”,

” 0.92531453 0.92172591 0.96474711 0.97572406 0.99159841 0.96972895\n”,

” 0.97614625 0.96795575 1. 0.99016297 0.99050072 0.96538039\n”,

” 0.98488559 0.97086887 0.94026007 0.87748037 0.83483915 0.85413324\n”,

” 0.77336823 0.77269273 0.88014017 0.84007431 0.89673225 0.85527316\n”,

” 0.83884995 0.74233725 0.82327113 0.78143207 0.6665963 0.7921557\n”,

” 0.64118044 0.68614371 0.66001013 0.65203074 0.58642236 0.56586169\n”,

” 0.66089673 0.65515494 0.70970193 0.66452757 0.69437642 0.69218104\n”,

” 0.63569197 0.65266402 0.63780292 0.7267162 0.71388162 0.74191506\n”,

” 0.75002111 0.77222832 0.83049059 0.8194292 0.8289707 0.8125475\n”,

” 0.78776492 0.75162543 0.78426074 0.77974331 0.81326522 0.8141096\n”,

” 0.79473106 0.83336148 0.85898843 0.83901883 0.85628641 0.87486279\n”,

” 0.88782403 0.90095415 0.92793211 0.948535 0.93333615 0.91746179\n”,

” 0.92544119 0.91771511 0.9483239 0.94064004 0.96635143 0.9563033\n”,

” 0.96491598 0.94413203 0.93795931 0.92865342]\n”,

“3 day output [[0.91987926]]\n”,

“4 day input [0.87836697 0.8986321 0.92582116 0.92877649 0.95676771 0.93869797\n”,

” 0.93304061 0.94950604 0.96424048 0.95512117 0.95989192 0.96635143\n”,

” 0.96246728 0.92295027 0.9598497 0.98792536 0.98594106 0.92531453\n”,

” 0.92172591 0.96474711 0.97572406 0.99159841 0.96972895 0.97614625\n”,

” 0.96795575 1. 0.99016297 0.99050072 0.96538039 0.98488559\n”,

” 0.97086887 0.94026007 0.87748037 0.83483915 0.85413324 0.77336823\n”,

” 0.77269273 0.88014017 0.84007431 0.89673225 0.85527316 0.83884995\n”,

” 0.74233725 0.82327113 0.78143207 0.6665963 0.7921557 0.64118044\n”,

” 0.68614371 0.66001013 0.65203074 0.58642236 0.56586169 0.66089673\n”,

” 0.65515494 0.70970193 0.66452757 0.69437642 0.69218104 0.63569197\n”,

” 0.65266402 0.63780292 0.7267162 0.71388162 0.74191506 0.75002111\n”,

” 0.77222832 0.83049059 0.8194292 0.8289707 0.8125475 0.78776492\n”,

” 0.75162543 0.78426074 0.77974331 0.81326522 0.8141096 0.79473106\n”,

” 0.83336148 0.85898843 0.83901883 0.85628641 0.87486279 0.88782403\n”,

” 0.90095415 0.92793211 0.948535 0.93333615 0.91746179 0.92544119\n”,

” 0.91771511 0.9483239 0.94064004 0.96635143 0.9563033 0.96491598\n”,

” 0.94413203 0.93795931 0.92865342 0.91987926]\n”,

“4 day output [[0.9128097]]\n”,

“5 day input [0.8986321 0.92582116 0.92877649 0.95676771 0.93869797 0.93304061\n”,

” 0.94950604 0.96424048 0.95512117 0.95989192 0.96635143 0.96246728\n”,

” 0.92295027 0.9598497 0.98792536 0.98594106 0.92531453 0.92172591\n”,

” 0.96474711 0.97572406 0.99159841 0.96972895 0.97614625 0.96795575\n”,

” 1. 0.99016297 0.99050072 0.96538039 0.98488559 0.97086887\n”,

” 0.94026007 0.87748037 0.83483915 0.85413324 0.77336823 0.77269273\n”,

” 0.88014017 0.84007431 0.89673225 0.85527316 0.83884995 0.74233725\n”,

” 0.82327113 0.78143207 0.6665963 0.7921557 0.64118044 0.68614371\n”,

” 0.66001013 0.65203074 0.58642236 0.56586169 0.66089673 0.65515494\n”,

” 0.70970193 0.66452757 0.69437642 0.69218104 0.63569197 0.65266402\n”,

” 0.63780292 0.7267162 0.71388162 0.74191506 0.75002111 0.77222832\n”,

” 0.83049059 0.8194292 0.8289707 0.8125475 0.78776492 0.75162543\n”,

” 0.78426074 0.77974331 0.81326522 0.8141096 0.79473106 0.83336148\n”,

” 0.85898843 0.83901883 0.85628641 0.87486279 0.88782403 0.90095415\n”,

” 0.92793211 0.948535 0.93333615 0.91746179 0.92544119 0.91771511\n”,

” 0.9483239 0.94064004 0.96635143 0.9563033 0.96491598 0.94413203\n”,

” 0.93795931 0.92865342 0.91987926 0.91280973]\n”,

“5 day output [[0.90777564]]\n”,

“6 day input [0.92582116 0.92877649 0.95676771 0.93869797 0.93304061 0.94950604\n”,

” 0.96424048 0.95512117 0.95989192 0.96635143 0.96246728 0.92295027\n”,

” 0.9598497 0.98792536 0.98594106 0.92531453 0.92172591 0.96474711\n”,

” 0.97572406 0.99159841 0.96972895 0.97614625 0.96795575 1.\n”,

” 0.99016297 0.99050072 0.96538039 0.98488559 0.97086887 0.94026007\n”,

” 0.87748037 0.83483915 0.85413324 0.77336823 0.77269273 0.88014017\n”,

” 0.84007431 0.89673225 0.85527316 0.83884995 0.74233725 0.82327113\n”,

” 0.78143207 0.6665963 0.7921557 0.64118044 0.68614371 0.66001013\n”,

” 0.65203074 0.58642236 0.56586169 0.66089673 0.65515494 0.70970193\n”,

” 0.66452757 0.69437642 0.69218104 0.63569197 0.65266402 0.63780292\n”,

” 0.7267162 0.71388162 0.74191506 0.75002111 0.77222832 0.83049059\n”,

” 0.8194292 0.8289707 0.8125475 0.78776492 0.75162543 0.78426074\n”,

” 0.77974331 0.81326522 0.8141096 0.79473106 0.83336148 0.85898843\n”,

” 0.83901883 0.85628641 0.87486279 0.88782403 0.90095415 0.92793211\n”,

” 0.948535 0.93333615 0.91746179 0.92544119 0.91771511 0.9483239\n”,

” 0.94064004 0.96635143 0.9563033 0.96491598 0.94413203 0.93795931\n”,

” 0.92865342 0.91987926 0.91280973 0.90777564]\n”,

“6 day output [[0.9047326]]\n”,

“7 day input [0.92877649 0.95676771 0.93869797 0.93304061 0.94950604 0.96424048\n”,