Advanced Time Series Modeling (ARIMA) Models in Python
Introduction
Time series algorithms are used extensively for analyzing and forecasting time-based data. One set of popular and powerful time series algorithms is the ARIMA class of models, which are based on describing autocorrelations in the data.
ARIMA stands for Autoregressive Integrated Moving Average and has three components, p, d, and q, that are required to build the ARIMA model. These three components are:
p: Number of autoregressive lags
d: Order of differencing required to make the series stationary
q: Number of moving average lags
In this guide, you will learn the core concepts of ARIMA modeling and how to implement it in Python. Let's begin with understanding and loading the data.
Data
This guide uses the fictitious monthly sales data of a supermarket chain containing 564 observations and three variables, as described below:
Date: the first date of every monthSales: daily sales, in thousands of dollarsClass: the variable denoting the training and test data set partition
The lines of code below import the required libraries and the data.
1import pandas as pd
2import numpy as np
3
4# Reading the data
5df = pd.read_csv("data.csv")
6print(df.shape)
7print(df.info())Output:
1(564, 3)
2
3<class 'pandas.core.frame.DataFrame'>
4RangeIndex: 564 entries, 0 to 563
5Data columns (total 3 columns):
6Date 564 non-null object
7Sales 564 non-null int64
8Class 564 non-null object
9dtypes: int64(1), object(2)
10memory usage: 13.3+ KB
11NoneThe next step is to create the training and test datasets for model building and evaluation.
1train = df[df["Class"] == "Train"]
2test = df[df["Class"] == "Test"]
3print(train.shape)
4print(test.shape)Output:
1 (552, 3)
2 (12, 3)You should also create train and test arrays with the code below.
1train_array = train["Sales"]
2print(train_array.shape)
3
4test_array = test["Sales"]
5print(test_array.shape) Output:
1 (552,)
2
3 (12,)With the data prepared, you are ready to move to the forecasting techniques in the subsequent sections. However, before building ARIMA models, it's important to understand the statistical concept of stationarity.
Stationary Series
One of the requirements for ARIMA is that the time series should be stationary. A stationary series is one where the properties do not change over time. There are several methods to check the stationarity of a series. The one you’ll use in this guide is the Augmented Dickey-Fuller test.
Augmented Dickey-Fuller Test
The Augmented Dickey-Fuller test is a type of statistical unit root test. The test uses an autoregressive model and optimizes an information criterion across multiple different lag values.
The null hypothesis of the test is that the time series is not stationary, while the alternate hypothesis (rejecting the null hypothesis) is that the time series is stationary.
The first step is to import the adfuller module from the statsmodels package. This is done in the first line of code below. The second line performs and prints the p-value of the test.
1from statsmodels.tsa.stattools import adfuller
2
3print("p-value:", adfuller(train_array.dropna())[1])Output:
1p-value: 0.3440379665909026The output above shows that the p-value is greater than the significance level of 0.05, so we fail to reject the null hypothesis. The series is not stationary and requires differencing.
The series can be differenced using the diff() function. The first line of code below performs the first order differencing, while the second line performs the Augmented Dickey-Fuller Test.
1diff_1 = train_array.diff().dropna()
2print("p-value:", adfuller(diff_1.dropna())[1])Output:
1p-value: 0.001The p-value now is below the significance level, indicating that the series is stationary.
ARIMA Model
You are now ready to build the ARIMA model and make predictions. You will be using the auto_arima function in Python, which automatically discovers the optimal order for an ARIMA model. In simple terms, the function will automatically determine the parameters p, d’, and q of the ARIMA model.
The important parameters of the function are:
- The time-series to which you fit the ARIMA model.
start_p: the starting value ofp, the order of the auto-regressive (AR) model. This must be a positive integer.
start_q: the starting value ofq, the order of the moving-average (MA) model. This must be a positive integer.
d: the order of first-differencing. The default setting is none, and then the value is selected automatically based on the results of the test, in this case the Augmented Dickey-Fuller test.
test: type of unit root test to use in order to detect stationarity if stationary is False anddis none.
You will now build the ARIMA estimator. The first step is to import the pmdarima library that contains the auto_arima function. The second step is to define a function that takes in the time series array and returns the auto-arima model. These steps are done in the code below.
1import pmdarima as pmd
2
3def arimamodel(timeseriesarray):
4 autoarima_model = pmd.auto_arima(timeseriesarray,
5 start_p=1,
6 start_q=1,
7 test="adf",
8 trace=True)
9 return autoarima_modelThe next step is to use the function defined above and build the ARIMA estimator on the training data.
1arima_model = arimamodel(train_array)
2arima_model.summary()Output:
1Fit ARIMA: order=(1, 1, 1); AIC=7974.318, BIC=7991.565, Fit time=0.425 seconds
2Fit ARIMA: order=(0, 1, 0); AIC=7975.310, BIC=7983.934, Fit time=0.011 seconds
3Fit ARIMA: order=(1, 1, 0); AIC=7973.112, BIC=7986.047, Fit time=0.177 seconds
4Fit ARIMA: order=(0, 1, 1); AIC=7973.484, BIC=7986.419, Fit time=0.084 seconds
5Fit ARIMA: order=(2, 1, 0); AIC=7974.012, BIC=7991.259, Fit time=0.274 seconds
6Fit ARIMA: order=(2, 1, 1); AIC=7973.626, BIC=7995.185, Fit time=0.989 seconds
7Total fit time: 2.000 seconds
8
9ARIMA Model Results
10Dep. Variable: D.y No. Observations: 551
11Model: ARIMA(1, 1, 0) Log Likelihood -3983.556
12Method: css-mle S.D. of innovations 333.866
13Date: Wed, 27 May 2020 AIC 7973.112
14Time: 11:37:46 BIC 7986.047
15Sample: 1 HQIC 7978.166
16coef std err z P>|z| [0.025 0.975]
17const 9.0108 13.085 0.689 0.491 -16.636 34.658
18ar.L1.D.y -0.0871 0.042 -2.053 0.041 -0.170 -0.004
19Roots
20Real Imaginary Modulus Frequency
21AR.1 -11.4797 +0.0000j 11.4797 0.5000 The output above shows that the final model fitted was an ARIMA(1,1,0) estimator, where the values of the parameters p, d, and q were one, one, and zero, respectively. The auto_arima functions tests the time series with different combinations of p, d, and q using AIC as the criterion. AIC stands for Akaike Information Criterion, which estimates the relative amount of information lost by a given model. In simple terms, a lower AIC value is preferred. In the above output, the lowest AIC value of 7973.112 was obtained for the ARIMA(1, 1, 0) model, and that is used as the final estimator.
You have trained the model and will now use it make predictions on the test data and perform model evaluation. One step before doing this is to create a utility function that will be used as an evaluation metric. The code below creates a utility function for calculating the mean absolute percentage error (MAPE), which is the metric to be used. The lower the MAPE value, the better the forecasting model performance.
1def mean_absolute_percentage_error(y_true, y_pred):
2 y_true, y_pred = np.array(y_true), np.array(y_pred)
3 return np.mean(np.abs((y_true - y_pred) / y_true)) * 100The next step is to make predictions on the test data, which is done using the code below. The second line prints the first five observations.
1test['ARIMA'] = automodel.predict(len(test))
2
3test.head(5)Output:
1 Date Sales Class ARIMA
2552 01-01-2014 6785 Test 6882.9
3553 01-02-2014 6856 Test 6889.8
4554 01-03-2014 6853 Test 6898.9
5555 01-04-2014 6400 Test 6907.9
6556 01-05-2014 6442 Test 6916.9The final step is to evaluate the predictions on the test data using the utility function as shown below.
1mean_absolute_percentage_error(test.Sales, test.ARIMA)Output:
19.7846The output above shows that the MAPE for the test data is 9.8%. The low value means that the model results are good.
Conclusion
In this guide, you learned about forecasting time series data using ARIMA. You learned about the stationarity requirement of time series and how to make a non-stationary series stationary through differencing. Finally, you learned how to build and interpret the ARIMA estimator for forecasting using Python.
To learn more about data science using Python, please refer to the following guides.