Author avatar

Deepika Singh

Advanced Time Series Modeling (ARIMA) Models in Python

Deepika Singh

  • Jun 4, 2020
  • 10 Min read
  • 40,988 Views
  • Jun 4, 2020
  • 10 Min read
  • 40,988 Views
Data
Python
Introduction

Introduction

Time series algorithms are used extensively for analyzing and forecasting time-based data. One set of popular and powerful time series algorithms is the ARIMA class of models, which are based on describing autocorrelations in the data.

ARIMA stands for Autoregressive Integrated Moving Average and has three components, p, d, and q, that are required to build the ARIMA model. These three components are:

p: Number of autoregressive lags

d: Order of differencing required to make the series stationary

q: Number of moving average lags

In this guide, you will learn the core concepts of ARIMA modeling and how to implement it in Python. Let's begin with understanding and loading the data.

Data

This guide uses the fictitious monthly sales data of a supermarket chain containing 564 observations and three variables, as described below:

  1. Date: the first date of every month

  2. Sales: daily sales, in thousands of dollars

  3. Class: the variable denoting the training and test data set partition

The lines of code below import the required libraries and the data.

1import pandas as pd
2import numpy as np 
3
4# Reading the data
5df = pd.read_csv("data.csv")
6print(df.shape)
7print(df.info())
python

Output:

1(564, 3)
2
3<class 'pandas.core.frame.DataFrame'>
4RangeIndex: 564 entries, 0 to 563
5Data columns (total 3 columns):
6Date     564 non-null object
7Sales    564 non-null int64
8Class    564 non-null object
9dtypes: int64(1), object(2)
10memory usage: 13.3+ KB
11None

The next step is to create the training and test datasets for model building and evaluation. 

1train = df[df["Class"] == "Train"]
2test = df[df["Class"] == "Test"]
3print(train.shape)
4print(test.shape)
python

Output:

1    (552, 3)
2    (12, 3)

You should also create train and test arrays with the code below. 

1train_array = train["Sales"]
2print(train_array.shape)
3
4test_array = test["Sales"]
5print(test_array.shape)
python

Output:

1 (552,)
2     
3 (12,)

With the data prepared, you are ready to move to the forecasting techniques in the subsequent sections. However, before building ARIMA models, it's important to understand the statistical concept of stationarity.

Stationary Series

One of the requirements for ARIMA is that the time series should be stationary. A stationary series is one where the properties do not change over time. There are several methods to check the stationarity of a series. The one you’ll use in this guide is the Augmented Dickey-Fuller test.

Augmented Dickey-Fuller Test

The Augmented Dickey-Fuller test is a type of statistical unit root test. The test uses an autoregressive model and optimizes an information criterion across multiple different lag values.

The null hypothesis of the test is that the time series is not stationary, while the alternate hypothesis (rejecting the null hypothesis) is that the time series is stationary.

The first step is to import the adfuller module from the statsmodels package. This is done in the first line of code below. The second line performs and prints the p-value of the test. 

1from statsmodels.tsa.stattools import adfuller
2
3print("p-value:", adfuller(train_array.dropna())[1])
python

Output: 

1p-value: 0.3440379665909026

The output above shows that the p-value is greater than the significance level of 0.05, so we fail to reject the null hypothesis. The series is not stationary and requires differencing.

The series can be differenced using the diff() function. The first line of code below performs the first order differencing, while the second line performs the Augmented Dickey-Fuller Test. 

1diff_1 = train_array.diff().dropna()
2print("p-value:", adfuller(diff_1.dropna())[1])
python

Output: 

1p-value: 0.001

The p-value now is below the significance level, indicating that the series is stationary.

ARIMA Model

You are now ready to build the ARIMA model and make predictions. You will be using the auto_arima function in Python, which automatically discovers the optimal order for an ARIMA model. In simple terms, the function will automatically determine the parameters p, d’, and q of the ARIMA model.

The important parameters of the function are:

  1. The time-series to which you fit the ARIMA model.
  1. start_p: the starting value of p, the order of the auto-regressive (AR) model. This must be a positive integer.
  1. start_q: the starting value of q, the order of the moving-average (MA) model. This must be a positive integer.
  1. d: the order of first-differencing. The default setting is none, and then the value is selected automatically based on the results of the test, in this case the Augmented Dickey-Fuller test.
  1. test: type of unit root test to use in order to detect stationarity if stationary is False and d is none.

You will now build the ARIMA estimator. The first step is to import the pmdarima library that contains the auto_arima function. The second step is to define a function that takes in the time series array and returns the auto-arima model. These steps are done in the code below. 

1import pmdarima as pmd
2
3def arimamodel(timeseriesarray):
4    autoarima_model = pmd.auto_arima(timeseriesarray, 
5                              start_p=1, 
6                              start_q=1,
7                              test="adf",
8                              trace=True)
9    return autoarima_model
python

The next step is to use the function defined above and build the ARIMA estimator on the training data. 

1arima_model = arimamodel(train_array)
2arima_model.summary()
python

Output: 

1Fit ARIMA: order=(1, 1, 1); AIC=7974.318, BIC=7991.565, Fit time=0.425 seconds
2Fit ARIMA: order=(0, 1, 0); AIC=7975.310, BIC=7983.934, Fit time=0.011 seconds
3Fit ARIMA: order=(1, 1, 0); AIC=7973.112, BIC=7986.047, Fit time=0.177 seconds
4Fit ARIMA: order=(0, 1, 1); AIC=7973.484, BIC=7986.419, Fit time=0.084 seconds
5Fit ARIMA: order=(2, 1, 0); AIC=7974.012, BIC=7991.259, Fit time=0.274 seconds
6Fit ARIMA: order=(2, 1, 1); AIC=7973.626, BIC=7995.185, Fit time=0.989 seconds
7Total fit time: 2.000 seconds
8
9ARIMA Model Results
10Dep. Variable:	D.y	No. Observations:	551
11Model:	ARIMA(1, 1, 0)	Log Likelihood	-3983.556
12Method:	css-mle	S.D. of innovations	333.866
13Date:	Wed, 27 May 2020	AIC	7973.112
14Time:	11:37:46	BIC	7986.047
15Sample:	1	HQIC	7978.166
16coef	std err	z	P>|z|	[0.025	0.975]
17const	9.0108	13.085	0.689	0.491	-16.636	34.658
18ar.L1.D.y	-0.0871	0.042	-2.053	0.041	-0.170	-0.004
19Roots
20Real	Imaginary	Modulus	Frequency
21AR.1	-11.4797	+0.0000j	11.4797	0.5000

The output above shows that the final model fitted was an ARIMA(1,1,0) estimator, where the values of the parameters p, d, and q were one, one, and zero, respectively. The auto_arima functions tests the time series with different combinations of p, d, and q using AIC as the criterion. AIC stands for Akaike Information Criterion, which estimates the relative amount of information lost by a given model. In simple terms, a lower AIC value is preferred. In the above output, the lowest AIC value of 7973.112 was obtained for the ARIMA(1, 1, 0) model, and that is used as the final estimator.

You have trained the model and will now use it make predictions on the test data and perform model evaluation. One step before doing this is to create a utility function that will be used as an evaluation metric. The code below creates a utility function for calculating the mean absolute percentage error (MAPE), which is the metric to be used. The lower the MAPE value, the better the forecasting model performance. 

1def mean_absolute_percentage_error(y_true, y_pred): 
2    y_true, y_pred = np.array(y_true), np.array(y_pred)
3    return np.mean(np.abs((y_true - y_pred) / y_true)) * 100
python

The next step is to make predictions on the test data, which is done using the code below. The second line prints the first five observations. 

1test['ARIMA'] = automodel.predict(len(test))
2
3test.head(5)
python

Output: 

1	Date	      Sales	   Class	  ARIMA
2552	01-01-2014	  6785        Test      6882.9
3553	01-02-2014	  6856        Test      6889.8
4554	01-03-2014	  6853        Test      6898.9
5555	01-04-2014	  6400       Test      6907.9
6556	01-05-2014	  6442        Test      6916.9

The final step is to evaluate the predictions on the test data using the utility function as shown below. 

1mean_absolute_percentage_error(test.Sales, test.ARIMA)
python

Output: 

19.7846

The output above shows that the MAPE for the test data is 9.8%. The low value means that the model results are good.

Conclusion

In this guide, you learned about forecasting time series data using ARIMA. You learned about the stationarity requirement of time series and how to make a non-stationary series stationary through differencing. Finally, you learned how to build and interpret the ARIMA estimator for forecasting using Python.

To learn more about data science using Python, please refer to the following guides.

LEARN MORE