Advanced Time Series Modeling (ARIMA) Models in Python

Jun 4, 2020 • 10 Minute Read

Introduction

Time series algorithms are used extensively for analyzing and forecasting time-based data. One set of popular and powerful time series algorithms is the ARIMA class of models, which are based on describing autocorrelations in the data.

ARIMA stands for Autoregressive Integrated Moving Average and has three components, p, d, and q, that are required to build the ARIMA model. These three components are:

p: Number of autoregressive lags

d: Order of differencing required to make the series stationary

q: Number of moving average lags

In this guide, you will learn the core concepts of ARIMA modeling and how to implement it in Python. Let's begin with understanding and loading the data.

Data

This guide uses the fictitious monthly sales data of a supermarket chain containing 564 observations and three variables, as described below:

Date: the first date of every month
Sales: daily sales, in thousands of dollars
Class: the variable denoting the training and test data set partition

The lines of code below import the required libraries and the data.

      import pandas as pd
import numpy as np 

# Reading the data
df = pd.read_csv("data.csv")
print(df.shape)
print(df.info())
    

Output:

      (564, 3)

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 564 entries, 0 to 563
Data columns (total 3 columns):
Date     564 non-null object
Sales    564 non-null int64
Class    564 non-null object
dtypes: int64(1), object(2)
memory usage: 13.3+ KB
None
    

The next step is to create the training and test datasets for model building and evaluation.

      train = df[df["Class"] == "Train"]
test = df[df["Class"] == "Test"]
print(train.shape)
print(test.shape)
    

Output:

      (552, 3)
    (12, 3)
    

You should also create train and test arrays with the code below.

      train_array = train["Sales"]
print(train_array.shape)

test_array = test["Sales"]
print(test_array.shape)
    

Output:

      (552,)
     
 (12,)

With the data prepared, you are ready to move to the forecasting techniques in the subsequent sections. However, before building ARIMA models, it's important to understand the statistical concept of stationarity.

Stationary Series

One of the requirements for ARIMA is that the time series should be stationary. A stationary series is one where the properties do not change over time. There are several methods to check the stationarity of a series. The one you’ll use in this guide is the Augmented Dickey-Fuller test.

Augmented Dickey-Fuller Test

The Augmented Dickey-Fuller test is a type of statistical unit root test. The test uses an autoregressive model and optimizes an information criterion across multiple different lag values.

The null hypothesis of the test is that the time series is not stationary, while the alternate hypothesis (rejecting the null hypothesis) is that the time series is stationary.

The first step is to import the adfuller module from the statsmodels package. This is done in the first line of code below. The second line performs and prints the p-value of the test.

      from statsmodels.tsa.stattools import adfuller

print("p-value:", adfuller(train_array.dropna())[1])

Output:

      p-value: 0.3440379665909026

The output above shows that the p-value is greater than the significance level of 0.05, so we fail to reject the null hypothesis. The series is not stationary and requires differencing.

The series can be differenced using the diff() function. The first line of code below performs the first order differencing, while the second line performs the Augmented Dickey-Fuller Test.

      diff_1 = train_array.diff().dropna()
print("p-value:", adfuller(diff_1.dropna())[1])
    

Output:

      p-value: 0.001

The p-value now is below the significance level, indicating that the series is stationary.

ARIMA Model

You are now ready to build the ARIMA model and make predictions. You will be using the auto_arima function in Python, which automatically discovers the optimal order for an ARIMA model. In simple terms, the function will automatically determine the parameters p, d’, and q of the ARIMA model.

The important parameters of the function are:

The time-series to which you fit the ARIMA model.
start_p: the starting value of p, the order of the auto-regressive (AR) model. This must be a positive integer.
start_q: the starting value of q, the order of the moving-average (MA) model. This must be a positive integer.
d: the order of first-differencing. The default setting is none, and then the value is selected automatically based on the results of the test, in this case the Augmented Dickey-Fuller test.
test: type of unit root test to use in order to detect stationarity if stationary is False and d is none.

You will now build the ARIMA estimator. The first step is to import the pmdarima library that contains the auto_arima function. The second step is to define a function that takes in the time series array and returns the auto-arima model. These steps are done in the code below.

      import pmdarima as pmd

def arimamodel(timeseriesarray):
    autoarima_model = pmd.auto_arima(timeseriesarray, 
                              start_p=1, 
                              start_q=1,
                              test="adf",
                              trace=True)
    return autoarima_model
    

The next step is to use the function defined above and build the ARIMA estimator on the training data.

      arima_model = arimamodel(train_array)
arima_model.summary()
    

Output:

      Fit ARIMA: order=(1, 1, 1); AIC=7974.318, BIC=7991.565, Fit time=0.425 seconds
Fit ARIMA: order=(0, 1, 0); AIC=7975.310, BIC=7983.934, Fit time=0.011 seconds
Fit ARIMA: order=(1, 1, 0); AIC=7973.112, BIC=7986.047, Fit time=0.177 seconds
Fit ARIMA: order=(0, 1, 1); AIC=7973.484, BIC=7986.419, Fit time=0.084 seconds
Fit ARIMA: order=(2, 1, 0); AIC=7974.012, BIC=7991.259, Fit time=0.274 seconds
Fit ARIMA: order=(2, 1, 1); AIC=7973.626, BIC=7995.185, Fit time=0.989 seconds
Total fit time: 2.000 seconds

ARIMA Model Results
Dep. Variable:	D.y	No. Observations:	551
Model:	ARIMA(1, 1, 0)	Log Likelihood	-3983.556
Method:	css-mle	S.D. of innovations	333.866
Date:	Wed, 27 May 2020	AIC	7973.112
Time:	11:37:46	BIC	7986.047
Sample:	1	HQIC	7978.166
coef	std err	z	P>|z|	[0.025	0.975]
const	9.0108	13.085	0.689	0.491	-16.636	34.658
ar.L1.D.y	-0.0871	0.042	-2.053	0.041	-0.170	-0.004
Roots
Real	Imaginary	Modulus	Frequency
AR.1	-11.4797	+0.0000j	11.4797	0.5000
    

The output above shows that the final model fitted was an ARIMA(1,1,0) estimator, where the values of the parameters p, d, and q were one, one, and zero, respectively. The auto_arima functions tests the time series with different combinations of p, d, and q using AIC as the criterion. AIC stands for Akaike Information Criterion, which estimates the relative amount of information lost by a given model. In simple terms, a lower AIC value is preferred. In the above output, the lowest AIC value of 7973.112 was obtained for the ARIMA(1, 1, 0) model, and that is used as the final estimator.

You have trained the model and will now use it make predictions on the test data and perform model evaluation. One step before doing this is to create a utility function that will be used as an evaluation metric. The code below creates a utility function for calculating the mean absolute percentage error (MAPE), which is the metric to be used. The lower the MAPE value, the better the forecasting model performance.

      def mean_absolute_percentage_error(y_true, y_pred): 
    y_true, y_pred = np.array(y_true), np.array(y_pred)
    return np.mean(np.abs((y_true - y_pred) / y_true)) * 100
    

The next step is to make predictions on the test data, which is done using the code below. The second line prints the first five observations.

      test['ARIMA'] = automodel.predict(len(test))

test.head(5)

Output:

      Date	      Sales	   Class	  ARIMA
01-01-2014	  6785        Test      6882.9
01-02-2014	  6856        Test      6889.8
01-03-2014	  6853        Test      6898.9
01-04-2014	  6400       Test      6907.9
01-05-2014	  6442        Test      6916.9
    

The final step is to evaluate the predictions on the test data using the utility function as shown below.

      mean_absolute_percentage_error(test.Sales, test.ARIMA)

Output:

      9.7846

The output above shows that the MAPE for the test data is 9.8%. The low value means that the model results are good.

Conclusion

In this guide, you learned about forecasting time series data using ARIMA. You learned about the stationarity requirement of time series and how to make a non-stationary series stationary through differencing. Finally, you learned how to build and interpret the ARIMA estimator for forecasting using Python.

To learn more about data science using Python, please refer to the following guides.