Author avatar

Ravikiran Srinivasulu

Broadcasting Numpy Arrays for Arithmetic Operations

Ravikiran Srinivasulu

  • Oct 29, 2018
  • 7 Min read
  • 673 Views
  • Oct 29, 2018
  • 7 Min read
  • 673 Views
Python
Data
NumPy

Introduction

Before understanding what broadcasting is and how it works, let's understand how arithmetic operations are performed on Numpy arrays. An arithmetic operation between any two arrays is always performed element-by-element. That is if you add two arrays, A and B, every ith element of A is added to the ith element of B to produce the array C.

ArithmeticOperations

Everything works fine if both the arrays have the same shape. If the arrays have different shapes, then the element-by-element operation is not possible. But, in real-world applications, you will rarely come across arrays that have the same shape. So Numpy also provides the ability to do arithmetic operations on arrays with different shapes. That ability is called broadcasting.

The Rules of Broadcasting

Although you can do arithmetic operations on arrays with wide-ranging shapes, there are a few limitations. So, it helps us to know the broadcasting rules before we look at a few examples. In general, you can do arithmetic operations between two arrays of different shapes, if:

  • The size of each dimension is the same, or
  • The size of one of the dimensions is one

And you can use these rules to perform operations even between a ten-dimensional array and a two-dimensional array. The dimensionality of the arrays does not matter.

Let’s look at an example and see how this works. For example, I have two arrays, A and B, below:

1
2
3
A = np.arange(12).reshape(3,4)
B  = np.arange(4)
A
python

Output:

1
2
3
array([[ 0,  1,  2,  3],
       [ 4,  5,  6,  7],
       [ 8,  9, 10, 11]])
python
1
B
python

Output:

1
array([0, 1, 2, 3])
python

And their shapes:

1
2
A.shape
B.shape
python

Output:

1
2
(3, 4)
(4,)
python

Note that array A is a two-dimensional array and array B is a one-dimensional array or a scalar value. When you add these two arrays, Numpy broadcasts the smaller array across the larger array and the operation is successful.

How Broadcasting Works

Broadcasting starts the comparison with the trailing dimension and moves toward the leading dimension.

DissimilarArrays

In the previous example, the size of the trailing dimensions matches, so it proceeds to check the next dimension. Since array B is a one-dimensional array, it does not have a leading dimension. So Numpy automatically broadcasts the value ‘1’ to the missing dimension in array B. So, after array broadcasting:

DissimilarArrays-1

From the image, it is clear that the broadcasting rules are satisfactory and Numpy allows the arithmetic operation between the two arrays. So, if you add them:

1
A+B
python

Output:

1
2
3
array([[ 0,  2,  4,  6],
       [ 4,  6,  8, 10],
       [ 8, 10, 12, 14]])
python

Broadcasting with Two-dimensional Arrays

Instead of a scalar value, let’s create B to be a two-dimensional array.

1
2
B = np.arange(3).reshape(3,1)
B.shape
python

Output:

1
(3, 1)
python

And if you print B:

1
B
python

Output:

1
2
3
array([[0],
       [1],
       [2]])
python

Arrays A and B have shapes (3,4) and (3,1) respectively. The broadcasting rules satisfy each of the dimensions in both the arrays, so the arithmetic operation is possible.

1
A+B
python

Output:

1
2
3
array([[ 0,  1,  2,  3],
       [ 5,  6,  7,  8],
       [10, 11, 12, 13]])
python

Broadcasting with a Three-dimensional Array

I have a three-dimensional Numpy array, A:

1
2
A = np.arange(24).reshape(2,3,4)
A
python

Output:

1
2
3
4
5
6
7
array([[[ 0,  1,  2,  3],
        [ 4,  5,  6,  7],
        [ 8,  9, 10, 11]],

       [[12, 13, 14, 15],
        [16, 17, 18, 19],
        [20, 21, 22, 23]]])
python
1
A.shape
python

Output:

1
(2,3,4)
python

And a one-dimensional array B

1
2
B = np.arange(4)
B
python

Output:

1
array([0, 1, 2, 3])
python
1
B.shape
python

Output:

1
(4,)
python

Here again, the arithmetic operation is possible as Numpy broadcasts the smaller array B to the larger array A. So, after broadcasting, the shapes of arrays A and B become (2,3,4) and (1,1,4) respectively. They follow the conditions for broadcasting and the arithmetic operation is successful.

1
A+B
python

Output:

1
2
3
4
5
6
7
array([[[ 0,  2,  4,  6],
        [ 4,  6,  8, 10],
        [ 8, 10, 12, 14]],

       [[12, 14, 16, 18],
        [16, 18, 20, 22],
        [20, 22, 24, 26]]])
python

Broadcasting with Two Three-dimensional Arrays

I have two three-dimensional arrays, A and B. Array A is the same one as the earlier with shape (2,3,4). Array B is defined as:

1
B = np.arange(8).reshape(2,1,4)
python

And has the shape:

1
B.shape
python

Output:

1
(2,1,4)
python

And has the value:

Output:

1
2
array([[[0, 1, 2, 3]],
       [[4, 5, 6, 7]]])
python

To find out if an arithmetic operation is possible with these two arrays, just revisit the rules. The leading and the trailing dimensions match. And, for the other dimension, one of the dimensions is one. So an arithmetic operation is possible.

1
A+B
python

Output:

1
2
3
4
5
6
7
array([[[ 0,  2,  4,  6],
        [ 4,  6,  8, 10],
        [ 8, 10, 12, 14]],

       [[16, 18, 20, 22],
        [20, 22, 24, 26],
        [24, 26, 28, 30]]])
python

But if you try to add two arrays which cannot be broadcasted due to incompatible shapes, you will get an error.

1
2
B = np.arange(18).reshape(2,3,3)
A+B
python

Output:

1
ValueError: operands could not be broadcast together with shapes (2,3,4) (2,3,3)
python

Here, the issue is with the trailing dimension:

In general, you can look at the shape of the input arrays and decide whether broadcasting will allow the operation to be performed or not. Also, you can predict the shape of the final array which is the highest size along each dimension in the input arrays.

For example, the following array pairs can be used in an arithmetic operation:

(3,2) and (1,) produces the output array (3,2) (3,1,4,1) and (1,7,4,3) produces the output array (3,7,4,3)

The following array pairs cannot be used in an arithmetic operation due to size mismatch in the trailing dimension:

(3,2) and (1,3) (3,1,4,2) and (1,7,4,3)

Conclusion

In the examples, we observed that a smaller array broadcast (or stretches) along the length of the larger array. However, note that this understanding is only conceptual and Numpy does not create any copies of the data. So the broadcasting operations are memory efficient.

8