python - numpy broadcasting - explanation of trailing axes

Question

Question

Please elaborate the answer in Numpy array broadcasting rules in 2012, and clarify what trailing axes are, as I am not sure which "linked documentation page" the answer refers to. Perhaps it has changed in the last 8 years.

As axes in trailing axes is plural, at least two last axes sizes must match (except sigular)? If so why at least two?

The given answer was:

Well, the meaning of trailing axes is explained on the linked documentation page. If you have two arrays with different dimensions number, say one 1x2x3 and other 2x3, then you compare only the trailing common dimensions, in this case 2x3. But if both your arrays are two-dimensional, then their corresponding sizes have to be either equal or one of them has to be 1.

In your case you have a 2x2 and 4x2 and 4 != 2 and neither 4 or 2 equals 1, so this doesn't work.

The error and the question raised were:

A = np.array([[1,2],[3,4]])
B = np.array([[2,3],[4,6],[6,9],[8,12]])
print("A.shape {}".format(A.shape))
print("B.shape {}".format(B.shape))
A*B
---
A.shape (2, 2)              # <---- The last axis size is 2 in both shapes.
B.shape (4, 2)              # <---- Apparently this "2" is not the size of trailing axis/axes

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-91-7a3f7e97944d> in <module>
      3 print("A.shape {}".format(A.shape))
      4 print("B.shape {}".format(B.shape))
----> 5 A*B

ValueError: operands could not be broadcast together with shapes (2,2) (4,2) 


Since both A and B have two columns, I would have thought this would work. 
So, I'm probably misunderstanding something here about the term "trailing axis", 
and how it applies to N-dimensional arrays.

---

References

• Array Broadcasting in Numpy

The Broadcasting Rule
In order to broadcast, the size of the trailing axes for both arrays in an operation must either be the same size or one of them must be one.

• Broadcasting

---

Update

Understanding based on the reply by @Akshay Sehgal. Consider 2 arrays A.shape = (4,5,1) and B.shape = (1,2).

A = np.arange(20).reshape((4, 5, 1))
B = np.arange(2).reshape((1,2))
print("A.shape {}".format(A.shape))
print("B.shape {}".format(B.shape))
---
A.shape (4, 5, 1)
B.shape (1, 2)

Firstly, look at axis=-1 and the shape 01 in A is broadcast from 01 to 02, because it is singular, to match that of B. Then shape 01 in B for axis=-2 is broadcast from 01 (singular) to 05 to match that of A. The result is shape (4, 5, 2).

print("A * B shape is {}".format((A*B).shape))
---
A * B shape is (4, 5, 2)

Based on the answer from @hpaulj, a way to simulate broadcasting.

print("A.shape {}".format(A.shape))
print("B.shape {}".format(B.shape))
---
A.shape (4, 5, 1)
B.shape (1, 2)

# Check ranks.
print("rank(A) {} rank(B) {}".format(A.ndim, B.ndim))
---
rank(A) 3 rank(B) 2

# Expand B because rank(B) < rank(A).
B = B[
    None,
    ::
]
B.shape
---
(1, 1, 2)

A:(4,5,1)
   ↑ ↑ ↓
B:(1,1,2)
----------
C:(4,5,2)

Answer 1

Trailing axes are axis=-1, axis=-2, axis=-3 ... . Broadcasting rules compare trailing axes as opposed to leading axes (axis=0 onwards).

This is specifically for applying broadcasting to different dimensional tensors (say 2D and 3D tensors). Trailing axes basically indicates the direction in which axes are considered for broadcasting rules. Imagine lining up the axes by shape. If you lead with axes you would have something like the following -

Consider 2 arrays A.shape = (4,5,1) and B.shape = (1,2)

#Leading axes

A  04  05  01
B  01  02
--------------
No broadcasting
--------------

To consider trailing axes you would instead look at them as -

#Trailing axes

A  04  05  01
B      01  02
--------------
C  04  05  02
--------------

That's all they mean with the term trailing axes in this context i.e. start backward instead of leading axes.

In other words, when considering to broadcast a (1,2) shaped array with say a higher dimensional array, we look at the trailing axes which are shaped 2 for axis=-1 and then 1 for axis=-2 in the reverse order.