Fix unsound decoding of `Matrix` and `Vector` from bytes without checking endianness

7 jobs for master in 4 minutes and 27 seconds (queued for 25 seconds)