Fix unsound decoding of `Matrix` and `Vector` from bytes without checking endianness

7 jobs for master in 4 minutes and 26 seconds (queued for 28 seconds)