Full dispersion of the surface modes of a 3D topological insulator with the transfer matrix method