Black phosphorus is a layered semiconductor with an intriguing in-plane anisotropic structure. We studied its lattice response to photoexcitation using femtosecond electron diffraction and found that the anisotropic structure impacts the evolution of the atomic vibrations. After photoexcitation, the lattice remains in a nonthermal state up to about 60 picoseconds, which is characterized by less anisotropic atomic vibrations compared to equilibrium. Our results provide timescales for electron-phonon and phonon-phonon thermalization in black phosphorus and show that in the presence of an anisotropic crystal structure, nonthermal phonon populations can transiently change the anisotropy of the atomic vibrations. More information is available here and a video presentation is available here.