The Youtuber Brandon F has a 4 part series talking about why they fought like this. Spoiler- it wasn't because they were stupid.
TLDR- if you split up you just get run down by enemy cavalry.
TLDR- a close formation lets you concentrate your firepower at one point.
TLDR- a close formation makes communication and controlling the army much much easier (or even possible at all).
TLDR- the formation makes the troops less likely to run away.