5.a3 doesn't do anything for White's development, so it would be nice to do without it and spend the tempo on something more useful. In the second place after 5.a3 Black might retreat with ...Be7 instead. Finally after 5.a3 Bxd2+ Black can put all the pawns on black squares (d6, e5). In such a structure the move 4...a5 is useful as it slows down White's queenside attack.
Of course 5.a3 has been played in practice, with depressing results (slightly more than 40%).