> This makes me wonder why Git can't just store an empty tree for empty dirs.

tl;dr: it can (see my other comment) and the empty tree is hardcoded. But since the index works with file paths and blobs, having no file means that there's no entry in the index