In this case, Move-Item performs a two-step operation in sequence.
First, it RENAMES the src/foo folder to the dest folder.
Next, it tries to MOVE the src/foo.txt file to the dest folder. However, there is already a foo.txt file in the dest folder and an error occurs.

That is, Move-Item will "move" the item if the destination already exists, but otherwise it will "rename" the item.
I don't know if this behavior is intended or if it's a bug. Please wait for reply from PowerShell Pros. :-(

An easy workaround is to create a dest folder before you run Move-Item. This will allow you to achieve the behavior you expected.

if (Test-Path src){
	remove-item src -force -recurse
}

if (Test-Path dest){
	remove-item dest -force -recurse
}

New-Item -ItemType directory -Path src | out-null
New-Item -ItemType directory -Path src\foo | out-null
New-Item src\foo.txt | out-null
New-Item src\foo\foo.txt | out-null

# Create a dest folder explicitly
New-Item -ItemType directory -Path dest | out-null

# Then, move items
move-item -path "src\*" -Destination "dest"

Couldn't Move-Item be modified to first create the destination if it doesn't exist? If it did, would that solve the original problem?

I see we need to review move-item design. I don't think it works as such alternative tools.
/cc @mklement0 if you have an interest.

We could follow the lead of the Unix mv utility:

• If there are multiple source items, the target item must (a) exist and (b) must be a directory - otherwise an error occurs (that is, @mkht's workaround is the proper solution to avoid this problem).

Additionally, @jdhitsolutions' proposal - creation of a target folder on demand (unless a file by that name already exists) - should probably be opt-in, via a switch such as -Force (though making -Force do many different things is problematic, I'm not aware of anything that it currently does in the file-system provider, so perhaps that is an option),