Instead of piping the result to a Select statement, pipe it to a ForEach, create a source path from the destination path ($_.FullName), and use that result in an XCopy command, like this:
(gci "E:\Departments\Marketing" -r | ? {$_.PSIsContainer -eq $True}) | ?{$_.GetFileSystemInfos().Count -eq 0} | ForEach {
$Source = ($_.FullName -Replace 'E:\\Departments\Marketing','\\ServerName\SomeDir')
XCopy $Source $_.FullName /s
}
This script works if your original copy operation left empty folders with no subfolders. If you have empty folders within empty folders, this script will need revising.