I searched high and low but got the answer "this is not possible" or create a GPO that injects your task for the user.
This is not the kind of hack I need, so after digging in deeper, I found the solution.
Basically, when you create a task, it will also say as what user it has to execute. This was set as my admin user obviously as it is a default setting. You can change it to many things including user groups. When I changed it to the group USERS, I got the desired result.
So in the General tab, where it states When running the task use the following user account:
you click Change user or Group
In the next screen, click Advanced...
Click Find now
Scroll down until you see the Users
group (notice it has 2 people instead of 1 indicating it is a group, not a user) Double click it to select it.
Then press OK to use that user, and save your task.
As far as I can tell, this will not launch a program interactively, but when I used it to launch xmpcontrol.exe to control my XMPlay it works for all users.
I created 2 tasks, one to stop playback on disconnect, and one to resume playback on connect, and it works perfectly. (previously it would stop playing, but not be in stopped state while the user is disconnected)