As always, it's all about having the right tool for the job. Without tools it's impossible, and with better and better tools, it becomes easier and easier.
You're not doing badly so far, but you seem to have drilled some of the holes quite far from the line. That didn't help.
Option 1. Keep drilling, preferably with holes nearer the line. Use side cutters to damage the metal between the holes, then man-handle the metal out with a screwdriver and / or hammer.
Clamp your panel upright in a vice, and use a coarse file to square up the hole.
Option 2. Drill a large hole at two opposite corners, and use a small saw blade to cut each edge. Then file it to shape.
Option 3. Use a dremel with circular cutting disc, then file to shape.
Option 4. If you want to spend some money on tools, you can do a lovely job with a hand operated milling machine. You can get nice little ones on eBay for a few hundred bucks.