When I remote into my work computer from home, I get an error message about 'windows security log is full'. Once I click OK, everything seems to work fine.
Is this something I should be worried about? how do I make it not happen?
Whereas you won't be adversely affected by it from a performance standpoint, you will not receive security notifications that get sent to that event log which can be worrisome.
You can go into Computer Management, expand the Event Viewer, right-click on the Security event log and set properties for the log size there which can make that notification go away. I'd suggest Overwrite events as needed in your case.
You might want to find out what events are filling up the log. Increasing the log file size and setting it to overwrite as needed will help capture the events. Usually you shouldn’t fill up the security log that quickly.
I've seen that if you have Audit Process Tracking enabled in the local computer policy it will flood your security log with events.
Run gpedit.msc Computer Configuration > Windows Settings > Security Settings > Local Policies > Audit Policy > Audit process tracking