As has been stated elsewhere, this would be best resolved by having a policy in place to address the issue. Yes, it is understandable that as a new company you've not set policies for every circumstance ... as a recent hire by a small, new company myself, I'm encountering some of the same thing.
Although you may have already formed an opinion about this new person's work it's best to ignore that when formulating the policy. That's because the next person may be the opposite of this one in terms of likeability, performance, etc., but the policy has to treat them equally.
Since you say there's no legal issue, it seems to me that this really comes down to you (and your partners, if any), and your employment market. The biggest questions in my mind would be:
What do other organizations who hire workers similar to your employees do? If they provide pay for jury duty, then people will expect to be paid for such. If word gets out that you do not provide jury duty leave, it may make it more difficult for you to hire people.
What kind of benefits do you want to provide as an employer? This may be a personal issue (e.g. maybe you want to encourage people to fulfill their civic responsibilities), or a competitive one (making it easier to hire the people you want to employ).