Non-payment due to non-compete?

I sold a small business and accepted a payment plan from the buyer. I also signed a non-compete clause that I would not do business in the field for a year. Though I have complied with the non-compete, they seem to think that I violated it and are now refusing to pay me my other 2 installments. No where in the original contract does it say they can void the payments based on an accusation of non-compete. Do they have to take me to court over the non-compete or will I have to sue them for my money? What can I do? They also have a late payment penalty so if they don't pay me on time, interested starts to accrue. Should I just let it accrue and then sue them after the final payment date or do I have some responsibility to reply to their decision not to pay me? Thanks to all who respond.