There is no option/feature to limit the number of boxes checked. It could be a feature request for this field. Meanwhile, you need to create some custom JavaScript code to do that.
I'll add my voice:
It would be great if this could be added to the roadmap at Trello as a feature request. I do it with JavaScript now, but it is a little time-consuming and inconvenient. It would be nice to have an option to limit the number of choices... Often it is necessary.
.
