Hi if I understand correctly, I also have this issue.
I believe OP means "more than 300 terms in a (any) custom taxonomy"
And when editing (or creating a new field group) it loads very slow.
Presumably because it's loading those terms for taxonomy-type fields, but this happens even when no taxonomy (or any) fields have been added to the field group.