If the ESXi Hosts in different Clusters were on different VLANs, you will could have problems. You would see something like this with DEV vs TEST vs PROD designs. All the DEV Clusters are on VLAN 1, 2, & 3 while all the PROD Clusters are on VLAN 4, 5, & 6. Management of the vDS is easier at that point with each cluster getting its own vDS.
Personally I have done it both ways with complete success. Single vDS for multiple clusters and Multiple vDS for multiple clusters. It comes down to Business Needs/Requirements and level risk in your configurations.
Thanks,
Boston Tech Guy