I travelled round France a lot in the 80s, picking fruit. All the unemployed Spaniards back then were in France, picking peaches and cherries for the French farmers who, thanks in part to the CAP, could afford to pay a decent rate. Meanwhile the Spanish farmers paid their own seasonal workers a pittance. Those where exactly the sort of anomolies the EU was supposed to address.
Incidentally, I also spent a summer picking apples in Kent. Those English farms were the worst payers of the lot. And the farmers themselves were mostly large scale and pretty wealthy.
The point is, France found a way to pay their seasonal workers a fair days pay, while preserving small farms and their rural traditions. No peaches rotted on the trees. Solutions are always possible if the will exists to find them.