You could say something like "I apologise for how my club and its supporters have stooped so low that they;re as racist as Chelsea's." So, yes, you could say more.
Why would I need to apologise for what some other idiots have done who happen to support the same club as me? And why would this sort of apology be made towards the Spurs board?
The same happened a few seasons back when they played Luton. Luton were 3-1 up at one stage, and the final score of 5-3 was hardly representative of anything other than Marlon Beresford being a duff keeper.