Yes, of course it is impossible not to intrude, but we should not shy away from the hard decisions just because it's a grey area - and by hard decisions I mean as much when not to increase laws restricting freedom "for the greater good" as when to introduce them.
The speed limit is an interesting example. As mentioned, people continue to break speed limits anyway (perhaps because they think they won't get caught). However, it also cannot be argued that for example in a built up area, a speed limit of 20mph would save more lives than restricting it to 30mph. Frankly a speed limit of 10mph, enforced by average speed cameras on every corner would be even safer. I would even argue that 10mph speed limits in cities would save even more lives than just about any anti-terrorism legislation ever proposed. But whether we think about it this way or not, we have accepted that 30mph is a balance between safety and restricting ability to get places in any reasonable time. In other words we've offset convenience and commercial need against lives. And of course traffic accidents account for many multiples more death than terrorism (indeed in the news at the moment is the sad case of five people being killed in a single crash overnight).