I have heard that before, but I don't really get it. That could be down to some of the 'rules' of the English language being mongrelised when they tried to write a set of rules for pretty much Germanic words but applying French rules...or something.
It's the office you go to to register hatched, matched and dispatched, so in that respect I get it, but those details are added to the registry, so it's also the office where the registry is kept.
In my mind, it can be both, but most use registry, which makes it correct as far as the rules on English Language go.
Back on topic, our relaxed use of the rules adds to some of the issues found when researching family trees. Many names have a host of different spellings, as quite often the person themselves was barely literate, so it depended what the registrar heard and how they chose to spell it.