A bit of each in my case. When I moved from Scotland to England, I had a rather broad Scots accent which attracted some rather nasty bullying at school - from kids and teachers alike, so I soon lost it, but quickly reverted on visits back there. Moving on the Australia, it took a few years longer to lose the English accent I had acquired - there being very little abuse over it and what there was was always more a case of good natured ribbing - but it did change, although certain vowel sounds still give me away. Having been back in Scotland for eight years now, I am still easily recognised as Australian as my accent hasn't reverted at all.
Although I have been accused of being a Kiwi a few times.