Memory may be ailing but I remember it as being to the left of Monument Bridge at the top of Whitefriargate.
The building itself is splendid, it's the monstrosity next to it that needs to go. No, not the art gallery
The offices above the shop were at one time the HQ for the British Transport police, Humber Ports (they policed the docks till withdrawn in the 1980's).
Interesting that you think the fishery was the other side of the bridge. Hopefully this link will work. It's from the '80's and despite the photo being fuzzy it seems that the fisheries were to the right of the building. Perhaps they moved. Taken from Hull-The Good Old Days. Sorry link not work. Shop next door seems to be Finlays.