Meandering down the rubble towards the water, I ended up at the drawbridge, which I assume was swallowed by the two day long fire in 1969. The ravine below seemed to have been slated to become an actual moat (a sign declares this as Moat Brae), but it was quite dry.
Also located here was a portcullis, a medieval style gate that lowered down to block the entrance to the arsenal buildings. The white structure on the right was known as the Portal Lodge, and is reputedly built on the foundation of an old tug boat. A rusted metal capstan (an object that can be rotated to wind rope) can be seen at the water's edge, although its purpose was purely ornamental.
Here's a 1945 photo taken from the water; the contemporary shot above was taken under the portcullis, seen on the left side. Note the rows of canons along the sea wall.