The Red House is an imposing and visually distinctive building with its heavily moulded pillars, arched window frames and ornate embellishment of the brickwork along the ridges of the roof.

The four-storey high main building seen here was not part of the original structure of the first Government building was erected on this site in the 1840's. Then, the northern and southern blocks were joined by an archway that allowed free passage to both pedestrians and wheeled traffic along what was then Prince Street. This was a stipulation of the City Council, that the passage should never be closed to the public.