Sometimes referred to as Fremantle Gaol or Fremantle Jail, this former prison was built between 1852 and 1859 on about 36 acres (15 ha) from limestone quarried on-site. A 15-foot (4.6 m) tall boundary wall encloses the prison grounds, with a gatehouse in the centre of the western wall, facing The Terrace. Other roads bounding the site are Knutsford Street to the north, Hampton Road to the east, & Fothergill Street to the south. 7 Cottages, which housed prison workers and officials, are located outside the wall either side of the gatehouse. Inside the walls, the parade ground is located east of the gatehouse. Beyond it is the Main Cell Block at the centre of the site, which contains two chapels. North of the main block is New Division, and west of that, in the north-western corner, is the former Women's Prison,previously the cookhouse, bakehouse and laundry. The hospital building stands in the north-eastern corner, while the former workshops are located in the south-eastern corner, as well as to the north of the gatehouse. A system of underground tunnels, constructed to provide fresh water from an aquifer, runs under the eastern edge of the site. Initially known as the Convict Establishment or The Establishment, it was constructed as a prison for convicts, using convict labour, between 1851 and 1859. The prison closed in 1991.

Fremantle Prison operates tours daily between 9-5pm. Some filming has been allowed previously during these times. A comprehensive filming policy applies for all filming onsite at Fremantle Prison.