The location of the Imperial Palace in Tokyo includes some of the Edo Castle remains that the Tokugawa Shogunate resided in until the end of the Edo Period.

The Honmaru (inner citadel), Ninomaru, Sannomaru, (outer citadels) and Nishinomaru (western citadel) were located within the inner moat of Edo Castle. Within the western portion were the Fukiage Gardens. The Fukiage Gardens once held residences, but after the Great Fire of the Meireki Era (1657), no buildings were constructed so the land could act as a future fire blockade.

Facilities for various court functions such as the Palace Hall and Imperial Household Agency office are located in the kyu-Nishinomaru and the royal family lives in the location of the Fukiage Gardens, now called the "Fukiage Imperial Gardens". The kyu-Nishinomaru and the Fukiage Imperial Gardens are separated by the Doukan Moat.

The parts consisting of the old Edo Castle are only the Honmaru, Ninomaru, Sannomaru, and Nishinomaru. The western portion that is separated by the Doukan Moat is not a part of the old fortress, thus where the royal family currently resides is not within the actual grounds of the former castle.

Despite the palace being located within Central Tokyo, the lushly green areas surrounding the castle moat are a popular jogging spot. As national property, the Imperial Palace has been estimated at a worth of 214,644,870,000 yen. (based on documents of the Financial Ministry as of May, 2009)