Caradhras (Redhorn), Celebdil (Silvertine) and Fanuidhol (Cloudyhead) were called the Mountains of Moria. But the actual city of Khazad-dûm was built under the east side of Silvertine. The East Gate at Mirrormere was the main gate and all the great halls of the city were quite close to it. The passage to the west side and the West Gate were probably made in early second age and mainly used for trading with the elves of Eregion. There was a close relation between the elves of Eregion (mainly Noldor) and the Dwarfs of Khazad-dûm. Celebrimbor of Eregion who made the Three Rings also made the inscription at the west Gate "Speak 'Friend' and enter". But the dwarfs did not live there but only near the east gate:

The Fellowship of the Ring - A Journey in the Dark said:

Gandalf seemed pleased (when they reached the 21st hall). 'I chose the right way', he said. 'At last we are coming to the habitable parts, and I guess we are not far from the eastern side.

Click to expand...

A bit later Gandalf explains that below Redhorn there were the Mithril mines:

The Fellowship of the Ring - A Journey in the Dark said:

The lodes lead away north towards Caradhras, and down into darkness. ... they delved too greedily and too deep, and disturbed that from which they fled, Durins Bane.

Click to expand...

I can't find anything about mines below Cloudyhead but I believe the term "Mountains of Moria" doesn't refer to the fact that Moria was build under all these Mountains but rather that these were three very promintent peaks which made it easy to spot the location of Moria from far away.