Wine sediment can form naturally in wine both during the fermentation process and while maturing in a wine bottle. Some wines are more likely to develop sediment and some wines will almost never form sediment.
Wine sediment isn't harmful and can be perceived as a sign of wine's quality, but you will normally want to separate the sediment from the wine before serving and drinking.
The initial sediment which forms in wine appears during the fermentation process and is called 'lees'. The lees sediment consists of dead yeast cells, proteins, stems, bits of skin, and other solid matter that has settled to the bottom of the fermentation tank or oak barrel (as shown).

 

In wine barrels more lees sediment can form, so it's common for wine to be racked out of the first barrel and into a second to separate it from the lees sediment, this process might occur several times depending on the wine. When people think about sediment in wine, they probably think about the formation of sediment in wine bottles that you have to take care to remove before serving.
The sediment which develops in red wine bottles and which you need to carefully remove is formed from tannins and other solid matter that gradually falls to the bottom (or side, if you are storing a cork closure wine). The presence of this material helps give the wine character and complexity, but you don't want to leave it in the wine when serving.
It is possible that a young wine will show some form of sediment, particularly unfiltered wines. It is important to know that this is not an indication of a fault. Even white wines can leave a deposit of harmless tartrate crystals inside the bottle, which is simply an indication that the wine was exposed to very cold temperatures after bottling.
Decanting a wine will clear the wine of sediment; allow the bottle to stand undisturbed in advance for about 24 hours, to allow the microscopic solids to settle at the bottom of the bottle. Do not shake or vibrate the bottle before opening, as this will stir up the sediment and make decanting difficult.