These two elements both come from the hop cone, the flower of the female hop plant.
The bitterness comes from alpha acid. The hops intended to bitter a beer are added during the boil. They are added early so the heat from the boiling wort has time to break down or isomerize the acid. This iso-alpha acid is even more bitter and, unlike the alpha acid, soluble. It is therefore dissolved into the wort and carried into the final beer.
Hops bitterness in beer is measured by International Bittering Units (IBU). One IBU equals about one milligram of iso-alpha acid per one liter of beer. That is almost completely meaningless so here are some common beer styles and their IBU range: Wheat - 10-12 American Pale Ale - 20-40 Pilsner - 30-40 IPA - 40 +
Bittering hops contribute no aroma and no flavor besides bitterness. The oils from the cone that produce the distinctive hops aroma are boiled away in ten to thirty minutes depending on the variety of hops.
To capture the essential oils of aroma or flavoring hops brewers add them toward the end of the boil or even after the boil, a practice called dry hopping. Added at this point hops contribute to the final beer's aroma, taste and even mouthfeel.
Another important historical quality of hops is their preservative ability. When hops first starting being used in beer, brewers quickly learned that they prevented many air and water-borne bacteria from infecting their beer.