Feta is a Protected Designation of Origin (PDO) product from Greece. The original feta is either made of sheep milk entirely or mixes with (up to 30% of) goat milk. With that, it gives feta a unique taste apart from other cow milk cheeses, such as mozzarella and parmesan.
Sheep milk does not have the strong taste like goat milk, but it has richer and buttery taste.
Like other aged cheese, feta comes with a unique mild umami taste. The taste is often described as savory — salty, spicy, and tangy. In fact, the longer it aged, the stronger the taste. When making the cheese, the lamb rennet will be added for the curd to aggregate. In particular, the rennet contains protein-cutting enzymes which later cut the peptides off from the casein (milk protein). The peptides are what give the bitterness for the cheese.
Other than that, the bitterness can also be contributed by glutamic acids. As the maturation of cheese involves breaking down of casein, which later leads to an increase of glutamic acid in the cheese — thus bitterness (1).
On the other hand, the salty taste comes from one of the steps in feta making, which is to brine the cheese for days, or sometimes months. After all, the taste of feta cheese goes well with the age.
Do you know why feta cheese smell like feet? Read the answer here.