Obviously your grinder is really important too. It is generally regarded as the most important piece of coffee making equipment. Its ability to grind coffee into uniformly-sized particles is paramount to the flavour. Everything else is "an accessory to the grinder".
Briefly, and as far as I understand, the reason is because lower quality grinders smash the coffee into pieces of various sizes. You end up with 'bricks and dust'. This affects the flavour because smaller particles will be over-extracted, causing bitterness, and larger particles don't get as much flavour extracted from them. It also affects the smoothness of the flow of water through the coffee. Lots of odd sized particles won't sit on top of eachother tightly, instead there will be a load of gaps. Instead of going through at a regular rate, the water will spurt and dribble through gaps, not properly extracting flavour from the grounds.
I think ceramic, conical burr grinders are thought to be the best. Grinder will obviously allowyou to adjust the particle size of the coffee grounds, which affects the rate at which the water in the machine passes through them and how much flavour/bitterness is extracted from them.
Also, it's important that the machine allows manual control of water temperature and pressure (the pressure will also determine the amount of time that the water is in contact with the grounds), as both of these variables have a huge effect on the coffee's flavour.