Insulation materials vary in their price and properties: thermal performance, strength, adhesiveness, shape, fire- and moisture-resistance.
What to choose, then?
To decide, you have to consider issues such as price and properties, but also the part of the house where the material is going to be installed: attic or vertical roof assembly, underfloor, basement, slab, wall cavities, vertical wall surfaces, ceilings, open floors or open ceilings.
Fiberglass and mineral wool come mostly in the form of batts and blankets. Fiberglass is mostly used to insulate wall and floor cavities.
Mineral wool (dense panels) can also be advantageously used on vertical wall surfaces (insulating sheathing) or to insulate underfloors and slabs.
Overall rigid mineral wool is a better insulation material than fiberglass batts. See: Fiberglass vs. Mineral Wool
Cellulose but also fiberglass, vermiculite and sheep’s wool may come shredded and granulated in order to be installed using a blowing machine.
These loose-fill materials are largely used both in attics and closed wall and floor cavities. Overall, cellulose is the best loose-fill insulation material as long as there isn't a significant risk of moisture. See: Cellulose and Loose Fill Insulation.
Sprayed insulation is another type of insulation; it has an important property that loose-fill materials do not share: adhesiveness, which allows them to be applied in vertical surfaces, or in open ceiling and floor cavities, where loose-fill materials can’t.
Sprayed insulation comprises several foams, some of them with excellent R-values, especially polyurethane and polysocyanurate. They are excellent insulation materials. See: Spray Foam and Fibers Insulation.
Fiberglass and cellulose loose-fill materials are inexpensive and can combine with a mixture of glue and water, in a blowing machine, to be sprayed.
Rigid foam panels are a fourth category (see: Rigid Foam Insulation). They include products like polystyrene and rigid polyso. They are great for foundation walls and to apply on vertical walls, or to insulate underfloors over basements and crawl-spaces..
Foam panels are more expensive than batts, but they have some important properties that can make them a better choice: good air sealing and water resistance, or higher insulation performance (polyso and extruded polystyrene).
See for details:
Spray Foam and Fibers Insulation
Batts are inexpensive and easy to install. Many DIYselfers love them. But loose-fill materials provide a covering without voids and gaps, and a better R-value.
The problem with loose fill materials - from a DIY standpoint - is that they require a blowing machine to be installed.
See, for details:
Fiberglass vs. Mineral Wool
Loose-fill materials are more cost-competitive (they are mostly used in closed cavities, in retrofits).
Sprayed foams stick to surfaces and open cavities, contrary to loose-fill materials. They are an excellent sheathing material (for walls, to avoid heat transfer through the wood, or steel, or other framing materials), and many provide good air sealing and are moisture resistant (contrary to cellulose, the most important loose-fill material).
Rigid foams are advantageous in situations where structural strength and moisture resistance are important.
Sprayed foams (polyurethane, polyso…) are advantageous in situations where adhesiveness is key.
There are a few health concerns about insulation products.
Some types of vermiculite, widely used some decades ago as an insulation material for walls and attics, contained traces of asbestos – a problem in many old homes.
Modern insulation products are much safer.
Besides, insulation materials can now be benign in their environmental impact, especially if we consider their indirect benefits. If these issues interest you, see: Environmentally friendly materials