Ammonites are a group of shelled molluscs, which were very successful during beginning in the Cambrian, with the explosion in the number and diversity of invertebrates. The present-day nautilus is a relative of this formerly successful, and now extinct group. Since the ammonites were shelled, a good fossil record was left behind. This record shows that many taxa which consisted of very many species became extinct early on while taxa which lasted for a long time often arose from minor branches in the evolutionary tree. Ammonites are very popular with fossil collectors. You may have even seen some ammonites at a collectibles shop at some mall. Look at this link for an example of what collectors like to buy!
We have good data from the fossil record on the number and duration on earth of various abundance of ammonite relatives. We see rapid divergence of ammonite relatives at several points in time and, of course, extinction of many of these groups. Note that many types of ammonite ancestors were produced during relatively short periods of time. These points of adaptive radiation often coincided with changed environmental conditions such as the growing of mountain ranges or changed sea levels.
In the case of animal phyla, the arthropods have a very large number of species. A very few other animal phyla have a large number of species (but none have even 1/10 as many as in the arthropods) but many phyla have a very small number of species. We would expect that diversification into many taxa at any given level would be a result of adaptive radiation. However, the presence of many species in just a few taxa indicates a role for progressive occupation.
2. Environmental change. There have been several changes in sea level during the Earth's history. During the Cambrian, more than 500 million years ago, the sea level rose, flooding continental shelves. This provided an opportunity for radiation into this new environment.
3. Archipelagoes. Islands and island groups are isolated habitats - a type of archipelago. (Another type of archipelago would be a mountain isolated in the center of a barren dessert.) Because they are isolated from other habitats, movement into them is a rare event. Typical of these environments is a rare colonization event, followied by rapid divergent evolution. This occurs, because like explotation of the air by birds, there is a lack of predators and competing inviduals and lots of vacant ecological niches. Radiation into archipelagoes requires a diverse habitat to provide the ecological niches - It is unlikely that much adaptive radiation would occur onto a barren island. Examples of adaptive radiation of archipelagoes includes Darwin's finches, Hawaiian honeycreepers and Hawaiian silverswords.
Natural selection, using the raw material provided by mutation, seems to be the fundamental mechanism driving adaptation to new habitats. However, some cases may involve other mechanisms, including genetic drift. In particular, movement between islands may involve genetic drift because of the small number of individuals which can move to a new island. However, radiation into many closely related species is difficult to explain by genetic drift alone.
1. Let's look back at the abundance of ammonite relatives. Note that the most successful groups, in terms of numbers of species within them, weren't always the ones that lasted the longest!
2. Steven J. Gould has discussed the Burgess Shale fauna. The Burgess Shale is a site in Canada which contains soft-bodied fossils dating back to the Cambrian. Soft-bodied organisms sometimes form fossils which look very much like a squashed blackened replica of the original organism. Some people originally thought that the black color was the result of the carbon in the biological structure but it is now believed that the carbon has been replaced by salts in a process not yet fully understood. The Burgess Shale shows many invertebrate taxa which were very common but are now extinct. It seems that some of the animals which were very successful in the Cambrian turned out not to be those which gave rise to the present groups. Among the weird animals found in the Burgess Shale is Wiwaxia a flattened worm-like creature orginally mistakenly placed together with the segmented worms but now known to not fit in any currently recognized phyla. Another wierd creature is Hallucigenia (The name says it all!) which does not fit into any present phyla. In the Cambrian, there were about 6 genera closely related to Wiwaxia but only two of small clams. However, the clams survived and all of the Wiwaxiids became extinct.
The conclusion from these examples is that one cannot seem to predict eventual success of an adaptive radiation based upon the number of taxa in that group. Some taxa with lots of genera or species become extinct and other taxa with few genera or species survive.
It would seem that organisms become more complex with time. The average amount of complexity increases. Gould has argued that this does not mean that evolution is directed towards increased complexity but rather is just the consequence of a simple principle of mathematics: Imagine that one has a simple organism which then radiates into different niches or adaptive zones. Random variation alone will produce more complex and less complex organisms. But, since the complexity of the "ancestral" organism is already very low, there is a limited range of organisms which could exist which would be less complex. There are many variations which would be more complex! So the average complexity of descendents increases, even just considering random variation. One does not have to propose some exotic mechanism involving adaptation to account for increasing average complexity.