Tolerance of other religions

When discussing the sabians, I came across this bit on the wikipedia page:

The Saabi`ah Mushrikun were a Yazidan group who were questioned by the caliph al-Ma'mún of Baghdad in 830 CE, according to Abú-Jusúf Abshaa'al-Qathíí, about what protected religion they belonged to. Not being Muslim, Christian, Jewish, or Magian, the caliph told them they were infidels and would have to become Muslims or adherents of one of the other religions recognized by the Qu'ran by the time he returned from his campaign against the Byzantines or he would kill them.

So... are only the "people of the book" tolerated, or was the guy in the wrong? What is the bigger picture?