Imaam Ibnul Qayyim (rahimahullaah) said: When you confuse one of these groups with the other, you will find yourself in evil.
The First of them:
They are those whose company is like nourishment that one cannot do without in the morning and evening. One leaves it after fulfilling his need from it, and then returns to it when he is in need. This is how it is on a permanent basis (with such nourishment). This similitude is more precious than red brimstone. The ones (intended in this similitude) are those who know Allaah, His commandments, the plots of His enemies, the diseases of the hearts and their remedies. They are sincere to Allaah, His Messenger and His creation. In this similitude (regarding) keeping their company, there is an all-embracing benefit.
The second category:
They are those whose company is similar to medication that is needed during illness. So, as long as you are healthy, you are not in need of intermingling with them. They are those you cannot do without in (seeking after) the beneficial things of livelihood and fulfilment of various dealings, partnerships, consultation, medication and the likes.
The Third category:
They are those whose company is similar to the diseases in their various classes and types, and the differences in their severity and mildness. Amongst them is one whose company is similar to chronic illness. It is this one who does not benefit you-neither in the religious nor worldly affairs, alongside the fact that you might either incur a loss in both your religious and worldly affairs or in one of them. So if intermingling with him is made possible, then it is that disease that is feared will lead to death. And amongst them is the one whose company is similar to a tooth ache whose pain becomes severe upon you, but when it leaves you the pain abates.
The Forth category:
It is the one whose company is absolute destruction and intermingling with him is similar to eating poison. How common is this similitude within the people! May Allaah not make them numerous, and they are Ahlul Bidah Wad Dalaalah (The people of innovation and misguidance). They are those who hinder (people) from the path of the Messenger (sallal-laahu-alayhi-wasallam) and call to that which is in opposition to it. They are those who hinder (people) from the path of Allaah and seek to make it cooked; so they present innovation as Sunnah and Sunnah as innovation, and they present virtue as evil and evil as virtue. And when you established Tawheed amongst them, they say: You have degraded the Awliyaa (friends of Allaah) and the righteous people. And when you establish ittibaa (correct adherence to the path of the messenger with evidence) they say: You have invalidated (the affair) of the followed Imaams. And when you command that which Allaah and His Messenger command and forbid that which Allaah and His Messenger forbid, they say: You are from those who put people to trial.
[Source: بدائع التفسير الجامع لما فسره الإمام ابن القيم الجوزيةVol: 3, Page: 467-469]
via Talibul 'ilm