Should all attributes in Insight be indexed?

[Alex Mclean]

Since setting up Insight and adding several large object schemas, reindexing after a restart has started to take an exceedingly long amount of time.

So I'm wondering, should I have all attributes indexed?
That's the default setting and that's how it's currently configured.

If an attribute is probably never going to be searched or placed in an IQL query, maybe I should turn off indexing for that attribute. Would that reduce the reindexing time at all? And would that be detrimental to system load if somebody does search one of the attributes I've stopped indexing? I have plenty of attributes that should certainly be present, but it's quite unlikely they'd ever be searched.

Just looking for best practice, recommended usage.
Would this help? Make things worse? Have unintended effects?