FASTQ: How Important Is Diversification?

=================
Sometime ago a Redditor with the handle TMN asked me several questions about the balance between diversification and stock screens.  It seemed to me that his screening process, while extensive, likely moved into the realm of unusable because of the number and type of variables under consideration... In essence, he likely screened himself out of candidates.

Here is our e-conversation.

TMN: I want to (re)start investing and I have a question. Don't you think it's more prudent to study one industry or sector in depth and then invest exclusively in that sector or closely related sectors? I know that diversification is important especially if you have a very long-term horizon like I do, but I just can't shake the feeling that you'll never really get it right unless you know everything there is to know, and how can you do that when there are millions of companies operating in hundreds of industries? Besides, if the industry allows surely you can diversify within it? Thoughts?

Me: Diversification is a risk management tool... but there are others... If it's growth you seek, you're on the right track. If you notice, the best traders/investors (best absolute performance) focus on a narrowly defined sector or asset.  To me, that pretty much nullifies the notion that diversification is the Holy Grail.  
[9/2020 Edit > This article pretty much speaks to the reality of underperformance of diversification against concentrated strategies... even though the author makes a pretty stupid comment about day trading at the end...]

In short, match appropriate risk management tools with whatever sector/asset/approach you choose and have at.

Good trading.

In a followup TMN continued...

TMN: Thanks. I prefer to work in a systematic way, so I'll definitely end up placing criteria on my investments. So things like this matter to me:
  • Very low P/E (11>)
  • Must pay dividend with healthy yield and have uninterrupted history of paying said dividends
  • Preferably small-cap or mid-cap
  • Must demonstrate growth-oriented (e.g. mergers, acquisitions, patents, R&D, etc.)
  • Must not be in saturated market
You get the idea. But here's where it gets muddy:
Industry must have strong prospects as well as the company

This sucks because growth is a two-sided coin. On the one hand, I would prefer to be in a growth industry in bullish times like now, and in a mature industry in bearish times. The problem I'm having is in bridging that divide. In other words, I want to have a "narrow" focus like you put it but I feel it would be best to get into two or three different sectors, with varying levels of risk. Say, green tech for high risk, database infrastructure for medium risk, and agriculture for low risk (these are just examples, I know each has challenges). In your opinion, would this be a good idea? Keeping in mind that I want slow, steady growth, but growth nonetheless.

Me: I would be interested to see how many candidates this type of screen presents on a regular basis. There are lots of fundamental and technical crosswinds that may present a problem… 
For example, the criteria you laid out include elements of both value, high growth and mature stocks...which could easily nullify each other.  I.e., most early-stage/small cap companies don't pay a dividend.

It seems to me that a screen that includes all of these elements is most likely to deliver candidates in the midst of a market collapse.  That's not a bad time to have candidates pop up, but it's not exactly a frequent occurrence.

 At any rate, while I understand company fundamentals, they are not my area of expertise, so I will leave that piece to someone better suited to the task. 

As for risk management (which IS my area) there's no reason that your overall approach can’t be effective (assuming sufficient candidates present). Just be sure to give LASER FOCUS to your overall and per trade risk... and manage both appropriately. 

Forever (long term) can be a great holding period... as long as your assets are working on your behalf (moving higher generally)...

Above all else, KIS [keep it simple].

Good trading.