LSIF indexing performance degraded

[gundermanc]

Version Used: 4.7.0-3.23316.3

Steps to Reproduce:

1. Use the LSIF generator on bin logs from http://devdiv.visualstudio.com/DevDiv/_git/VS-Platform repo.
2. Compared to earlier versions, indexing seems to take 2-4X as long.
3. You can see the impact here in this build: https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=7962757&view=logs&j=0bc77094-9fcd-5c38-f6e4-27d2ae131589&t=ebb084b3-ee0b-5e49-8d20-930a88149ff4. Indexing goes from a ~5 min step in earlier builds to 20+ minutes in this one.

[gundermanc]

**cc: ** @jasonmalinowski

[jasonmalinowski]

@gundermanc To avoid any confusion, what was the prior version?

[gundermanc]

I think 4.7.0-1.23218.1

[jasonmalinowski]

At least one cause for this regression was we weren't reading XML documentation comment files before when in binlog mode; this was accidentally fixed in 70e160b when we started creating references that used our XML documentation comment reader. It turns out that it's not fast, but it's even more not fast when it's rereading and reparsing stuff each time we process a project. For repos with lots of small projects (like VS-Platform) this ended badly.