MdbWriter caches the MVID to write upon construction of the writer from the module's Mvid property. But in case of deterministic MVID the actual MVID is computed after the MdbWriter is constructed. So the symbols end up using the wrong (Typically all zeroes) MVID.
I looked at how PortablePdb does this, and the writer in this case holds on to the ModuleDefinition and reads its Mvid property only when actually writing the symbols (which is after it was populated with the right value).