VF3's FM is... interesting. It looks like all FM samples used have STWINH set and that the driver itself writes values directly into the ring buffer for FM processing.
After trying various things, it appears that the effective modulation being applied is currently too low. Increasing the MDL gain by a factor of 2 improves things slightly. If I tweak the gain slightly further to 2.2, I get something that closely matches the PCB recordings.
So it seems that either the values that the driver is writing to the FM buffer are too low (I wonder if the driver reads any SCSP registers to generate these - if it doesn't, then I wouldn't expect this to be the problem) or that the modulation gain is something different than an power of 2 of the MDL value.
Setting FM_DELAY back to 2 makes most of them at least reasonable but that's clearly not the only problem.
That's actually a side-effect from me forgetting to check for STWINH when writing the delayed samples into the FM ring buffer.