check why video using the HSYNC is slowing down the system

Unfortunately this is practically unavoidable. The HSYN output has to turn on and then off at a set period during every scanline, including during the vertical sync and blanking period. Currently this timer is disabled when the HSYN callback isn't configured, since many systems use it only to drive the monitor or not at all (since the actual device also generates a composite sync output).

I rather doubt that VSYN is really meant to be inverted. The TMS9927's VSYN output is active high, and the 8085 RST 6.5 input is also triggered at a high level both on actual hardware and in MAME (which generally treats IRQ lines as active high, even though they happen to be active low on actual hardware more often than not).