Hacking RAM image works as intended - and that is the only good news I bring. Dreamcast DSP output is stable, so it must be a bug.
This is what I get - upper track is Dreamcast DSP, lower is my player. I've ommitted AO because it's pretty much the same (there's slight variance on the time line but that's my code still lagging a tiny bit).
EDIT: And one more, perhaps important thing. This wave comes mostly from ADPCP type 2 looped sample. If the decoder adds an offset and the output shifts too much away from zero it could be the source of the problem. I'll need to test it some more though. Do note that the very beginnig of the sample seems to be OK, and the moment it goes wrong (dives down) is, if I'm not mistaken, loop jump.