With our current hlsl ntsc shader I get this somewhat better output.
With some more tweaking of the initial angle, the colors could probably be a bit better.
Adjusting the hue (and the phase of the pixel grid relative to the colour burst) will only get you so far. To accurately reproduce the colours you get from doing CGA composite artifacting with arbitrary direct colours, you will need to take into account the analogue behaviour of the CGA's chroma multiplexer chip (it's a digital chip, but for these colours the analogue characteristics matter).
One way to do this (the best one I've found) is to generate the output signal from this chip sample by sample (at a sample rate of 14.318MHz). Each output sample depends on three things: the direct colour of the (14.318MHz) pixel, the direct colour of the previous pixel and the phase (of the four possible pixel phases relative to the colour burst). Then the multiplexer behaviour can be completely characterized by a 256 element table (8 colours for current pixel times 8 colours for previous pixel times 4 phases).
I've made some careful measurements of the output from my CGA to populate this table (a bit like dumping a ROM, but analogue instead of digital). The table (and algorithm) can be found in the patch I made for DOSBox (latest version is at http://www.vogons.org/viewtopic.php?f=41&t=12319&start=640#p431721
) or (if you'd prefer to avoid patches to GPLed code) on my github at https://github.com/reenigne/reenigne/blob/master/8088/cga/calibrate/cga2ntsc/cga2ntsc.cpp
(all the code there is released under UNLICENSE so can be re-released under whatever license you like).
To get the final composite output, you need a weighted sum of the chroma multiplexer output and the intensity bit. For new CGA you also need a weighted sum of the red, green and blue bits as well. A nice thing about using a 256 element table instead of a 1024 element table is that the same table can be used for both old and new CGA. You can see the weightings in both the links above.
Feel free to ask me any questions about this code - I'm happy to help add it to any emulator that wants it (although it's a long time since I've touched the MAME/MESS source).