For some time last year I looked into creating a digital coaxial out for audio.
I noted all the semiconductor codes within the box (relevant to audio) and then turned amateur sleuth, searching online, for these and/or similar semiconductor's. Nope. Nothing. Best I got was a pin layout for a Philips MP@2L MPEG decoder (good if you have a Philips digibox?).
Whatever - fact is, its a digibox? Phone Panasonic UK and ask if a schematic exists and they say "no, only SKY have them". Fact is, Panasonic don't have data sheets for even their semiconductors used in their digiboxes.
Fact was, I got bored, and gave up. I imagine the same would go for any other mod. Fact is, unless you have some darn good kit and know what an MPEG datastream looks like AND have the ability to (safely) poke around with it - carry on using the RGB out.
Albeit, for those with a knack for these things - the block below might help. In any event I guess you would be looking to tap in to the datastream at the output of the graphic controller (in many cases this is incorporated into the MPEG decoder) or, at the the input of a suitable video DAC.
At this point, the data will be both decrypted and will be in a state where you can nab it (i.e. as with any standard DAC). Seems simple enough - hark!
Have Fun,
Mark.