Analogue Nagravision (Syster) encoder

[fsphil]

It's a very slow python program. I was going to rewrite it in C but to be honest, we're not in a hurry - the python version is fine. I'll tidy it up a bit and publish it.

How is your video capture device connected to the VCR, RF or composite? Just wondering if it could be improved that way.

Unfortunately, I can see the top of the picture located where the bottom half of the VBI, it seems the top half of the VBI has been truncated by my friend's VCR and everything is shifted upwards

Oh that's really odd. I'm not sure how that could happen, or be fixed.

 

[homercartman]

@Captain Jack Just FYI, the same document, in French, which seems to have been originally filed 04/14/1987 (The initial US patent was filed 04/15/1987 with Ser. No. 07/038,727)
If you feel the US document bears a typical heaviness of literal French-to-English translation, the French document sounds sometimes even heavier, sometimes simpler.

@fsphil My VCR is connected to the capture device via SCART to composite+stereo adapter.

 

[homercartman]

@fsphil @Captain Jack

Some updated documents from the same author as vbi_nagra.pdf (Zebulon)
(for the record, vbi_nagra.pdf)

It's here:
Canal_teletexte_LIGNE03.zip and 1 more file

you'll find:

• Canal_teletexte_LIGNE03.zip where there are updated considerations on the Syster VBI lines, including frames upgrading the internal flash, detailed semantics of the now known audience mode (aka vbi[5]), position and timing of the CW challenge ("la question"), frames dedicated to internal flash update, recurring 38 seconds frame periodicity, potentially discret-related stuff... In a word: an updated version of vbi_nagra.pdf
• Canal_teletexte_Zeb031297.zip where there are VBI captures from French "Paris Premiere" satellite channel and French terrestrial Canal+. You'll also find other things you might recognize that might be of lesser interest here. All .TXT files seem important, yet some exploration of the rest of the archive is valuable too for the sake of acknowleding the research that has already been done.

Note: regarding the Antiope-Didon encoded frames related by the US and EU patents above, you'll see Zebulon tried to suppress them with a monostable and it didn't seem to impact the decoder.

Enjoy.

 

[fsphil]

Thanks for these. I've only had a quick look but I see they had already discovered the XOR error checking we just recently found. This look good!

 

[Captain Jack]

Wow excellent info! We had the vbi_info file already but not the others.

 

[fsphil]

I tried extracting some data from those VBI samples but I was only able to get a small percentage of the lines. I wonder if the recording itself is the issue or the VHS player.

Some other progress lately, thanks to @Captain Jack capturing lots of data we where able to figure out how the PRBS works. This is what generates the R and S values that are used to scramble the image. It ended up being very simliar to the PRBS used in D/D2-MAC and Videocrypt.

One thing that's still a mystery is the EMM message checksum:

49 FE 28 B1 00 00 00 00 00 00 00 00 00 FF FF FF FF 44 55 4D 4D 59 'I.(..............DUMMY'
A1 45 4D 4D 44 55 4D 4D 59 45 4D 4D 44 55 4D 4D 59 45 4D 4D 44 55 '.EMMDUMMYEMMDUMMYEMMDU'
5E 4D 4D 59 45 4D 4D 44 55 4D 4D 59 45 4D 4D 44 55 4D 4D 59 45 4D '^MMYEMMDUMMYEMMDUMMYEM'
B6 4D 44 55 4D 4D 59 45 4D 4D 44 55 4D 4D 59 45 4D 4D 9E 4D DC F0 '.MDUMMYEMMDUMMYEMM.M..'

The numbers in red at the end are a checksum, the PDF mentions them but doesn't describe how they're calculated.

 

[fsphil]

Here's a copy of the VBI extractor. Please excuse the code quality, it was thrown together quite quickly. Usage example:

$ ./syster_vbi_dump.py -i ~/Downloads/c+_france_tape_1_10mn_v1.dat -w 720 -s 13500000

The -w/--width and -s/--samplerate arguments are the values from the capture card. There is some filtering on it but it will still pass some incorrect lines. There is a filter to rescale the data but I'm not sure if that's even helping. There is plenty of room for improvement.

Download: https://www.sanslogic.co.uk/hacktv/syster_vbi_dump.py

 

[fsphil]

I just realised your first VBI sample has two independent sets of lines. I was wondering why I was seeing the same sequence numbers close together, thought my extractor was faulty. Apparently the subscriber key selects which set to use.

 

[fsphil]

how can I retransmit captured full VBI data (using TBC) in hacktv correctly?

Sorry, I missed this comment. There's no simple way but if you don't mind modifying the program you can just change the data going into vbidata_render_nrz() to something you've captured from an external source.

 

[homercartman]

@fsphil

Thanks for your investigation and for the python script.
For maintainability, would you consider adding it in hacktv repo?

Just wondering out of the blue if this checksum is 32 bits or 2x 16 bits?
Something similar to the patent @Captain Jack uploaded , like
Crc16(L49 xor LA1), crc16(L5E xor (LB6 + pad/init)) ?

Yes there are two sets of lines in the terrestrial C+.
This is what Zebulon mentioned in the updated documents I shared earlier.
You can also see two sets of lines per field in Canal00.txt file from the same archive.

I'm still looking for other cassettes. Those I dumped were from my own setup in 1995, not sure of the tape recorder quality back then.

 

[fsphil]

The only text I can find that directly refers to the EMM CRC is this from LIGNE03.pdf:

"les 4 derniers servant de checksum de contrôle (fonction 3D42)"

I'm unsure what function 3D42 is in this context. Is it a CRC poly? I can't match it against any.

 

[homercartman]

Hi @fsphil

Maybe 3D42 is a polynom in the context of a Crc16 computation :

How to use table based CRC-16 code?

I was hoping someone might be able to shed some light on why my CRC-16 implementation does not seem to run correctly on Visual Studio? I got the algorithm from a manual for a specific device, and w...

stackoverflow.com

There is this tool, MiscEl, that might help :

MiscEl: CRC and checksum

Programs from HKJ, MiscEl electronic calculations, WinGrep, Magnify glass

miscel.dk

 

[dreambox1959]

@fsphil @Captain Jack

Canal_teletexte_LIGNE03.zip and 1 more file

this file has vanish !
can you give me again , please

BR

 

[Szycha881]

@dreambox1959 Here you are: Syster_docs.zip

BTW
@fsphil
You probably already know that, but I'm gonna write it in public Besides, thank you for Python script

HackTV works with original Polish C+ keys. After @Captain Jack's experiments and advise we changed the position of the generated lines from

if(line == 14 || line == 15 || line == 327 || line == 328)

to

if(line == 10 || line == 11 || line == 323 || line == 324)

and... voila! Decoder kicked in

I wonder if it's possible to force a second key table (French- post 1998) in the decoder...

 

[Captain Jack]

Yes @fsphil noticed that lines are different in the French VBI dump so it was a matter of experimentation for Polish key. I still haven't been able to get French C+ or Russian NTV+ keys to kick in - assuming they use the same lines as they have the same ATR.

 

[fsphil]

Nice! It's great to see this finally working. Right now I'm tidying up my code a bit and will push it to github shortly with everything that's been discovered recently.

At the weekend I accidentality activated some kind of combined line shuffle and cut-and-rotate mode. I don't think this is a real mode, but probably just the decoder getting confused. I was able to reliably get it into this state at the weekend but now it won't work. The scrambling pattern was static, the cut points and line order didn't change between frames.

 

[homercartman]

@fsphil
On the cut and rotate descrambling topic, @Captain Jack posted this one earlier :

If you play this video extra slow or frame by frame, you can see the cut and rotate pattern changing on every frame it occurs.

Also, maybe a hint : in regular Nagra line shuffling mode, when syster kicks in, I observed it first applies the static permutation for a couple of seconds, then actually applies the s, r parameters.
During the static permutation phase, the lines are shuffled once, exactly the same way across frames. I saw this by disabling the line shuffling code in hacktv, out of curiosity, to see what the decoder does along with VBI kick and sync.

I wouldn't be surprised the decoder behaves with the same static-then-dynamic phases in cut and rotate mode. This would conciliate what you saw with what @Captain Jack saw.

 

[fsphil]

The static order can happen when the PRBS is initialised with zeros, its output is then all zero for every frame.

I took video of the glitch too:

 

[fsphil]

I got it scrambling again. To enable it on my Premiere box I just need to set the LSB of one byte:

55 D0 18 6C 49 FE 28 B1 ...

Changing the 28 to 29 works. On my Canal+ FR box this just seems to distort the image a bit. Maybe not all Syster decoders support this type of scrambling?

 

[homercartman]

@fsphil
I saw some literature mentioning the drawbacks of Cut'n'Rotate when used in terrestrial channels.
It is said that multipath propagation creates echoes and signal fades which, when combined with CnR, seem to generate awful visual effects such as random streaks of luminosity due to AGC + local, fast-paced static near cutpoint.
This is what BBC claimed in a quite serious document they published when they chose VideoCrypt-S for terrestrial broadcast.

Regarding Premiere box: Premiere was on satellite broadcast only, right?
Then the Premiere box was maybe targeted for satellite usage only, so it was also maybe ready for CnR.
Regarding Canal + FR box: assuming CnR was not an option for terrestrial broadcast, they may just have simply (soft?) disabled it.

By the way: is Premiere box also decoding Discret11?
If so, compared to Canal+ FR box, Premiere box would be a "lucky" one in which 3 descramblers (Line Delay, Line Shuffle, Cut'n Rotate) are all active. But I guess CnR is just a general case of Line Delay.