Teensy working with HDMI input

Status
Not open for further replies.

Laki

Active member
Hi, I'm working on a project where I need to read rgb values from specific pixels from hdmi input. Would teensy 4 be able to work with hdmi?

I also found a board that processes hdmi input and outputs to 40 pin ttl, would teensy be able to work with that? The board is TFP401 HDMI/DVI Decoder.

Thanks!
 
If the HDMI signal is content-protected (look up HDCP), you will not be able to get anything out of it without a decoder that has whatever officially licensed tech is necessary to decode the stream. That tech is very tightly controlled. HDCP works over DVI as well, so unless that decoder is "shady," I don't think you are going to get a raw signal out of it.

If your video source is not protected (e.g. it's coming from a computer that is not playing a BD or DVD), it should be possible to read it without having to jump through any hoops beyond figuring out how to get the signal off the wire and into your Teensy.
 
My video signal is coming from a computer that's just doing desktop stuff and sometimes playing games, so I don't think it's hdcp protected. I tried to google the hdmi signal structure but I didn't find anything about how to read the image so that's why I thought the board I mentioned could be useful for processing the signal. But also speed is important for my project so maybe it wouldn't be good to add another board between the teensy and the signal.
 
HDMI is differential pair at high speed. Also, it can be multilink = mutipair.
I do not see any chance to read a specific pixel out of that data stream without adding a really fast FPGA with this type of signal inputs to the Teensy.
 
VGA would be fairly simple, as it's analog, although if you're interested in individual pixels you will have to track the rasters and then sample at just the right moment.

With DVI, you will have to do a handshake over I2C and then read the RGB channels according to the pixel clock.

Either way, you're reading data in raster format, which is the easiest possible way short of using a framegrabber. That would be another thing to consider, although not nearly as fast as reading the data directly.
 
I liked vga as a solution but I checked my gpu and motherboard and they only have dvi, hdmi and displayport... I'll look into dvi too, do you think it will be easier than hdmi? My gpu only has dvi-d, so no analog.
 
DVI will involve an I2C handshake, and then reading the R/G/B data lines whenever the clock line transitions to the right state.

Reading further into this, it's going to be a little tough. The pixel clock can run anywhere from 25-165MHz, and every clock cycle ten bits are transmitted through each of the three channels using low-voltage differential signaling (LVDS). Eight of these bits are meaningful, and the other two are not. At 25MHz, which is presumably a low resolution like 640x480, you would have to read bits at 250MHz x 3 lines. This pushes the boundary of my knowledge, but it seems to me you'd looking at a PLL driven by the pixel clock signal that tells the DMA to read the ten bits at the right frequency (which is 10x that clock.)

If you look at page 3 of this datasheet, you can see some termination resistors and op-amps are required. Adafruit sells a breakout of that chip (TI TFP401) but the output is a 40-pin ribbon connector designed for driving LCD panels, and it doesn't seem like any provision is made anywhere on the board to pull the raw RGB data.

I think at a minimum you'd need termination resistors and op-amps. As to whether the MCU's DMA controller can drive three channels at that speed off a PLL (which would itself have to be driven by the clock line), I don't know. If that is possible, you would have the ability to put something together without much external to the MCU.

It might be necessary to implement the PLL and three shift registers (the four blocks reading Latch, Latch, Latch, and PLL, just inside the left boundary on that page) with external chips, and then pump the output into three DMA channels.

Or, you can buy that chip on a breakout, and figure out how to pull the RGB channels off it. With the Adafruit breakout I guess you'd have to get a 40-pin ribbon connector and then figure out how to pull the signals off. I don't know if they would be in some funny flat panel-specific format, or if they would be literal RGB bitfields.

Someone with more knowledge of this stuff is welcome to chime in. To me it's an interesting problem, but I've never attempted anything of the sort.

Edit: https://cdn-learn.adafruit.com/asse...al/adafruit_products_tfp401sch.png?1414183039

The Adafruit breakout gives you parallel RGB out on that 40-pin cable, so if you can figure out a way to get that into the Teensy fast enough (looks like three 8-bit shift registers to three DMA channels, or a 24-bit shift register to one DMA channel) you'd be set. Depending on resolution and framerate, three shift registers to three DMA channels might be necessary, because the transfer frequency will be quite high.

Edit 2: The output of this board is mentioned as TTL, meaning 5V, so if using with a 3.3V board you will need level shifters.
 
Last edited:
Thank you, I've only done simple projects before so I'll probably order that board since it looks easier to work with, I'll update here if I have any problems or if I get it working.
 
By the way do I have to use the dma channels? Never used them before so not sure how they work.

Also I bought a breakout board for the 40 pin connector, not sure if I bought the right one, though.

And I'm not 100% sure if I understood correctly but from what I've seen, vsync signals a new frame, hsync signals a new row and the pixel clock are the individual pixels, so I could count that and calculate the current pixel position.
 
Yes, if you wait for a vsync pulse you should be able to derive the exact pixel location simply by counting the pixel clock: row = (int)(pulses / width), column = pulses % width.

Suppose you are using a fairly low resolution:

640*480 = 307,200 pixels per frame
* 3 = 921,600 RGB values (bytes) per frame
* 60Hz = 55,296,000 bytes/second
* 8 bits = 442,368,000 bits/second

I have no idea whether you need to read every frame, and if you only read 1x/second you only care about 921,600 values at most (although you are still going to have to sample at >55MB/sec.)

It is conceivable that you can use the pixel clock to count until you reach the exact pixel you want to read, and then sample right at that instant. Whether you will get close to the right frequency, using code alone, to read just the one pixel at exactly the right instant (and you will have to be dead accurate to catch the RGB bits at just the right time) I don't know.

The DMA controller should have no problem with this.

If you want to try doing this without DMA, you could count pulses on the pixel clock until you get within a few pixels of what you want to read, and then pause until you get to exactly the right time to read the RGB values. After using fastWrite() to trigger the latch on the shift registers, and fastRead() to read the incoming bits, you should have everything necessary to extract the RGB value of one pixel.

Making this work obviously requires getting your delays extremely accurate, which is probably best done by padding your operations with some number of NOP (assembly instruction meaning "do nothing for the next clock cycle") so that when the last NOP is done, and the next useful instruction is executed, it happens at "close enough" to the right time. You can see more information about using NOP for delays in this thread.
 
I would ideally check every frame. I assume there is no way for it to work on 1080p? Or 720p?
There are a lot more pixels in there and I'm not sure how fast I can count the pixel clock.

Edit: What if I counted the hsync pulses that signal new rows, and calculate the current row that way, and then do nop delays to get the specific pixel?
 
Last edited:
It can work at 720p or 1280p, or whatever resolution you want, as long as the Teensy can go fast enough to keep up with the incoming data.

I don't think you can count hsync pules, as they are only transmitted with analog RGB. You already have to keep up with the vsync and pixel clock so that you can keep the shift registers synced up with the input signal.

I don't know what your application is, but if you only care about one pixel, and if you can place that pixel wherever you want, it might be slightly simpler to place it at (0, 0) and read it immediately after vsync.
 
The pixel is a specific pixel around the center of the image so that won't work.

Can teensy keep up with 125 million pulses per second? If I calculated it correctly, the pixel clock on 1080p(1920x1080x60) is about 125 million per second.

And what's the fastest way of counting the pulses?
 
Status
Not open for further replies.
Back
Top