USB Video Class on Teensy 4.1; Missing isochronous packets

[miciwan]

Hi,

I'm trying to implement USB Video Class on Teensy 4.1 to stream video feed from the camera attached to my Teensy. Setup of all the descriptors was fairly painless, and the device quickly started showing up in the Device Manager (I'm on Windows) as a regular camera. I could pick it as a video source in Skype or in DirectShow directly. The only problem is that it refuses to show anything.

I spent a day analyzing different pieces, and it all generally looks as it should - the driver properly negotiates the streaming parameters with Teensy and starts up and isochronous pipe, and Teensy starts sending the data back. However, it looks like PC gets the packets in fairly random order, sometimes some of them are missing (it seems?) - or at least this is what it looks like when viewing raw data under PC-based sniffer (I don't have access to any hardware USB sniffer). I'm aware that this is an isochronous transfer, and the whole idea is that certain things can be missed, but in the current setup I'm using ~2Mbits of bandwidth, and there's nothing else going on there, so I wouldn't expect such problems. Have anyone experienced similar problems with isochronous transfers? Or maybe there are some gotchas in the USB software stack that might be causing this? As for the isochronous endpoint setup, I mostly followed the usb_audio example, the only thing that looked like it could be causing problems like this was the reuse of the transfer structure and the data buffer - which I would imagine in case of heavy use could potentially be still not transferred when a new interrupt comes - but adding more buffering here didn't really fix the problem (and maybe there's something in the USB stack that actually prevents that, I haven't really dug into that). For what it's worth: when the streaming starts I also lose the Serial communication

Attaching very rudimentary code if anyone wants to play with it. It's pretty sloppy, but shows the problem.

View attachment UVC_Teensy41.zip

 

[miciwan]

Trying to debug it more, though it's pretty annoying without any serial communication... anyway, I switched to using multiple transfer descriptors and buffers, to ensure I'm not overwriting any in-flight transfers. This way I also have access to the past transfer descriptors and I can inspect them before reusing. According to the manual, the controller should decrement the length by the number of bytes actually transmitted - and this is actually always 0 - which indicates that everything should be transferred. I also check the status - and it's all 0 - which suggests there were no errors (also the "active" bit is set to 0, which indicates that indeed the transfer is done). There's however the "frameCompleted" field there as well, that indicates the frame in which the transfer was completed. What I would expect is this number to be increasing. This is however not the case - it sometimes goes to zero, it sometimes decreases, definitely not monotonically increasing.

Anyone?

 

[miciwan]

Oh, I just noticed that the transfer callbacks are only called on non-active/finished transfers from the endpoint's list - so technically it's not possible to overwrite an in-flight transfer - which is good to know, but doesn't really change much, I still don't really know what's going on