Teensy 3.6 plays video from SD-Card

Status
Not open for further replies.
Quick update... got ffmpeg to work :) after downloading and looking at it last night.

command line was "ffmpeg.exe -stats -i file.avi -f rawvideo -pix_fmt rgb565be -s 128x128 -vcodec rawvideo -frames 1000 file.raw" for video only, no sound and loss of aspect ratio.

Best part was... only took a few minutes to add another function to make it fit into existing code within the resource manager.

Code: 
gDIB.Font.LoadFromRaw("file.raw", 128, 128, 16);

  for (int i = 0; i < gDIB.Font.FrameCount(); i++)
  {
    uint32_t t = micros();
    gDIB.drawBitmap(0, 0, i, DIB_BLIT_STORE);
    gOLED.checkSCKDiv(); // need to check as SD prob changed it
    gOLED._1351renderScreen(gDIB.getBuffer(), true);
    while (micros() - t < 40000) {}  // 40ms delay between top and here
}

delay(2000);
 
Hi Frank,

Thought I would start playing around again and see if I could add some of the DMA stuff to my version of this library. In particular I am interested in adding something like refreshOnce. In particular I would like to maybe do a begin refresh and then maybe add other calls that can ask if the refresh completed yet or not...

Looking over the current ili9341_t3dma code, I think I have a grip on a lot of how it works.

My guess for the one shot without wait, the differences I would probably need to change the start code to not disable the on completion?

Also to make it worthwhile, need to figure out how to not need to do the transfer of one full screen of data before the DMA takes over.

If I am reading this correctly the DMA is transferring 16 bits at a time to the PUSHR register and as such it is pushing only data and none of the command information, like which CTAR so will always use CTAR0, so CONT and PCS will be zero.

So makes me wonder how the CS stays asserted? Maybe it works since you are only transferring 16 bits to the register, that maybe it remembers the previous High word and so it remembers the upper word of the last full 32 bit transfer?

I am pretty sure don't have enough memory to have the data store 32 bit values for the whole screen so that approach probably would not work...

A couple of other thoughts, include:
a) Could take a perf hit and have the CS/DC pins do digitalWrites. Would take a hit as would have to wait until queue empty to toggle...

b) I was thinking about trying to layout the DMA memory such as the first position in DMA memory is for the 2nd entry not the 1st.
Then for 1 shot processing I was thinking I could then simply do something like:
Code: 
 SPI.beginTransaction(SPISettings(SPICLOCK, MSBFIRST, SPI_MODE0));
  setAddr(0, 0, _width, _height);
  writecommand_cont(ILI9341_RAMWR);
  writetata16_cont(screen[0][0])
  (do the stuff to startup DMA)
}
 
For continuous updates, not sure best way to recover to then properly output [0][0] again...  Could do something like: 
Initially setup dmasettings[0].TCD->SADDR = &screen[0][1];

And then somehow after we know that the dmasettings[0] was initiated change it to:  dmasettings[0].TCD->SADDR = &screen[0][0];
(Not sure best way to setup this timing).  Or maybe create an extra dmasettings object that only contains 1 element? [0][0], which we link in for continuous mode? 

Probably would want some clean way to remove this itme, if you issue the stop as to not output 1 new byte... But may not be much of an issue?

Thoughts?  Do you think I am maybe barking up wrong tree? 

Thanks,

Kurt

PS: if I start hacking this into my version of library will be doing it to memory allocated as a vector instead of two dimensional array, but same difference.
 
Almost got my video converter front-end for ffmpeg ready. Final testing phase and a few additions.

Must admit I tried the BGR8 8 bpp mode and was surprised that it has quite good picture quality for 1/2 the size of RGB565 format. With the correct dithering there really is not much difference when viewing animations on a small display however having to grab 1/2 the number of bytes per frame is a great outcome. Also, because there is only 256 possible pixel combinations a quick lookup can be made to convert BGR8 -> RGB565 to avoid heaps of bit shifting every pixel.

Couple of screen shots below. Tested on many inut files including avi, mp4, mkv, mov . Animated GIF files had one small problem... FFMPEG PROBE won't return the number of frames so I left that as a todo for later.

Has options for keeping the aspect ratio and if kept whether the device screen dimensions should be fully filled and thus output cropped.

vid1.png

vid2.png
 

Attachments

  • vid1.png
    vid1.png
    44.2 KB · Views: 218
  • vid2.png
    vid2.png
    68.5 KB · Views: 204
Frank B said:
Very nice !!!
Is this project available somewhere ?
Click to expand...

Hi, still WiP but progressing quite well.

I'm looking at the audio part of it at the moment however it will work with or without audio as audio will prob tax most smaller MCU's.

Had to have some deep thought on how audio was to work.

Basically to play back video with audio looping has to be a bit different and have to know the FPS of the video (which I had to add to the header).

The audio needs to be on an interrupt timer and consume a circular buffer that is topped up.


Pseudo code is :-

init video and audio (load first frame etc which has initial audio and video buffer, importantly buffer audio a few frames ahead and try to keep it a few ahead)

start audio timer

while frames to show
load frame(timer)
if new frame and not behind { display it } else { skip displaying this frame }

stop audio timer


Basically in the loop it will determine from elapsed time in ms and FPS what frame number it should be showing. Audio and video will only be loaded if the current frame has not yet been loaded. Basically have to make sure the audio is loaded and added to the buffer a bit ahead of schedule to avoid it running down. Video on the other hand, frames can be missed if getting behind.


ATM for no audio the file is <header><video frame><video frame><video frame> ... all of fixed known size from the header info.

With audio the file is <header><video frame><audio samples><video frame><audio samples> ... each block of video and audio is fixed size as the header tells how large the video frame block is and how large the audio block is.

In the converter I have given a few audio sample rates... so have to work out from this how many bytes will need to be in each audio block. It will be determined by the video FPS and the audio sample rate.

Higher video FPS will mean each frame will have smaller chunks of audio and higher audio sample rate will increase the number of audio samples needed for each frame.

Only problem I see is that sending audio to DAC is slow. Will have to use built in DAC as the MCP4921 atm is a bottleneck, especially when called so often from an interrupt.

One thing I noticed is video FPS varies a lot and the samples will not always fit evenly.

For example the first one I calculated has 316.67 as the number per block.

So... I think I round down to nearest integer and each frame calculates where the audio it needs is and grabs in this case 316 bytes. Means occasionally there will be an overlap of 1 byte but it will stay in sync.
 
Hello,
I am using "5 inch TFT LCD Module 800x480 Display w/Controller I2C Serial SPI" in one of my project. Communication interface is 4 wire SPI. I have ordered the display with 4 wire SPI mode, after receiving the display i have not made any changes to the hadrware. Just soldered one connector for SPI connectivity with microcontroller. Display is 3.3V.

I am using SAMD21J18A xplained pro board as my development board. Configured for 8MHz clock.

I have initialized the TFT and SPI communication. But, i am unable to see any thing on the TFT display. When i probed SCLK and chip select(CS) i could see signals in oscilloscope. On MISO able to see signals but little distorted. But MOSI is completely high.

Herewith I attach the initialization code i written for TFT along with this post.

I request your help, to identify issue with my code or any modifications i have to do wiring or other settings.

Thanks,
J. Chandra Sekhar
 

Attachments

  • TFT_Initialization.txt
    6.5 KB · Views: 260
Frank B said:
Very nice !!!
Is this project available somewhere ?
Click to expand...

Finally got a release out. New version 2.0 of MCU Resource Maker has the video conversion utility built in. Had to get a new site as FFMPEG blew the size of my setup over the 10MB limit of the last one. When creating frames gives a few choices for audio (including none) and the video rate can be changed.

The file is saved with the standard header [header][video#1][audio#1]........[video#n][audio#n]

Audio is 8 bit unsigned mono, video can have numerous formats... but all uncompressed.


New site address https://kiweed-software.000webhostapp.com/

Screenshot of newest version of the video convertor.
mcu_sc13.png
 
I have been trying to reproduce the video playing from an SD card on a Teensy 3.6 for the past week. I am stuck here where Digitana was in this post where I finally have a response on my LCD, but it is 3/4 solid blue with a black bar flashing. The bar sometimes alternates with a taller or wider black bar. The right side of the screen about 1/4 is light blue made up with very thin (maybe one row of pixels) vertical stripes. The file I am trying to display is the BigBuckBunny.bin file. At this point I am lost. The sum total that I know about this project is not enough top put in someones eye and have them notice. I am not a coder by any possible description, I have been following the posting in this forum, but apparently there is not enough information to allow me to reproduce the results of playing the video. I am using an Adafruit 1983 TFT touch screen I bought for a Raspberry Pi and have connected jumper wires from the Teensy 3.6 to the appropriate terminals on the connector of the display.

Having never posted anything anywhere, I am not sure I am going about this in the correct way as I am posting this on 09/27/2017 and this post was from February. If I am in the wrong place please advise. Thank you in advance for any help you can provide, Bob
 
I realize this is a long shot given that this thread is over two years old but here goes:

I'm trying to get FrankB's code to work with my own MP4 video and there appears to be a timing issue. The sound lags the video despite trying multiple MP4 videos from various sources (phone, internet, etc..)

When I run the BigBuckBunny sample file that FrankB posted in GitHub it works flawlessly so I know the code and the hardware is working fine.

I suspect that that the issue is in the Processing script that does the actual conversion of the MP4 to a BIN format.

I've generated the audio file using the following commands in ffmpeg

ffmpeg -i "myfile.mp4" -f s16le -ar 44100 -acodec pcm_s16le output.raw

and modified the .pde code to put to my .mp4 and this output.raw file.

I used Handbrake to identify the frame rate of the mp4 video file and used this value in FrankB's pde script as he mentions.

I've tried multiple MP4 files however they all have the sync issues.

Just looking for some guidance on what might be causing my issues.

thanks in advance
 
Status
Not open for further replies.
Back
Top