Call to arms | Teensy + SDRAM = true

I'm at a bit of a loose end trying to decide which direction to go forward for video output.
I really don't want to get a screen with an RGB interface, would rather have generic output which means VGA or DVI/HDMI.
VGA requires either R2R ladders, which are reasonably cheap but messy (roughly 50 resistors for 24bpp) or a 3-channel VGA DAC like an ADV7123.
DVI requires an encoder such as a SiI164 (<$5) which also works for HDMI, or a proper HDMI encoder that can do audio as well like the ADV7513.
Yet another option is making a new USB device interface based on UVC that would basically make the Teensy look like a webcam to the USB Host. The drawback there is you need a PC or similar to view the output, and it wouldn't use any of the LCDIF stuff so it's really out of scope here.

I'm inclined to just go for the R2R solution, since it could be thrown together in an afternoon (assuming my local store actually has the right resistors, it's a new store I haven't visited since they opened) while the other ideas would require getting a board made... although if anyone else wanted to make a simple encoder board like that (basic digital RGB -> DVI/HDMI) I think they'd have no trouble covering their costs by selling extras back to the community.
You’re a bit of a genius when it comes to graphics and display stuff. That’s much appreciated! Feels important to mention that.

I would love HDMI output with audio, that would simply be amazing! I can only support you with hardware so that you don’t have to use your own wallet. That’s about what I can do to assist you with that endeavor.
 
Last edited:
Somebody did that already for uvc. Do a search on Boson or UVC - here is one of the links:

The boson camera project is not about making a device that looks like a UVC device to a host. It is the development of a Teensy Host driver to read data from an actual FLIR Boson camera connected to the T4.1 host USB port. The driver is possible since the Boson, unlike most web cams, uses USB Bulk endpoints, rather than Isochronous endpoints. The driver is under NDA, but I don't consider that too much of a problem since Boson cameras retail in the $3K to $4K region. They are also subject ITAR export controls. Those factors probably mean a limited demand in the Teensy community. I'm still looking for a less expensive camera with bulk endpoints. If I find one, I'll try to develop an open-source driver.
 
Quick update on the MicroMod board update, that I am not sure yet if I will order yet or not...

I did not use the pass through pins for the camera, as, two rows implies you can not easily connect from either side with same object...
But did enter one here for the ILI9341/9488 displays. For the Camera, I put in two connectors one if I want it on front and other for the back...
Upsized it to fit the ILI9488 boards as well...



1707686595256.png

1707686758705.png


Shown with the 2.8" display, also hopefully the holes for 3.2 or 3.5" 9488...

Unclear if I will order some or not as I have good MMOD boards for the stuff I mostly dinker with.
 
@mjs513 I have a working example of LCDIF & SDRAM if you want to reference it in the SDRAM examples folder?
I still have some small updates to do to it, but it does work as is.

Im thinking, as we cant use alignment attributes if the compiler does not places the entire array in SDRAM (using EXTMEM for example), then the sdram_malloc should have an alignment argument.
I hacked up something in the example to do that using chatGPT - perhaps we can implement something similar into the library?
 
the sdram_malloc should have an alignment argument.
Prior work to assure alignment just over allocated by the alignment bytes needed then set the usable address masked upwards to that alignment boundary - perhaps that is what you did?

This 32 byte example from ...\arduino-1.8.19\hardware\teensy\avr\libraries\ILI9488_t3\src\ILI9488_t3.cpp
Code:
    if (b) {
        // First see if we need to allocate buffer
        if (_pfbtft == NULL) {
            // Hack to start frame buffer on 32 byte boundary
            _we_allocated_buffer = (uint8_t *)malloc(CBALLOC+32);
            if (_we_allocated_buffer == NULL)
                return 0;    // failed
            _pfbtft = (RAFB*) (((uintptr_t)_we_allocated_buffer + 32) & ~ ((uintptr_t) (31)));
            memset(_pfbtft, 0, CBALLOC);   
        }
 
I think I may have been wrong about that and the LCDIF alignment requirement is only 64 bits rather than bytes. Then the usual DMA alignment requirement would take precedence (32 bytes).
The default alignment of the smalloc library is still an issue (in both the sdram library and core), it should be at least 8 rather than 4 since that's the natural alignment of the CPU's largest native datatype (double = 8 bytes). I don't think it would trigger an alignment fault since the CPU supports unaligned access but it could cause horrendous performance penalties if a frequently accessed double ended up split over two cachelines instead of one.
 
The default alignment of the smalloc library is still an issue (in both the sdram library and core), it should be at least 8 rather than 4 since that's the natural alignment of the CPU's largest native datatype (double = 8 bytes). I don't think it would trigger an alignment fault since the CPU supports unaligned access but it could cause horrendous performance penalties if a frequently accessed double ended up split over two cachelines instead of one.
@Rezo:
Im thinking, as we cant use alignment attributes if the compiler does not places the entire array in SDRAM (using EXTMEM for example), then the sdram_malloc should have an alignment argument.
I hacked up something in the example to do that using chatGPT - perhaps we can implement something similar into the library?

Sorry been distracted with getting the HM01B01 working with the micromod board before testing on the SDRAM board. Something changed in the core so had to change a few things. Also incorporated some things learned from the OV7670 testing. More to follow.

Anyways right now the way we have been doing it for at least DMAMEM is the way @KurtE recommended a while back:
Code:
DMAMEM uint16_t pixels[320 * 240]  __attribute__ ((aligned(32)));
and basically the same thing when you use EXTMEM.

As for changing malloc think we want to avoid changing to using aligned_malloc bacause as Kurt commented you may want to use free with malloc at some point so would rather do it like GIGA does it for their framebuffer:
Code:
SDRAM.begin();
  Serial.println("Before camera start"); Serial.flush();
  if (!cam.begin(CAMERA_R640x480 /*CAMERA_R320x240 */, IMAGE_MODE, 10)) {
    blinkLED();
  }
  uint8_t *fb_mem = (uint8_t *)SDRAM.malloc(640 * 480 * 2 + 32);
  fb.setBuffer((uint8_t *)ALIGN_PTR((uintptr_t)fb_mem, 32));

Would then only have to add
Code:
#define ALIGN_PTR(p,a)   ((p & (a-1)) ?(((uintptr_t)p + a) & ~(uintptr_t)(a-1)) : p)
to the library
 
As for changing malloc think we want to avoid changing to using aligned_malloc bacause as Kurt commented you may want to use free with malloc at some point
Not really sure what you mean, any pointer returned by an allocation function has to be able to be directly passed to the equivalent release function (the same ways pointers returned by aligned_alloc() can be passed to free() when using stdlib). The simplest way to do it with smalloc would involve increasing the size of smalloc_hdr by 4 bytes, which would also make the default alignment 16 (a power of 2, as it should be) instead of the bad value it currently uses (returned pointers are always multiples of 12). The extra field would hold the amount of padding preceding the header of the allocated block.
 
Not really sure what you mean, any pointer returned by an allocation function has to be able to be directly passed to the equivalent release function (the same ways pointers returned by aligned_alloc() can be passed to free() when using stdlib). The simplest way to do it with smalloc would involve increasing the size of smalloc_hdr by 4 bytes, which would also make the default alignment 16 (a power of 2, as it should be) instead of the bad value it currently uses (returned pointers are always multiples of 12). The extra field would hold the amount of padding preceding the header of the allocated block.
FWIW - I believe we should use the KISS strategy and in this case change as few system things as possible.

That is with EXTMEM or others who may use the sm_malloc code, I would probably avoid making changes to the underlying sm_malloc.
Especially for example with extmem_malloc, which if it fails to allocate memory from external memory it then calls malloc, which may have
different rules or ways it does things.

So what @mjs513 was saying about needing to keep a pointer to the actually allocated block. That is what for example the display library
does. That is if the user says to free the frame buffer, it needs to know the actual address returned by the memory allocator and not just
the address we use as the first bytes of our logical frame buffer.
 
That is with EXTMEM or others who may use the sm_malloc code, I would probably avoid making changes to the underlying sm_malloc.
Especially for example with extmem_malloc, which if it fails to allocate memory from external memory it then calls malloc, which may have
different rules or ways it does things.
malloc() already guarantees 8 byte alignment. It's defined this way in the C standard - any dynamically allocated memory must have alignment suitable for any native datatype e.g. 8 bytes for a double. smalloc is deficient in this regard.

So what @mjs513 was saying about needing to keep a pointer to the actually allocated block. That is what for example the display library
does. That is if the user says to free the frame buffer, it needs to know the actual address returned by the memory allocator and not just
the address we use as the first bytes of our logical frame buffer.
And that's why I'm saying the changes should be in smalloc, so there is no possible confusion on the user side about which pointer is the aligned space and which is the allocated space. GNU C has had aligned allocation functions for decades, it shouldn't be up to the user to manually overallocate and align pointers (which btw isn't completely safe from a cache point-of-view - the size must also be 32-byte aligned to ensure the allocated space will not share its final cacheline with a separate memory allocation).
 
And that's why I'm saying the changes should be in smalloc, so there is no possible confusion on the user side about which pointer is the aligned space
Sounds good,

I would suggest that you might make the changes and try it out. And then if appropriate create a Pull Request into core and see if/when Paul will pull it in. But still need the same stuff as we mentioned for DMA buffers aligned to 32 byte boundaries.
 
I would suggest that you might make the changes and try it out. And then if appropriate create a Pull Request into core and see if/when Paul will pull it in. But still need the same stuff as we mentioned for DMA buffers aligned to 32 byte boundaries.
Just popped back on line for a few minutes - I agree - its going to Paul;s call to change sm_alloc.
 
And then if appropriate create a Pull Request into core and see if/when Paul will pull it in.

I'm currently working on improvements to Teensy Loader to properly handle more than 1 Teensy. So please understand I probably won't make time for quite a while to review pull requests for new features or anything that isn't an immediate problem.
 
Finally got the HM01B01 camera working again with Teensy. So hooked it up to the SDRAM board using EXTMEM for the Buffer for still images. Did find that if you hook up a display best to use pins 0/1 for dc/rst if you are using 9/8 for other things.

The one thing I found is that the Arducm HM01B01 only works with 4bit mode enabled while the Sparkfun camera (hm01b0 - ribbon type works in 4 or 8bit mode

Still image and small gif
07-51-32.jpg


Arducam_HM01B01 - Made with Clipchamp.gif
 
Would be cool to see this on @Rezo 5" with 24bit and LCDIF. :cool:
Go for it. I believe we gave links to all of the cameras you can order from Arduino, like the 7675
Which also shows the other 3...

All you need to do is to hook up something like 16 jumper wires to the camera.
And maybe 9 more for these TFT displays...
Not sure how many for Rezo setup .
 
24 bit color requires 24 pins for bus and 4 pins for control.
28 in total from B0_00 to B1_11 - that leaves 4 pins left on FlexIO2

But, if one uses a 16 bit or 18 bit bus, that leaves 12 to 10 FlexIO pins left which is enough for the camera.
I don’t think the camera supports a higher color depth than 565 anyways? Or am I wrong?
 
I thought these cameras only sampled one channel per pixel, so they get away with using an 8 (or 10 for higher quality) bit data bus.
I'm waiting on an FFC breakout board to connect mine, figuring out the pins will be fun...
 
Random thought for the day: the SEMC interface (the part of the IMXRT1062 that talks to the SDRAM chip) also supports 8080 display interface (DBI) mode, letting the CPU write directly to a display's memory rather than updating a framebuffer in RAM and then sending it using LCDIF/FlexIO - don't the 948X displays support that mode? Might be another thing to consider for a later board revision.
 
Back
Top