Interest in 3D graphics lib?

JarkkoL

Well-known member
Hi,

Couple of years ago I posted some videos about a 3D rendering project I was doing on Teensy. Finally I have decided to start to work on publishing the library and was wondering if there's any interest in this community for such a thing? Or if you have some suggestions, questions, etc to help me focus on the right stuff would be appreciated :)


Thanks, Jarkko
 
Cool, very nice to hear there's interest for 3D graphics in the MCU community! :eek: I haven't really seen that many 3D projects running on MCUs.

Here's the current feature list of the lib:
  • Configurable tile-based rasterizer (can be configured to run in very tight memory budget, e.g. 4.5kb [video])
  • Supports custom vertex, pixel/fragment and tile shaders to implement custom geometry and lighting effects [video].
  • Top-left triangle filling convention as in OpenGL/DirectX (no pixel overdraw at triangle edges, particularly important for transparent rendering)
  • Extensive templated math library (vector/matrix/quaternion/color/transform/camera classes with a huge set of operations)
  • 31 pixel formats (and easily extensible further) that can be used for intermediate tile buffer, frame buffer & texture formats
  • 8/16/32bit depth buffer support, 8 depth test functions and depth write on/off toggle
  • Tool to split 3D models (obj/dae/fbx/lwo/3ds) to clustered p3g models with custom vertex formats (e.g. pos,normal,uvs, etc. in custom formats) that can be rendered straight from flash memory.
  • Tool to convert image files (png/jpg/tga/tiff/webp/jp2) to textures of desired pixel format to be sampled straight from flash memory.
  • Geometry cluster hi-z occlusion culling to optimize vertex transforms and rasterization [video]
  • Geometry cluster visibility cone culling for early elimination of unseen clusters [video]
  • Mesh and texture classes to easily utilize the assets exported by the tools.
  • 2D texture sampling with wrap/clamp address modes, bilinear/point sampling [video]
  • Vertex post transform caching to reduce vertex transform cost
  • Custom vertex input layouts (e.g. position/normal/uv/etc. data in custom formats)
  • Perspective and non-perspective barycentric coordinate interpolation
  • Support for up to 4 render targets (with programmable blending)
  • Optimized ILI9341 and ILI9488 display drivers with DMA support. Easily extensible with new drivers.
  • Rasterizer stats to trim memory usage
  • Debug logging, asserts/warnings/errors, etc.

I got still some more work to do before publishing the lib:
  • Dust off my old Teensy boards and displays to test that the lib still runs fine on them
  • Write README.md to explain what this lib is about and explain the concepts.
  • Make some more examples + videos to show how to use the lib and test that all the features actually work.
  • Fix some remaining issues, further simplify usage, testing (should probably write automated unit tests)
  • Find out a good name for the lib, super hard! :D I have been thinking EmberGL, thoughts?
  • Figure out how to push the lib to show up in Arduino IDE library manager.
 
I have created 3 examples so far to get started using the lib:
1) Init the device and draw a rectangle on the screen
2) Render a cube
3) Render a textured cube
Any other ideas for examples you would like to see? :)
 
Only other idea is your monkey or teapot example to show how to import objects and to shade them
 
Only other idea is your monkey or teapot example to show how to import objects and to shade them
Oh yes, I'll add this as well. People have also asked about the gauge sample so that would probably be a good addition too and to show that it's not only 3D but you can do 2D too. Maybe also the super memory optimized sample too which enables 3D rendering on Teensy LC.

Finally got the samples running again on Teensy & ILI9341 after some lib changes :D Have to still test ILI9488. For testing swapping between MCUs and displays is a bit of a pain on breadboard to do all the wiring manually, so have been thinking of making some standard HW interface to enable snapping different MCU-display combos quickly.
 
Finally got the samples running again on Teensy & ILI9341 after some lib changes :D Have to still test ILI9488. For testing swapping between MCUs and displays is a bit of a pain on breadboard to do all the wiring manually, so have been thinking of making some standard HW interface to enable snapping different MCU-display combos quickly.
Yes, I've been planning, making breadboards, and soldering up prototype boards that have 2 sets of SPI pins in a standard order with the idea that I can quickly swap either different displays or different Teensies. I then make custom cables that attach to each display that plugs into the standard 10 pin header that goes to a specific display. The order I've chosen for the 10 pin header is:

  • Ground
  • Power (either VIN or 3.3v)
  • SCLK
  • MOSI
  • MISO
  • CS-A
  • D/C
  • Reset
  • CS-B
  • Blink (typically connected to 3.3v)

For the first SPI header, I use the standard SPI pins (13, 11, 12), and I only use the first 24 pins to allow any Teensy to be plugged into the system. The two CS pins and D/C are chosen from the 'fast' pins in the Teensy 3.2, 3.5, and 3.6 to allow me to use the uncanny eyes program. Pin CS-B can either be used for a display's SD card reader, touch screen interface, or with uncanny eyes in Teensy 3.x to drive 2 displays. Pins CS-A through blink have dip switches, so there is a default value, but I can override the default by turning off the dip switch and using a jumper wire. My current defaults are:
  • Ground - ground
  • Power - connected to a switch between 3.3v and VIN
  • SCK: 13
  • MOSI: 11
  • MISO: 12
  • CS-A: 22
  • D/C: 9
  • Reset: 5
  • CS-B: 10
  • Blink: 3.3v

For the 2nd SPI header, I only use pins that both the Teensy 4.0 and 4.1 support (though you have to bring out some of the Teensy 4.0 pins from the solder pads underneath the Teensy 4.0). Fortunately, the 2 or 3 Teensy 4.0's that I have wired up with the underneath pins use the Teensy 4.1 pin layout for pins 24-32. In addition to dip-switches for CA-A through Blink, on the 2nd SPI there is an extra dip-switch to switch MISO from pin 1 to the alternate pin 39 for the Teensy 4.1:
  • Ground - ground
  • Power - connected to a switch between 3.3v and VIN
  • SCK: 27
  • MOSI: 26
  • MISO: 1
  • CS-A: 0
  • D/C: 24
  • Reset: 25
  • CS-B: 28
  • Blink: 3.3v

I soldered up the first generation of prototype board, but I'm a little unhappy with some of the decisions, and some of the soldering for the non-display pins may be wrong. But any way, the headers this board has are:
  • dual 24-pin rows for Teensy, and to use jumper wires for any pin;
  • The two SPI headers, plus the dip-switches;
  • Three pin jumper header to switch both SPI displays from 3.3v to VIN;
  • I2C header, including pull-up resistors for I2C (this doesn't work in the current board);
  • Pinout for ground, 3.3v, 15, and 14 to allow access to Serial3, 2 analog input pins, S/PDIF, or PWM;
  • Pinout for ground, 3.3v, 28, and 29 to allow access to Serial7 or PWM;
  • Pinout for ground, pin 16, and 3.3v to allow plugging in a potentiometer (with a dip switch to use a different pin);
  • Connect pin 3 to a momentary switch and ground to use as a general button;
  • Connect pin 4 to a dip switch or jumper to allow for a dedicated switch;
  • Pinout for ground, VIN, and pin 17 (connected via level shifter) for use with neopixels (this doesn't work in the current board);
  • The next board should have a separate pinout for accessing ground, 3.3v, and pin 17 without doing the level shifting and a dip switch to change pin.

Some of the design decisions include:
  • Don't use pins 2, 6, and 7 as these are used by the prop shield;
  • Don't use pin 10 as the main CS pin, since this is used by both the prop and audio shields;
  • Don't use pin 21 as this is used by the feather adapter for battery gauge;
  • Don't use pins 7, 8, 20, 21, and 23 as these are used by the revision D audio shield;
  • Don't use pin 16 for the SPI headers, since this is used by the audio shield (it is brought out in the Serial3 breakout);
  • Use the Teensy 3.x 'fast' pins for the first SPI display (22, 9, and 10);
  • Use a TX pin for neopixel (pin 17) to allow using the WS2812Serial library on Teensy 4.0 and 4.1;
  • The I2C breakout order allows connecting the Sparkfun I2C to QWIICK header directly;
  • Except for potentiometer & neopixels, always put ground first, power second in the pinouts;
  • Potentiometer pinout allows plugging a potentometer directly into the headers (ground, data, 3.3v);
  • Neopixel matches my existing cable label (ground, power, data);
  • I've used pin 3 as a momentary button in previous shields.
 
Last edited:
Thanks for the tips Michael! I have these 6 displays and 3 Teensys (LC, 3.6 and 4.0) so far, so quite a bit of combos already :) These are connected to 36 pin connector boards but I haven't figured out the pin layout for the connectors yet. Some displays use SPI while others have parallel data interface (up to 16 bits I think) which is why the wide connector. So once I have the standard for the connector designed, I just need to wire the display/MCU connectors boards accordingly.

embergl_devboards.jpg
 
Below is the interface design I'm thinking of. Should handle SPI/Parallel up to 16bit + some NC pins for extensions later.

hw_interface_pinout.JPG
 
I am only switching between various 4 pin SPI displays which makes it simpler. If you are dealing with parallel data pins and different Teensies, you may need to deal with needing to use different pins for the parallel I/O pins.

Similarly if you want to deal with the 2nd SPI controller on the LC, 3.5/3.6 vs. the 4.0/4.1 (or using the alternate SPI pins). Unfortunately with the long term chip shortages, going forward, I expect that anything other than 4.0/4.1 will be limited to just dealing with legacy designs with chips at hand.

Pololu has setups if you want to make custom cables. where they have a selection of jumper wires with pre-crimped ends (M/M, M/F, F/F). wires of various sizes (1", 2", 3", 6", 12", 24", and 60"), as well as the male/female ends if you want to make custom sized cables. They also sell crimp connector housings to insert the cables (from 1x1 up to 2x20 pins):

I find in practice that I need to make sure the male pins in the housings don't get pushed back when inserting them in breadboards with smaller holes. I also find that it isn't worth it for me to make custom sized cables, and instead, I just buy 3" and 6" cable packs (3" tends to be a little short for my usage). For Teensies, it would have been nice if the connect housings came in 14 or 24 pin housings.

Adafruit has a smaller selection of these cables and headers:
 
Got the first connector boards soldered and running, and now testing different MCU-display combos is as easy as swapping graphics card on PC :) Nice thing is that I need only one of each MCU and display, and can also detach them from the connector boards to put on breadboard to figure out the connections. So far I got ILI9341 (SPI) and ILI9488 (8bit parallel) display boards, and T4 (SPI & 8bit Parallel) and T3.6 (SPI only) MCU boards. Still need to figure out a fast way to bit bang on T3.6 for parallel and then solder the pins. And solder the boards for Teensy LC and other displays as well. Also need to solder 8 more parallel pins for 16bit parallel interface, it's getting crowded :D

t4_front.jpgt4_back.jpgt36_front.jpgt36_back.jpgili9341_front.jpgili9341_back.jpgili9488_front.jpgili9488_back.jpgt4_ili9341.jpgt4_ili9488.jpg
 
Hi,

Couple of years ago I posted some videos about a 3D rendering project I was doing on Teensy. Finally I have decided to start to work on publishing the library and was wondering if there's any interest in this community for such a thing? Or if you have some suggestions, questions, etc to help me focus on the right stuff would be appreciated :)


Thanks, Jarkko[/QU OTE]


According to you, What is the wrong perspective in 3D rendering?
 
Been working on the lib to fix issues, improve the API, writing doc, etc. Amongst other things writing some unit tests for the library and got the first few running shown below :D

first_unit_tests.png
 
These are actually unit tests to guard against library regressions and not the examples. The list of the actual examples I have been thinking:
  • Graphics device initialization & colored rectangle rendering (shows how to setup the hardware and pick the accompanying driver + use the simple "immediate" rendering API)
  • Scaled & rotated textured 2D rectangles (shows how to import & sample PTX texture assets. point/bilinear sampling, texture address modes)
  • Rotating monkey head with simple lighting (shows how to create, import & render P3G meshes and write shaders. basic camera & object transform setup & render loop)
  • Multiple dispatches (how to render 3D objects with multiple materials)
  • Rotating textured cube (shows how to use PTX textures with 3D models and how to interpolate vertex attributes with barycentric coordinates)
  • Speedometer (2D rendering example with textures. e.g. for GUI rendering with the lib)
  • Extreme memory optimized sample (how to config the renderer to operate in extremely tiny RAM budget. run on Teensy LC, use of tile shaders for custom pixel conversion)
  • More complex lighting (rotating monkey head with multiple lights, normal maps, GGX BRDF model, etc.)
I try to make the learning curve reasonably gentle and that users can start to modify the samples to get better idea how things work (e.g. take the "monkey head" sample and import own p3g meshes, render multiple meshes, etc.) But this still requires some background knowledge about 3D graphics because I don't want to turn this into 3D graphics primer (like explain what shaders, samplers, etc. are, how 3D transforms work, etc.) and there's tons of material about it already and would be crazy amount of work :D Also to take full advantage of the library requires more advanced C++ knowledge, but with examples even C++ beginner users should be able to get some nice stuff going, I hope :)

I just pushed a private "development" repository to GitHub and would love to have someone with some 3D graphics and Arduino experience to try it out and give feedback before pushing to public. There is still work to do (for example write those example projects) before the first public release, but the library should be in a decent state for some beta testing :) It requires Windows environment since the tools are Windows-only currently.
 
Looks very neat :)

Just to let you know I also wrote a 2D/3D graphics lib (TGX). The 3D API provides similar features to your library except that it does not (yet) provide sub-mesh culling and is screen/hardware agnostic (it draws on memory framebuffer). The lib mostly target ESP32, T4 and other "powerful" 32bits MCU.

It will be good to have alternatives and hopefully both projects will benefits from each other !
 
Oh yeah, I saw your lib! It's the only other lib I have seen with comparable set of features :) I hope these will help push 3D graphics adoption to MCUs :D
 
Got the aforementioned examples created for the library and here's a Youtube playlist of them running on Teensy & ILI9341.

Hopefully can get the library released by the end of the year :)

Cheers, Jarkko
 
Back
Top