Highly optimized ILI9341 (320x240 TFT color display) library

[PaulStoffregen]

Glad it's working now. Hopefully the increased speed will be worthwhile!

Now I just need to decide on a name. If it's going to remain "ILI9341", I'm not really excited about prefixing with "PJRC_". Yes, I know Adafruit's doing that, but if it's going to be just the part number, I'd much rather have the meaningful number first and add a suffix. Perhaps "_t3" like Nox's I2C library?

If anyone has any ideas for an awesome or catchy name, now's the moment. I'm going to rename it later this week and put it into 1.20-rc3.

[PaulStoffregen]

By the way, PJRC now has one of the 320x240 ILI9341 TFT displays in stock. :-)

http://www.pjrc.com/store/display_ili9341.html

[KurtE]

Yep - I think the speed really helps!

Not sure about naming convention. I agree having something like T3 in the name would be good as it is specific to these. I also don't mind something like PJRC at the start of the name as it makes it easy to remember where it came from...

FYI - I also tried it out on the Adafruit 2.2" TFT display that used the ILI9340 chip and it appears to work fine with it as well. Earlier I compared the Adafruit drivers for both the two and at the time I did not see any logical differences in their drivers. I have not checked recently as there have been deltas made on the Adafruit tree for both drivers, but I think they are reasonably compatible...

May have to pick up one of yours in my next purchase....

Kurt

[nlecaude]

Paul: in the picture there seems to be a daughter board, is that also included ?

[pictographer]

I like the _t3 suffix a lot! T3 would make a good trademark. It's short. It's techie. A very cursory web search didn't turn up other microcontroller board that might already have the mark. It would be easy to stylize into a distinctive mark (a logo or logotype). The loose association with Terminator 3 isn't bad either.

[syso2342]

Hi Paul,
I just got an ILI9341 Display (2.2" without Touch Interface) pretty cheap and I will be able
now to help you with debugging etc.

[PaulStoffregen]

I renamed it to "ILI9341_t3"

[syso2342]

I made a "TrueType" font to Bitmap converter in Python. Are you planing to support Bitmap/Sprite-Fonts?
https://cdn.mediacru.sh/UkOMOUzWNh6N.png (example for fixed width 20x32 font)

Each Letter is 20*24 pixels. X = 20*numASCIIchar.
Font selection = Y = 32*fontNumber

[KurtE]

Looks good, I updated, my clone... Recompiled my test program and it still works.

I don't know if it really matters, but in the Readme file it says you need the Adafruit_GFX library. With your current version I don't think that is true any more as you brought that code directly into your class...

Kurt

[PaulStoffregen]

Many years ago I implemented variable width fonts on that old 128x64 LCD. It was able to draw variable size characters, and it supported negative leading space, for fancy fonts where part of each character needs to overlap the rectangle where the previous character was drawn.

Yesterday I dusted off that old code, considering porting it to this display. It had a maximum character size of 32x32 pixels. The font data was converted from the .BDF format fonts that come with X11. They're up to 24 point, which maps to about 32 pixels tall for the largest ones. BDF could represent larger bitmaps, but I couldn't find any BDF files online for larger fonts, nor any simple tools for converting/rendering from Truetype to BDF.

I might bring that old code and old font collection into this library without any significant changes. It'll look a lot better than Adafruit's 5x7 fixed font. But ideally, the 32 pixel size limit should be removed and a collection of larger bitmaps needs to be created. Even just those changes could turn into quite a bit of work.... and I have so many other important things to do!

[syso2342]

import ImageFont, ImageDraw, Image

fontSize = 32
fontWidth = 20
numFonts = 1
numChars = 127-32 # Because the first 32 characters are not visible.

image = Image.new( 'RGB', (fontWidth*numChars,fontSize*numFonts), "black")
draw = ImageDraw.Draw(image)
font = ImageFont.truetype("whitrabt.ttf", fontSize)
font2 = ImageFont.truetype("saxmono.ttf", fontSize)
font3 = ImageFont.truetype("MODENINE.TTF", fontSize)

# ASCII Characters from 32 DEC to 127 are visible
for x in range(32,127):
draw.text(((x-32)*20, 0),chr( x), font=font)
draw.text(((x-32)*20, 32),chr( x), font=font2)
draw.text(((x-32)*20, 64),chr( x), font=font3)

image = image.convert('L')
image.show()

is my python converter code. you can access the pixel data by image.getpixel((x,y)).
I think you should be able to use this data in a good way

[syso2342]

Done it:

https://github.com/DatanoiseTV/Bitma...er/chartable.h
20x32 Font.

[PaulStoffregen]

I've been thinking a bit more about how to implement large fonts. There are 2 big challenges.

#1: A compact data format is needed. For example, that 20x32 font array is 60800 bytes, or 7600 bytes if reduced to 1 bit per pixel. For a 60x96 font, which will make beautiful large letters on these 320x240 displays, it'll grow to 68400 bytes, using 1 bit per pixel. That still fits in Teensy 3.1's flash, but it's getting really large.

#2: Drawing to the display memory is done most efficiently with rectangles. Setup to draw a rectangle costs 11 bytes of SPI communication, and 2 bytes for each pixel. If each pixel is treated as a 1x1 rectangle, as Adafruit's implementation does, then 13 bytes per pixel are transmitted to the display. Drawing a 3x3 rectangle costs 3.22 bytes/pixel, which is 4 times faster then drawing all 9 pixels individually.

A really ideal data format would describe each character as a list of rectangles. A pretty sophisticated Python script would be needed to convert the bitmap into a rectangle list. A really awesome approach would compute the SPI communication cost for alternate rectangle sets. An ideal solution might even allow overlapping rectangles. But even a less-than-ideal script could make a huge improvement in drawing speed.

How to efficiently encode the rectangle list is also a good question.

There's probably not much point using more than 4 bits to encode the rectangle size, allowing rectangles 1 to 4 pixels in size. A large region to be painted in with 4x4 rectangles would require about 2.7 bytes/pixel on the SPI bus. Limiting the number of rectangles to only 16 sizes could also place some nice bounds on the number of different combinations to be compared in the Python script to find the most efficient packing.

If an integer number of bytes are used for each rectangle, 3 bits could be used to encode the relative position of each rectangle (perhaps in relation to the previously drawn one), and 1 bit could specify that a 2-byte format is used to expand the position spec to 11 bits. Or maybe there'd be 3 relative position options, 3, 10 and 18 bits (or maybe other encodings...), so the script could always manage to encode any bitmap when some rectangle is a long distance away from all others.

As a quick experiment, I took the number "7" from chartable.h and converted to ASCII art.

             11111
   012345678901234

 0 ***************
 1 ***************
 2 ***************
 3             ***
 4             ***
 5             ***
 6             ***
 7            ****
 8           *****
 9          *****
10         *****
11        *****
12       *****
13      *****
14     *****
15    *****
16   *****          
17  *****           
18 *****            
19 ****             
20 ***

Here's one possible packing into rectangles:

             11111
   012345678901234

 0 AAAABBBBCCCCDDD
 1 AAAABBBBCCCCDDD
 2 AAAABBBBCCCCDDD
 3             EEE
 4             EEE
 5             EEE
 6             EEE
 7            FFFF
 8           mFFFF
 9          GGGGn
10         oGGGG
11        HHHHp
12       qHHHH
13      IIIIr
14     sIIII
15    JJJJt
16   uJJJJ
17  KKKKv
18 wKKKK
19 LLLx
20 LLL

This may not be the most efficient packing (i just made it up), but here's the rectangle list. "Offset" is the position from the pixel to the right of the top row of the prior rectangle.

Rect    Size    Offset
----    ----    ------
 A      4x3       -
 B      4x3     0, 0
 C      4x3     0, 0
 D      3x3     0, 0
 E      3x4     -3, 3
 F      4x2     -4, 4
 F      4x2     -6, 2
 G      4x2     -6, 2
 H      4x2     -6, 2
 I      4x2     -6, 2
 J      4x2     -6, 2
 K      4x2     -6, 2
 L      3x2     -5, 2
 m      1x1     7, -11
 n      1x1     2, 1
 o      1x1     -6, 1
 p      1x1     2, 1
 q      1x1     -6, 1
 r      1x1     2, 1
 s      1x1     -6, 1
 t      1x1     2, 1
 u      1x1     -6, 1
 v      1x1     2, 1
 w      1x1     -6, 1
 x      1x1     2, 1

SPI Communication:
131 pixels = 262 bytes pixel data
25 rectanges = 275 bytes overhead

If each rectangle is encoded using 2 bytes, this character takes 50 bytes to store (not including metadata for font metrics and indexing). That's not wonderful, since the bounding box is 15x21 = 315 pixels, would would take only 40 bytes to store as an uncompressed bitmap.

Encoding the rectangle list efficiently will really depend on getting as many rectangles as possible into a compact single-byte format. Perhaps one way to accomplish this would be using the 4 bits non-size bits as the index into a lookup table (provided by the script which encoded the font), of 14 X-Y offset pairs. The other 2 combinations could be specify a 2 or 3 byte rectangle, allowing offsets -16 to +15 or -128 to +127, so the Python script could always encode any offset for the ones that don't fall into a list of the 14 most common.

Then again, there could be other smart ways to do this stuff. This is just my current thinking about it.....

[syso2342]

Actually there is a daily efficient compression algorithm for micro controllers for
exactly this purpose. Its called "Heatshrink" - no joke.
But I must say that I find your idea with the rectangle idea really nice. I look
forward to see it! Tell me if I have to look and convert some nice fonts for you.

Here's a link: http://spin.atomicobject.com/2013/03...a-compression/

Also: These are my test results:
Display Power Mode: 0xDE
MADCTL Mode: 0x6C
Pixel Format: 0x7
Image Format: 0xDE
Self Diagnostic: 0xE0
Benchmark Time (microseconds)
Screen fill 280148
Text 19226
Lines 73348
Horiz/Vert Lines 23154
Rectangles (outline) 14668
Rectangles (filled) 581733
Circles (filled) 95774
Circles (outline) 96186
Triangles (outline) 17815
Triangles (filled) 197594
Rounded rects (outline) 40206
Rounded rects (filled) 637590
Done!

[PaulStoffregen]

Actually there is a very efficient compression algorithm for micro controllers for
exactly this purpose. Its called "Heatshrink" - no joke.

My main hope is to optimize performance.

Generic compression of image data will yield a bitmap output, not a list of rectangles that's optimized for fast drawing on this type of display. My prior attempt to optimize Adafruit's drawChar() only made about a 20% speedup, by combining groups of 2, 3, 4 and 5 horizontally adjacent pixels. Perhaps better could be done... I only spent about half a day on it, but my feeling from that effort is mapping raw pixel data to rectangles at runtime on Teensy 3.1 isn't ever going to achieve the best possible performance.

A rectangle list, carefully crafted ahead of time by the Python script to minimize the number of rectangles, it the key to great performance. Teensy 3.1's SPI port runs very fast with only a short FIFO, so organizing the data in an optimal way is the key to crafting code that both minimize the number of bytes needed on the SPI bus and keep then flowing at 24 Mbit/sec.

For small fonts, the data size may often be larger than simply storing uncompressed bitmaps. For a large font, if the Python script is smart enough to get most of the rectangles to represent larger than 8 pixels and use the compact 1 byte format, the encoded size might end up smaller.

[PaulStoffregen]

Do you think it might be possible for your Python script to try converting to a rectangle list?

Even a 2x2 square reduces the SPI communicaton from 52 bytes to only 19 bytes, for nearly a 3X speedup to fill in the majority of the area.

[KurtE]

This is great stuff. As with many others wish there were better fonts available. Personally I don't think I would use really large fonts that often, but having a few fonts that are like 2x and 3x of the current size font would be great. Also would be great if potentially each program can use their own optional fonts, so if I do want a marque program with large fonts then it makes sense to use up that memory space for the large font.

Again great work.

Kurt

[PaulStoffregen]

I'd like to implement fonts as a C++ class, and a setFont() function which takes a reference to the font object.

The reference will be initialized to the default 7x5 font, so that data will always be linked into your code, whether you use it or not. But any other fonts should only end up in your final program if you use them with setFont() or some other code in your program that references them. The linker automatically omits any unreferenced code, which should allow this library to provide a very large collection of fonts spanning a lot of different sizes. Only the once you actually use will be included in your program.

I also have some ideas about structuring the data in a way where a table of index numbers into the raw (variable size) data allows individual characters to be commented out. So if you build a project using a 100 pixel font to show some important data in very large digits, you could comment out the letter and symbols, so only the characters you actually need would be in the compiled font.

But all this depends on someone wanting to take on the Python script side....... (hint, hint)

[syso2342]

I think I will have to take over this part. Let me see if I can find a little time and brain power (currently I have still a bit
of a brianlag because of work + work on my synth).

By the way: Do you want to implement line drawing (from point x0,y0 to x1, y1)?

I ended up with something like this:

void LCD_DrawLine(int x0, int y0, int x1, int y1)
{
int dx = abs(x1-x0), sx = x0<x1 ? 1 : -1;
int dy = -abs(y1-y0), sy = y0<y1 ? 1 : -1;
int err = dx+dy, e2; // our error val.

for(;{

tft.drawPixel(y0, DISP_HEIGHT-x0, ILI9341_WHITE);

if (x0==x1 && y0==y1) break;

// Line calculation using the
// Bresenham algorithm.
e2 = 2*err;
if (e2 > dy) { err += dy; x0 += sx; }
if (e2 < dx) { err += dx; y0 += sy; }
}

}

so drawing an ADSR-Envelope looks like:

LCD_DrawLine(20, 60, 0, 0);
LCD_DrawLine(50, 40, 20, 60);
LCD_DrawLine(80, 40, 50, 40);
LCD_DrawLine(110, 0, 80, 40);

(note that in this version i swapped the X and Y pairs)

[syso2342]

By the way: We can probably use this algorithm for smooth anti-aliased fonts
http://en.wikipedia.org/wiki/Xiaolin...line_algorithm

You might also want to look into http://freespace.virgin.net/hugo.eli...ics/x_main.htm

[PaulStoffregen]

I think I will have to take over this part. Let me see if I can find a little time and brain power (currently I have still a bit of a brianlag because of work + work on my synth).

Yeah, I know the feeling. Really, I should be focusing on getting a final 1.20 release done.....

By the way: Do you want to implement line drawing (from point x0,y0 to x1, y1)?

Adafruit_GFX already has Bresenham algorithm line drawing. A few days ago, I wrote this speedup, and later this other minor speedup.

By the way: We can probably use this algorithm for smooth anti-aliased fonts

This display does give us the ability to read its frame buffer. Anti-aliasing and alpha blending seem like really awesome features, but they also seem like potentially a huge time sink.

I think "only" implementing arbitrary size bitmap fonts, with a large collection of typefaces and sizes provided in the library, would be the feature most people would find most useful.

[syso2342]

I think "only" implementing arbitrary size bitmap fonts, with a large collection of typefaces and sizes provided in the library, would be the feature most people would find most useful.

Yes, I totally agree. Although I would suggest implementing Grayscale Fonts (instead of just pixel on / off binary notation).
I just optimized my font generator to squeeze the font down to 11x15 characters each leaving only a 1px space on each side
for separation between the strings (of course this could be cut down as well, but i would recommend keeping it for cosmetic
reasons.



Opening this in a Sprite-Editor (here tiled level editor) looks correct:

[PaulStoffregen]

My very old font code (for the 8051 chip) used 2 or 3 bytes for metadata on each character, and 2 more bytes per character in an offset table, because each character could be encoded with variable length data. The metadata allows the size, offset, and leading/trailing space to be specified, so each character only needs to have its actual data stored. This results in quite a savings for the many lowercase letters.

This stuff does make the font encoding script more complex, and it adds a bit more code on the Teensy side, but it's a trade-off that improves speed and saves storage in flash memory.

[Jayhawker]

I think this might be a place to start: http://www.seas.gwu.edu/~simhaweb/cs...6/module6.html
Search for "maximal rectangle problem", about halfway down the page.

You could start with identifying the largest rectangle within a given character and then find the next largest of the remaining area, etc. This probably wouldn't give you the optimal solution as I'm sure there would be cases where you'd be better off with a few "medium" rectangles rather than one larges and many small. You could always ignore the largest one (or two) and then compare to see which is better. The nice thing about this is that the code generating the rectangles doesn't need to be efficient. It can be a clunky, brute force type method since it only needs to run once for a given font.

(found another link: http://www.drdobbs.com/database/the-...blem/184410529)

Edit: This looks to be exactly what you're proposing: https://www.cise.ufl.edu/~sahni/papers/part.pdf
Unfortunately this only works for polygons without holes (such as a,b,d,e,g, etc.) Apparently it gets very complex when trying to deal with that. Here's a paper discussing that (behind paywall but I work at a university). Wish I had time to work on this, sounds like an interesting problem.

[scarmichael]

I've modified the optimized 8-bit version of the AdafruitTFTLIB for teensy 3.1 if anyone is interested. it uses one contiguous GPIO port for the data pins