For an example of the slowdown, does it tie up a single shared memory bus so the CPU is effectively stalled (except for cache) while the DMA is running?
No, the potential issues aren't hardware based. Well, except that the hardware provides features with limited scope.
CPU efficieny-wise, the hardware side is excellent. It has a bus matrix which even allows multiple memories to be used simultaneously. For example, in the common case where the CPU is mainly accessing code from ITCM and data from DTCM, the DMA controller can access RAM2 (aka DMAMEM) simultaneously. Of course, if both CPU and DMA want to access the same memory region at the same exact moment, one must wait. But that is only a matter of a few clock cycles, not tying up the CPU.
The possible slowness issues are all about software design you might need if the hardware's features aren't a good fit to the requirements of the serial communication protocol you will implement. Even with the best possible hardware, when that hardware capability doesn't nicely fit your needs, it's still very easy for software that fills in the gaps to become complex and end up slower than a simpler approach.
but I'm still kinda fuzzy on the details of why.
It all depends on the nature of the serial communication protocol you wish to implement. The DMA controller fundamentally works by configuring fixed size transfers. If your protocol uses large fixed size blocks or "packets", DMA might be very efficient.
But most serial communication protocols aren't built around large fixed size messages. If your communication protocol will use small or variable size messages, or messages requiring parsing incoming bytes, then it just isn't a good fit to how the DMA hardware works. The overall performance depends on deep low-level details, much lower level than shown in this high-level pseudo code, of how you will deal with that mismatch between the DMA controller's fixed size transfers and whatever your specific data sizes and parsing the serial protocol requires.
So to explain, I'll try make some bad-case assumptions. This isn't to say you will write such slow code. I'm only trying to answer your question to explain what could potentially go inefficiently...
DMA transfers use a TCD (Transfer Control Descriptor) structure which eight 32 bit words. TCD is documented in the
reference manual at page 115. But some of those 8 words are sub-divided into smaller fields and even bits for various status. The DMA controller has some awesome hardware features which potentially let you reuse most or even all of the previously set TCD when your next transfer will repeat the same way as the last. But that hardware becomes less useful if you will need the TCD to be different for each transfer. Since we're talking about a bad case scenario that ends up inefficient, let's assume your program ends up reconfiguring all the TCD fields for every transfer, even if you don't need to add complex code features you can't get from the TCD (many examples of those serial protocols below).
DMA can still be a huge win, even if you spend a lot of cycles configuring the TCD. The more bytes each transfer moves, the more you amortize the cost of configuring TCD and other overhead to manage everything. If you move thousands or maybe even dozens of bytes per transfer, you can probably achieve a huge improvement over regular interrupt-based serial where a small ISR function runs for every few bytes (assuming the small FIFO helps a bit). DMA is really good for uses like audio where you send or receive the same buffer every time, so the TCD just gets reused.
But if use is a very small transfer, or in the worst case each transfer moving only 1 byte, all that overhead to set up the TCD and manage the process can become a substantial cost. Why would that be a concern? Well, let's look at these lines from your pseudo code:
C++:
while(true)
{
uint16_t rx_char = SerialX.read();
if (rx_char < 0)
{
break;
}
This suggests your application would want to receive the incoming data 1 byte at a time. Maybe you can find an effient way to do this, but the concept of digesting the incoming data in 1 byte increments is probably a poor fit to the way DMA works efficiently in relatively large transfers. If this pseudo code showed checking the total number of bytes and handling them all as a block, DMA would likely be a better fit to the nature of the DMA controller which transfers whole buffers which are generally intended to be much larger than a single byte.
The DMA controller really shines when trasfers are large and uniform in size. Part of your pseudo code has BUFFER_TX_SIZE which seems to suggest you're planning to use big fixed size buffers for transmitting. So DMA can be a good fit to some serial protocols like
DMX512 where you repeatedly transmit a large message. But DMA can be a poor fit to many other serial protocols where small different size messages are transmitted.
When we talk of performance, often there can be many conflicting goals. So far your question seems focused on minimizing CPU overhead. Indeed that can be a worth goal if you need the CPU available for other tasks. But often with serial protocols, other goals are much more important.
Communication latency is often of overriding importance. Many serial protocols involve one side sending commands or questions, where the other side (often using a microcontroller like Teensy) replies with status or answers. In those situations, you want to reply as quicly as possible, because the other side is waiting. Even if you expended CPU cycles to receive each byte as the arrived, and you will spend more CPU cycles as the response transmits, that "wasted" CPU time occurs mostly during the data transfers without impacting your response latency. With DMA, you (probably) suffer all the overhead at the moment that impacts response latency. But as before, this involves a lot of speculation. Maybe the overhead is minimal, especially if the TCD is fully or mostly reused.
Latency can really suffer if your receiving side uses a large buffer which must fill up (complete the DMA transfer you set up in the TCD before receiving even the first byte). It might not be able to work at all if your receiving code doesn't respond until the buffer fills but the other side which send a short command expects a response before it will send another command. Maybe you can find some way to work around this and still use a large transfer size, but then you'll be adding code to inspect the DMA registers before the transfer completes, and maybe even look in the buffer or play other tricks. Meddling with hardware while it's active risks horricly tough bugs like race conditions, so this would need to be done cautiously. Hopefully it's easy to imagine how that could become a very complex problem that gets solved by more and more code which consumes more CPU cycles. Then again, Teensy 4.x CPU is fast, so even quite a lot of software overhead might be acceptable, especially with medium or slow baud rates.
But as a general rule of thumb, the DMA controller might be a huge win for protocols where you regularly transmit the same large message, and where receiving where the incoming messages are large and/or fixed size, especially where the nature of transmitting and receiving looks more like independent messages or "packets" where neither side is waiting on the other. Sadly, that's not how most serial communication protocols work, or at least not the ones I've seen and implemented.
DMA might is probably not a good fit for serial protocols which use small or variable size messages. If the protocol is built around command-response messages, DMA might be more CPU efficient overall but might hurt latency.
If the protocol uses different size messages and you need to parse the just-arrived message to determine how much to transmit, you will end up reconfiguing the TCD. How much overhead that adds if difficult to predict. But hopefully this gives you a way to imagine how DMA might not necessarily be the best solution.
Even worse are serial protocols where you must parse the first bytes of an incoming message to know how long the entire message will be. The TCD doesn't give you any way to parse data as it arrives. It requires a fixed size to be configured before any data transfer starts. Likewise for serial protocol with 1 or more reserved bytes, which use an "escape character" where certain 8 bit data becomes 2 bytes. For transmitting you end up sending different size messages, but the huge problem with DMA is you just don't know how large the incoming message will be. Maybe you'll configure the TCD for the minimum size if no bytes are "escaped", then once you've parsed the bytes you receive configure another TCD if you discover any escape chars, and keep repeating if the next TCD receives more 2 byte sequences until you have the entire message. Or maybe you go with some sort of scheme to inspect the incoming data before the TCD completes, possibly with a large TCD reused over and over? Whatever you do, the point is extra complexity.
Many protocols which use a reserved byte typically also are implemented with parsing on every incoming byte, where you immediately discard any previous partial message and begin receiving an new message upon unexpectedly seeing the reserved byte which indicate the beginning of the message. This just doesn't fit the DMA controller's fixed size transfer approach. Again, you might find a way to make it work, but at what cost of complexity and extra CPU usage on the software side.
Some serial protocols, like Modbus RTU, are often implemented with a inter-byte receive timeout which is similar or even shorter than the full imcoming message time.
On top of all that, many serial protocols are implemented with ways to recover from errors. Serial isn't like USB or Ethernet where the hardware detects corrupted data. Again, your concern should be how well any sort of error recovery scheme fits to the TCD transfers. If you will need to possibly take some recovery action immediately upon parsing every incoming byte, the DMA controller is probably a poor fit.
Hopefully this admittedly vague answer helps explain why DMA might not end up working efficiently, despite the amazing hardware features and bus switch matrix that allows it to run very effiently with little or no impact on the CPU during the DMA transfer. It really all boils down to the DMA controller working in fixed size transfers which can't parse data as it arrives. If you try to use DMA for those sorts of protocols with requirements that don't easily fit the way TCDs work, a lot of potentially quite complex code will be needed. Complex code isn't necessarily slow or ineffecient, but I hope this long-winded explanation helps you to imagine how it could go wrong.
I should also quickly mention this is only about the general purpose DMA controller. Not all DMA is with the general purpose DMA controller. Some peripherals have built in DMA, especially USB, SDIO, Ethernet. Those don't use TCDs and everything I've said here doesn't apply. But with hardware serial and most of the other peripherals which don't have it built in, DMA support is always done with the general purpose DMA controller with TCD configuration.
