Teensy 3.2 running at 96Mhz can handle 6MBit without flow control, as long as you process the incoming data fast enough.
Serial1 and Serial2 have 8-byte hardware FIFOs. An interrupt is triggered when they are half full. The interrupt handler transfers the data into a RAM buffer that's 64 bytes by default (can easily be increased). As long as there is no mis-behaved code that disables interrupts for a long time, there is no issue with FIFO overflow.
That SP3485 is perfectly adequate. Just wire up your RS422 to A / B and enable the receiver.
If you wanted to transmit data, you could just use 2 of them, one as transmitter and one as receiver.