Several years ago, when I was playing with RPIs on a robot, and using a teensy as the Servo controller. I would do this headless as well.
Sometimes I did this, using Arduino IDE running on the RPI using VNC setup to my PC.
Most of the time however, I would build on the PC. I would logically use the Sketch->Export Compliled Binary command, which would put a copy
of the hex file in the sketch folder. I would then SCP the file over to the RPI, where I would then use teensy_loader_cli to program the teensy.
As I mentioned before in other threads, at one point I hacked up the boards.txt/platform.txt stuff, such that I had menu item like upload using that could choose local or remote build and on the remote one I hard coded in network address of the RPI, with a command line SCP that would automatically transfer the file to specific directory on the RPI (or at that time might have been BBBK or Edison or Odroid), and I would have a script running on the RPI (or other), that would detect if a file was modified, and if so call of to teensy_loader_cli with it....
But I have since lost that setup as it was at least 5 years ago.
At some point you will probably be able to do it using ArduinoCLI running on the RPI.
I believe they are currently building 64 bit and maybe 32 bit ARM versions these days.
https://arduino.github.io/arduino-cli/0.29/installation/
But I don't believe we have version of the ARM teensy builds yet?