Someone has probably written about this earlier, but since it is still marked as "TODO" in Teensyduinos example sketch "MIDI_name" and you are asking for someone to contribute with information, I will do it anyway.
The reason that osx "remembers" the old MIDI name of you teensy midi device even after you have changed its name, is that osx will automatically create MIDI devices in "Audio MIDI Setup" when you plug in your MIDI gear. So to get osx to use the new name of your teensy midi device, open Audio MIDI Setup, go to your MIDI configuration, and delete the teensy MIDI device with the old name. Next time you plug in your teensy MIDI device, Audio MIDI Setup will create a new MIDI device with the new name you have created.
That's it!
The reason that osx "remembers" the old MIDI name of you teensy midi device even after you have changed its name, is that osx will automatically create MIDI devices in "Audio MIDI Setup" when you plug in your MIDI gear. So to get osx to use the new name of your teensy midi device, open Audio MIDI Setup, go to your MIDI configuration, and delete the teensy MIDI device with the old name. Next time you plug in your teensy MIDI device, Audio MIDI Setup will create a new MIDI device with the new name you have created.
That's it!