Audio TDM is a high data rate signal, or rather 3 signals (two clocks and data), one of which is a clock with requirements for low-jitter.
For long distance transmission you'll need something like RS485 or LVDS for all 3 signals. CAT5 would be adequate for carrying the signals.
There is the issue of what happens if one end is powered down?
There is also the issue of different ground potentials at different ends?
This might imply use of logic-speed galvanic isolators.
Audio TDM is meant for inter-IC communication on the same PCB, its not easy to drive long haul (partly
because its multiple signals and partly the presence of a DC component on the data signal, and partly
the jitter requirements).
Typically for multichannel digital audio long-haul comms something like ADAT lightpipe (optical) would be used, but this looks
complicated/expensive I fear.