You should be able to as long as you don't use a device with either 0x0A (SGTL5000) or 0x1A (WM8731) addresses, since those are on the audio board. The audio board has pull-up resistors on it, so you don't have to add pull-up resistors on the Teensy. I imagine there might be a few problematic devices, but it should work.