How to Change the Font in Messages in Yosemite
Some research revealed a trick to bring back some, but not all, of Messages’ previous configurability. It turns out that if you select Reduce Transparency in System Preferences > Accessibility > Display and then relaunch Messages, the Text Size slider is swapped for a pop-up menu that lets you choose from the Default (which I think is the system’s Helvetica 13), Optima 12, Palatino 12, and Other. Optima 12 and Palatino 12 are holdovers; they’re default options in the equivalent My Font/Sender’s Font pop-up menus in Mavericks.
Choose Other and you can use the standard Fonts panel to pick any combination of font family, typeface, and size you want. Your selections apply both to your bubbles and those from your correspondents.