Was he gay? Hard to know. He had a series of female friends all his life, without it seems having sex with them, though that could have been the heroin, but he married twice; he probably had a love affair with Dick Twardzik his piano accompanist; he was beaten up once after a show, allegedly for trying to buy drugs; and anyway, he was far too beautiful to be straight. Gay or not, women and gay men adored him, and you can see why in the video. The song is called My Buddy, and it is (deliberately? -- this was the mid-50s after all) ambiguous. But this:
Miss your voice, the touch of your handCome now. That's fairly direct, no? Bi*, I suspect, but like Jack Kerouac homophobic. It drove Kerouac to alcohol and Baker to drugs. Perhaps. Baker's emotional aloofness could have been because of that. Lots of clues, but who knows in the end? The song, though, is revealing.
Just long to know that you understand
*If I might be permitted to use a label I've just dismissed as useless and misleading.