I think (as far as I understand it, which isn't very - though thanks to your book rec that may change!) that Latin and French both fulfil some of the functions for English that Chinese did for Japanese. Latin provided a writing system and a body of classical literature, just as Chinese did. But in terms of actual linguistic borrowings Old English took relatively few words from it, certainly by comparison with the amount that came from Norman French after the Conquest, which permeated the English language to a degree perhaps more analogous to Chinese linguistic influence on Japanese. (Of course we have a lot of Latin words now, but I think the majority came later, from the Renaissance on.) That's my picture of it, anyway, but I'm happy to be corrected.

My favourite set of pairs are the circumflex twins, where the same word was borrowed twice, once with the 's' pronounced as in Norman French, and then with the 's' discreetly hidden under a circumflex as in modern French: paste and pȃté, feast and fȇte, hostel and hôtel, etc. When learning French this always seemed like a magic trick!