The Oxford Classical Dictionary (1970 edn) has this explanation:
'I was originally a simple stroke, V perhaps represented a hand, X is V doubled, while L, D and ∞ are thought to derive from the aspirated letters of the Greek alphabet for which early Latin had no use—L from [symbol that cannot be represented] (Chalcidic chi) … ∞ from Φ (Chalcidic phi) and [symbol that cannot be represent: D with a bar through the upright stroke] from Φ halved, with the medial bar added later to distinguish it from the letter D; C may derive from Θ, the third unused Greek aspirate, acquiring its final form by a combination of the tendency to assimilate figure-signs to letter-forms in current use and the influence of the initial of centum* …'
no subject
'I was originally a simple stroke, V perhaps represented a hand, X is V doubled, while L, D and ∞ are thought to derive from the aspirated letters of the Greek alphabet for which early Latin had no use—L from [symbol that cannot be represented] (Chalcidic chi) … ∞ from Φ (Chalcidic phi) and [symbol that cannot be represent: D with a bar through the upright stroke] from Φ halved, with the medial bar added later to distinguish it from the letter D; C may derive from Θ, the third unused Greek aspirate, acquiring its final form by a combination of the tendency to assimilate figure-signs to letter-forms in current use and the influence of the initial of centum* …'
* centum = hundred