Small Jibs and Big Mains
Empirical approach.... (?)
Use main halyard to hoist end of tape to masthead and measure to foot of mast. Repeat with jib halyard. Precision to the last mm is not vital, IMHO.
(I used to race and cruise a Niagara 26 for a decade, with a 3/4 (approx.) fractional rig. It routinely outsailed boats 4 foot longer.
Worthless trivia: we were really looking for a big fractional-rig boat when we found the deal on the Olson... Liked it so much for the rest of the design parameters that we bought it anyway...
and another decade has passed...
Loren in Portland
("What I really wanted to be... was... a Lumberjack!")