I agree, the rear body would be a major job, It would need gutting inside & a new frame & walls, I have rebuilt a caravan that had major rot to the entire front & right side, it took around 50 hours to repair, and cost me around £270 in materials plus another couple of days to remove, reseal & replace the outer aluminium trims.
If it was closer to me, I would still buy it, as I have a scrap van with good mechanics & plenty of spare time to take on a challenge

I quite like the idea of starting from scratch, as the interior could be custom built to my own spec, and a really nice modernised interior would result

I looked at a fair few Bambi's before buying mine, all of which had damp/rot issues, but were well overpriced.
Even after showing the sellers the results of a damp meter, none were prepared to take a fair price, there is always some poor sod who will buy them without a clue to the hidden problems!
I was very lucky with mine, the only bits that needed repair were minor & quickly sorted out, the chassis & cab were as new, even the front arches are original & rust free, as the van was only used in summer & kept under cover during the winter.