Your typical voter in an April special is going to be looking at the proposed maps before voting. There's no D or R label on the referendum ballot, so having trained their voters to vote blue no matter who doesn't help that much on this one.
When you look at the map, it's obvious these polities are contradictory and nonsensical, won't benefit the people who live in these districts, won't benefit the Commonwealth as a whole, or any part.
For the yes side, their only argument is that it will likely result in 4 more Democrat seats in Congress. For the no side, there is a mountain of reasons not to do this - principled arguments against gerrymandering, practical ones about incoherent districts that would be difficult to represent given the vast differences in the regions joined together, and much more.
Anyone who is locally focused, regardless of party, gets totally hosed by this map; despite appearances, most politics is still local. But under this map, half the state would have to drive hours into DC metro traffic to get a meeting with their Congressman in his local office. A voter whose political interest is not strictly partisan isn't going to have any reason to approve this plan.
Perhaps most crucially, independent voters are being asked to go from king of the hill to the back of the bus, in rigged districts where their opinions no longer matter too much.
The referendum is boiling down to hard Ds vs literally everyone else; with a persuasion toolbox empty save for naked partisanship and insulting deceptions, it's going to be a major uphill climb to pull this off no matter how much money they pour into it.