The B20 swaps and LSVtecs and what not are visibly different, that's why the referee fails them. They fail on a visual, in Cali it has to be the same model year motor or newer, and originally available in the body.
The 5.4 will have adapter plates between the intake and head, powdercoat them black, and it will look like stock. Now on the exhaust side, you'll want to keep the stock manifolds, and get a really good exhaust shop to mod your factory mid-pipe, so when they look at it, they see 6 cat's and no welds.
Of course, in addition, your tune will need to be spot on, so budget for a dyno session with someone reputable.
On the roller's there's no reason it shouldn't pass.
Legally though, no it is not legal in California, since the 5.4 was never available in the SN95 body. Shelby swaps are legal for S197 owners though...