If the digital sources are tri-stated or set low then it's OK. The problem arises when the source goes high driving into a non-powered input. The current flow path would be:
source power rail --> source resistance --> trace resistor
--> sink ESD protection --> sink power rail --> through sink device
--> return back through ground path.
This applies to all designs, not just video circuits or the ADV7343
The ESD protection circuit is generally not designed to handle continuous current but momentary ESD hits.
Here's a thread about this issue: Re: AD9985A Analog Inputs [VD to 0.0 V] ABSOLUTE MAXIMUM RATING
So I would avoid powering up the ADV7343 input pins if the device is not powered up.