From Table 10, 5th line down, Pixel port P19], the up arrow indicates that B[7] is valid on the rising edge of the clock, and the down arrow indicate R[3] is valid on the falling edge of the clock. Figure 5 shows the timing for valid data based on the rising and falling edges of the clock.
If you are in DDR mode the clock (LLC) will be 1/2 the pixel clock. Regardless of how you set up DDR mode, the FPGA will have to sample on the rising and falling edge of the clock to get the full RGB data.
The ADV7181C can only output RGB in DDR mode since it only has 20 output pins. In general this part is not used for RGB due to this limitation. If you want to get 24 RGB SDR then look at other parts like the ADV7842.