This reverts commit 63756575b4.
Since this commit a imx6qdl-pico board boots extremely slowly
in both SPL as well as U-Boot proper.
Fix this regression by reverting the offending commit for now.
Signed-off-by: Fabio Estevam <festevam@gmail.com>
Reviewed-by: Jaehoon Chung <jh80.chung@samsung.com>
Tested-by: Pierre-Jean Texier <texier.pj2@gmail.com>