How to block direct access to my custom 404 page in Apache?

Question

I am using Apache2 as web server in Ubuntu and my root is /var/www/html. In there I have an .htaccess file with

RewriteEngine on
ErrorDocument 404 /custom404.