Solved:
The best answer was given by sbi, but in the interest of completeness, some people might also want to know about a new way now available from Windows 10 RS1/1603.
It involves calling the SetFileInformationByHandle API with class FileDispositionInfoEx, and setting flags FILE_DISPOSITION_DELETE | FILE_DISPOSITION_POSIX_SEMANTICS
. See the full answer by RbMm.