Found the issue:
I was trying to check if the field was null in views with twig, but it seems it's never null. Checking if it's empty worked.
Looks like the twig date filter returns the current date when the given variable is empty, so that was throwing me off.