You have every right to be there. Easier for him if you aren't but he can't force you to leave the house.

Edit: yes plenty of notice, most won't organise home opens until today anyway. Its becoming more a standard procedure to have ID when viewing for a range of reasons but definitely take jewellery with you or hide things from view.