This can be done with a nanoflow, you will need the ‘Nanoflow commens’ module.
Create a nanoflow with 2 actions: Sign out and redirect.
Redirect to a logout.html file that you need to create in your project folder, and on that logout page do a redirect to your IdP wich does a logout of that IdP.
It is also best practice to add a button to the logout.html wich redirects to login.html so the same user (or another) can log in again easy.