If you activate GPS logging (show track on map) you can see every place you have been, unless you delete the GPS log.
Would it hurt to keep overlays until they are deleted?
The routing|delete feature has already separate check boxes for route and overlay
Manfred