All players have the same issue, just not to the same degree. There are two parts to your issue.
1. "especially when I have a nice advantage"
This is due to relaxation. To overcome this, you need to cultivate a healthy paranoia. For this I recommend endgame studies. The specific reason is that unlike other puzzles, studies are guaranteed to have defensive resources that must be overcome. Your opponent is the study composer, and the first good idea that comes to your mind
does not work, for a fiendish reason that the composer has prepared for you. (Maybe the second good idea also does not work.) Two books I like for the practical player are Ban
Tactics of Endgames and Averbakh
Chess Tactics for the Advanced Player; of course there are others. Endgame studies are about
the exception, and are good for developing intuition for resources.
2. "completely overlooking strong replies my opponents can play"
It can't always be relaxation, or it would
only happen when you have the advantage. Do you find that you often see the strong reply
immediately after making your move on the board? If so, then it's a failure of visualization. To specifically strengthen visualization, I recommend blindfold chess. It's pretty easy to work into your current practice. If you play against an engine, look for the setting which will hide the pieces. If you are going over master games (which you should), then you can try to play through the game without a chessboard. In both cases, the important skill is to actually see the position on the board. If you can accurately calculate variations, even short ones, from the current position, then you are on your way.
And suppose you have good visualization but still have this problem? In that case it's not that you can't visualize, it's that you fail to do it. Here something like Blumenfeld's Rule can help. It was in Kotov's classic
Think Like a Grandmaster. That book is worth reading if you have not done so. Kindle edition is a bargain at $2.99 on amazon
https://www.amazon.com/Think-like-grandmaster-chess-book-ebook/dp/B00QT4G56E.