Another issue is scrolling visual lines one by one that have been wrapped. (N)vim can only scroll by literal lines, so if a line is longer than the screen and wrapped, there is no way to step through line by visual line.
That’s strange. I just tested it and both GVim on windows and vim on a vps linux terminal have no problem with going anywhere on a 170k long line. It lags a little but visually is fine.
And just in case you missed it, you may gj gk to navigate visual lines or remap j, k (and <Up>, <Down>) to gj gk to make that default.
I have a genuine question why use nvim at all if it has such issues with presenting text in a window.
https://github.com/neovim/neovim/issues/8715