In this approach the server is calculating all the game logic in a steady loop while clients send input events with attached timestamps. Important note: the server receive events ahead of time, when the server is working with the step #1000, it can receive events for #1001, #1002, etc. So when the server begins step calculation all input events for that step already received and the server don't cares about lags.
The term 'client side prediction' comes from the fact that clients must send input events ahead of the server time. If a client, for example, have RTT (ping) of 10 steps then while receiving from the server step #1000 that client must send input events for at least step #1010 to arrive in time. So, to achieve zero visible lag, that client must display step #1010 while having data only for #1000. And there client side prediction comes: given server's data for step #1000 and current player input client predicts world state for step #1010.
Prediction works good for local player (given stable RTT and low packet loss), but client haven't last input events for other players. So one of the main concerns is dealing with prediction errors (usually some sort of interpolation).
If I am not totally wrong this is exactly how Valve does when they code their server/clients.
So this would reduce the latency problem by smoothening it out locally from the predictions. This might take some time to code, but I will be back with some updates if I need any help or reviews.