I've just thought up a conceptually simple but tremendously difficult to implement and CPU-consuming way of working round this: avoid latency by seeing into the future.
Modern GPUs give you a ton of parallel processing. Old gamepads are binary with a fairly limited number of buttons not all of which can be pressed at once by someone with two thumbs. The emulated RAM state is also fairly small - kilobytes.
So, run lots of copies of the emulator. One for each possible change in button press status from the current state. At each frame (50/60Hz), look at the current actual inputs and pick (frame,20ms audio samples) from the available precomputed choices. Start calculating the next frames based on the winning version of the emulator state, and discard the rest.
(This is effectively branch prediction at the macrostate level).
> There are magic tricks beyond this, such as emulating every possible input one frame into the future, to cut out a single frame of latency. But with only one controller, this would require higan to emulate up to 4096 simultaneous SNES systems and well ... higan just isn't that fast, sorry.
Why does it need to emulate every possible input, rather than emulating a single path assuming that the controller state remains unchanged? You can only display one future frame to the user, after all, and that seems like the best one to display.
I realize a lot of the 4096 states are extremely unlikely (especially Up+Down or Left+Right ... not physically possible on an unmodified controller); but if the inputs you predict end up wrong, then you have no choice but to run the frame again normally. This is going to cause an extreme jittering where the input lag doubles for some frames.
Imagine audio stuttering from a scratched CD, or a framerate that suddenly dips from 60fps to 30fps for a moment and then resumes. With input, the effect is going to be even more jarring.
If you're going to do this, you absolutely can't have a miss. Ever.
The point would be to render the frame(s) before the input arrives, to reduce latency. You can only display one frame, but you don't know which frame until the user actually inputs something.
Oooh, I see! I was thinking of running the simulation forward by N frames optimistically, and then fixing it up when something else comes in, but the idea here is just to precompute every possible result. Nifty!
I do wonder how many states simpler systems like the GB or NES would have since they have fewer buttons, simpler audio, and less ram. Still probably a tough call -- for an accuracy-focused emulator that requires a lot of processing power this sort of solution might be out of reach for some years. You could probably do it on less accurate emulators, but then you're already relying on hacks & cheats so you might as well use other solutions for lowering input lag.
I think it's just barely possible for the fastest NES and GB emulators to use this trick for one single frame of lag reduction.
Here, you have eight inputs (assumes one player on the NES.) But there's a trick: up+down and left+right aren't physically possible due to a rocker in the D-pad. So as a whole, there are nine possible D-pad states instead of sixteen. With four buttons, that's sixteen possible states. So the total is 144 states.
Now look at QuickNES, blargg's masterpiece NES emulator that runs fullspeed on a 66MHz PowerPC system. If you had the fastest, overclocked octacore CPU Intel makes, you may be able to pull off 144 instances of the emulation. Especially since you don't have to actually output the audio and video for all the missed predictions to the real hardware.
Or perhaps more sanely ... most games don't have you using start+select while playing. I think people would be willing to accept a large jitter when pressing those two buttons. So now you're talking 36 instances, which starts to look very reasonable. At least, for emulators that are okay with sacrificing accuracy.
The most popular NES emulators today require 800MHz (Nestopia), 1.6GHz (Nintendulator), and I'm not sure on puNES, but presumably just as high. 36 instances of those are unlikely.
Still, this would be a fun experiment if anyone were willing! :D
Of course, the end result is a 16ms lag reduction. And again, I need to stress ... I get that latency stacks, but ... you're really going to have a hard time even telling the difference between the two. Even if you're one of those people who can beat Dodonpachi or Ninja Gaiden or Battletoads on one life.
Modern GPUs give you a ton of parallel processing. Old gamepads are binary with a fairly limited number of buttons not all of which can be pressed at once by someone with two thumbs. The emulated RAM state is also fairly small - kilobytes.
So, run lots of copies of the emulator. One for each possible change in button press status from the current state. At each frame (50/60Hz), look at the current actual inputs and pick (frame,20ms audio samples) from the available precomputed choices. Start calculating the next frames based on the winning version of the emulator state, and discard the rest.
(This is effectively branch prediction at the macrostate level).