winpin: Pinball Soccer '98

[avanturist888]

First of all, thank you for this project. I found it through your blog post and it is easily the most fun thing I have read about in a long time. Watching an exe turn into Rust that just runs is quite something. After spending a while in the code I decided the best way to say thanks was to actually pitch in rather than watch from the sidelines, so I picked a game and worked until it ran.

This adds a new target: Pinball Soccer '98, a 1998 Windows pinball game. It gets to actual gameplay, both natively and in the browser.

A word on size first. The diff is 385k lines, but 379k of that is generated code under out/winpin/. The hand-written part is about 6k lines across 68 files. If you would rather take this in pieces, or not at all, just say so and I will split it up or close it.

tc

Most of it already worked. The gaps were all around finding code:

• MSVC switch tables indexed backwards from the displacement (sub ecx, 4; jb; jmp [ecx*4 + table])
• Taking the table length from a preceding and mask, so a table whose first slot is padding does not cut the scan short
• Scanning for function prologues in the gaps between known blocks
• A feedback loop: the runtime appends addresses it could not resolve to a file (THESEUS_MISSING_ADDRS), which you feed back in with --entry-points-file

Together those take .text coverage on this exe from 6% to 93.9% (24113 blocks).

Generated output is now split into part_NN.rs files of roughly 1MB, which took a cold build of this target from 60s to 24s.

winapi

New files: dinput (keyboard and mouse), the mmio family in winmm, ole32 and msacm32 stubs, and a shared input state in user32 that both dinput and the message pump read from.

Filled in: a software mixer for dsound (resampling, volume, pan), colorkey blits and palette handling in ddraw, LoadLibrary/GetProcAddress backed by a module registry, and a fair amount of kernel32's file and NLS surface.

One piece worth separating

runtime/src/ops/misc.rs fixes a bug that predates this branch: setge tested ZF == OF instead of SF == OF. It is buried in 0c7df91 along with a lot of unrelated work. Happy to lift it into its own commit if you want it on its own.

Things I am unsure you will want

• rust-toolchain.toml pins nightly for the whole workspace. tc already required nightly (push_mut), so this only writes down what was true already, but it does apply to crates that build fine on stable. Easy to drop.
• catch_unwind around each instruction in codegen turns an unhandled instruction into a todo!() in the output instead of stopping the build. The aggressive scanning turns up junk blocks that would otherwise be fatal, but the cost is that a genuine mistranslation gets logged rather than failing loudly. I would rather do whatever you prefer here.
• static-server.go grew /log and /frame endpoints so a script can watch a page it cannot see. Reasonable to want those out, or behind a flag.
• translate.sh has my local path to the game.

Known gaps

I went over this before sending and there is a list of things I know are wrong but have not fixed, nearly all in code this branch adds. The main ones: wsprintfA does not cap output at 1024 the way the real one does, wildcard_match in kernel32 backtracks exponentially, the wasm filesystem leaks a string per path component, and refcounts on ddraw and dsound objects are fake, so a balanced AddRef/Release frees a live object. I am happy to work through them. I just did not want to keep sitting on the branch without knowing whether you want it at all.

State of the game

Intro, language select, table select, and gameplay with working flippers. Sound works in the browser.

[evmar]

Wowwwww this is incredible! It will take me a bit to get to this but at a skim it is great and I want it all. I am fine with code that partially works as long is it is in the right direction, so the missing things you mentioned sound fine.

I am extra impressed you navigated through all of the random places I did something quite hacky, sorry for those!

A few first-glance comments:

• Maybe we shouldn't check in the generated code for this? It seems pretty large. In general I probably shouldn't be checking in any of this generated code, it's just been very convenient for diffing purposes to be able to see what changes when I change the generator. I briefly experimented with putting the generated code in a separate repo but it quickly got kind of annoying to manage. Having all of these programs checked in is also annoying because if I change any API then my editor re-type-checks all of the programs and spews a bunch of errors...

• Is BlockMap a hash table? Why not a Rust HashMap? (I only skimmed.) The binary search is definitely bad! I did some experiments in retrowin32 which had a similar thing, and found that a trivial cache https://github.com/evmar/retrowin32/blob/7c9243072755da54054bcaa9e1351371af638ded/x86/src/icache.rs#L142-L143 had a surprisingly great hit rate, like 90+%, because generally you have small hot loops. My long-term idea here is we could use a perfect hash table.

• Regarding splitting up the generated code, what you have seems fine, though it's disappointing it takes so long to compile! The main idea I have here is that I could build a block dominator graph to find isolated subcomponents that then could maybe get compiled in parallel in their own crates, but it might be a stretch. The other thing that will help is not splitting into so many tiny functions if it can translate loops/ifs/etc to Rust-level loops/ifs/etc. I've been toying with such tiny toy programs I haven't thought about it too much yet.

• I think I only depended on nightly for the wasm bits, because I need -Z build-std to enable Rust's support for JS workers.

[evmar]

Oh and re the catch_unwind thing, one trick I've been doing you can replace most todo!() (which is a crash at compile time) into a self.todo() (which puts a todo!() in the output, which is a crash at runtime only if the code is hit). There's also a bit of support for the translation returning errors to drop a block entirely, like see the "suspicious block of 0" bit of code.