Persistent Unicorn JIT cache + Android fixes - #13
Merged
Conversation
Adds a persistent TCG code cache to Unicorn via a version-portable build-time patch system, plus the TCG and Android work that came out of making it releasable. Patch infrastructure. Brovan/native/unicorn/ holds Brovan's additions to the Unicorn tree; Brovan.Unicorn.targets applies them after extraction through a RoslynCodeTaskFactory inline task — no git/patch/python dependency, works on Windows, Linux and Android build hosts. Edits are single-line anchors and whole-file appends rather than hunks, so they survive upstream drift; every rule is idempotent, and a missing anchor fails the build naming all misses. The patch-set hash is part of the source and build directory names, so editing a patch forces a clean re-extract. Validated against the previous two Unicorn tags. Code cache. Saves the TCG buffer and reloads it on the next launch. Helper addresses route through an indirect slot table; uc/tcg_ctx are pinned in a replayed address reservation chosen from a probed candidate list. Every restored block is re-verified against the guest bytes it was translated from, and page_addr/hash are recomputed from the live mapping so self-modifying-code invalidation still finds them. A save-time audit scans emitted code for host pointers a reload could not repoint and refuses rather than writing a poisoned blob — it caught a real miss (the i386 backend tail-jumps into qemu_st_helpers rather than calling it). Blocks in pages the loader hasn't reached yet are retried later instead of re-translated, and the blob is dropped once mostly dead code, bounding growth. Inline hooks stay enabled. Slots carry a kind — image-relative for helpers, hook identity for callbacks re-resolved from uc->hook[] on load — so the cache no longer has to disable Unicorn's inline-hook path. That was costing ~13% throughput. Per-block exit poll inlined. Unicorn calls a helper on every block to test icount_decr; this emits a load and a not-taken branch, with the helper kept for the slow path, following QEMU's shape of branching out of the block to a trailing label. Direct register access. brov_reg_ptr hands out host pointers into the guest CPU state, used by ReadRegister/WriteRegister through unsafe loads and stores. Only registers Unicorn stores verbatim are exposed, and only those it would write with a plain store are writable — the program counter is read-only because writing it also raises quit_request and flushes blocks, and nothing is exposed in 16/32-bit mode where the same storage is reached under different truncation rules. Android. JIT caching switch in Settings, default on. The Vulkan shim is a guest PE, so it now ships as an APK asset and is deployed into the guest's System32/SysWOW64 on launch, refreshed on app update. The cross-build no longer shares a CMake cache with the host build. API set map from the installed image. Read from the .apiset section of the imported apisetschema.dll instead of being dumped from the host or synthesised, so the contract names match the DLLs actually installed. Regenerated when the schema is newer than the map.
Owner
Author
|
@claude review |
|
Claude encountered an error after 2s —— View job I'll analyze this and get back to you. |
Owner
Author
|
Weird. already reviewed locally anyway. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a persistent TCG code cache to Unicorn via a version-portable build-time patch system.
Patch infrastructure
Brovan/native/unicorn/ holds Brovan's additions to the Unicorn tree; Brovan.Unicorn.targets applies them after extraction through a RoslynCodeTaskFactory inline task — no git/patch/python dependency, works on Windows, Linux and Android build hosts. Edits are single-line anchors and whole-file appends rather than hunks, so they survive upstream drift; every rule is idempotent, and a missing anchor fails the build naming all misses. The patch-set hash is part of the source and build directory names, so editing a patch forces a clean re-extract. Validated against the previous two Unicorn tags.
Code cache
Saves the TCG buffer and reloads it on the next launch. Helper addresses route through an indirect slot table; uc/tcg_ctx are pinned in a replayed address reservation chosen from a probed candidate list. Every restored block is re-verified against the guest bytes it was translated from, and page_addr/hash are recomputed from the live mapping so self-modifying-code invalidation still finds them. A save-time audit scans emitted code for host pointers a reload could not repoint and refuses rather than writing a poisoned blob. it caught a real miss (the i386 backend tail-jumps into qemu_st_helpers rather than calling it). Blocks in pages the loader hasn't reached yet are retried later instead of re-translated, and the blob is dropped once mostly dead code, bounding growth.
Inline hooks stay enabled
Slots carry a kind. image-relative for helpers, hook identity for callbacks re-resolved from uc->hook[] on load. so the cache no longer has to disable Unicorn's inline-hook path. That was costing ~13% throughput.
Per-block exit poll inlined. Unicorn calls a helper on every block to test icount_decr. this emits a load and a not-taken branch, with the helper kept for the slow path, following QEMU's shape of branching out of the block to a trailing label.
Direct register access
brov_reg_ptr hands out host pointers into the guest CPU state, used by ReadRegister/WriteRegister through unsafe loads and stores. Only registers Unicorn stores verbatim are exposed, and only those it would write with a plain store are writable — the program counter is read-only because writing it also raises quit_request and flushes blocks, and nothing is exposed in 16/32-bit mode where the same storage is reached under different truncation rules.
Android
JIT caching switch in Settings, default on. The Vulkan shim is a guest PE, so it now ships as an APK asset and is deployed into the guest's System32/SysWOW64 on launch, refreshed on app update. The cross-build no longer shares a CMake cache with the host build.
API set map from the installed image
Read from the .apiset section of the imported apisetschema.dll instead of being dumped from the host or synthesised, so the contract names match the DLLs actually installed. Regenerated when the schema is newer than the map.