Skip to content

Latest commit

 

History

39 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

wiikit

A game-agnostic toolkit for Wii reverse engineering and native PC ports: disc images, executables, the Gekko CPU, a static recompiler from Gekko code to C++, and a runtime that replaces the Wii's hardware under the recompiled game. The GameCube, the Wii's older self (the same processor, GPU and DSP), is served by the same pieces and its own few: its discs, its disc drive, ARAM, the controllers on SI.

The same idea as ps2kit, for the Wii. Each Wii game has its own engine and formats, but a large part of every port is the same work: the same disc encryption, executable formats, CPU, SDK and GPU. wiikit collects that shared part. It grows inside the ports: each piece is written because a game needed it, then kept free of that game's knowledge. Game formats and game fixes live in the ports.

Ports built on it

Port Game What it asked of wiikit
pc-victorious Victorious: Taking the Lead (2012) everything so far: the disc, the symbolised ELF, the recompiler, the runtime from __start to a played, heard, 16:9 game
pc-dragonquestswords Dragon Quest Swords: The Masked Queen and the Tower of Mirrors (2007, PAL) a stripped executable (function discovery, names by signature), RVZ images, the PAL boot and EuRGB60, the locked cache's DMA, thousands of draws a frame, audio under load, the Remote's speaker, swings from mouse drags
pc-conduit2 Conduit 2 (2011) the 2010 SDK's KPAD Ex forms, IOS replies after the call (a NAND race), the GP FIFO and display lists in MEM2, RG8/GB8 EFB copies, the relative mouse, the Classic Controller and SDL gamepads (split-screen), the F12 GX trace
pc-arcrisefantasia Arc Rise Fantasia (2009) the 2007 SDK's KPAD and WPAD (the Classic read from WPAD's own samples, and from the copy of them only that KPAD keeps), thousands of tiny skinned strips merged into single draws, the disc's files sized from the FST, the renderer's profiler
pc-megamanxcm Mega Man X: Command Mission (2004, GameCube, PAL) the GameCube: its discs, its clocks and the IPL's globals, the disc drive at its registers, 16 MB of ARAM, the controllers on SI, the GameCube's AX micro-code under MusyX (samples in ARAM, per-millisecond updates), drive and DMA times as long as the console's
pc-monsterhunter3 Monster Hunter Tri (2009, PAL) RSO modules (most of its code, loaded at run time) recompiled with the executable, the WPAD data format and pointing camera, USB keyboards' /dev/usb/hid, the CPU's reads of the EFB, the AI clock waiting for the DSP's interrupt
pc-thelaststory The Last Story (2011, PAL) FIFO breakpoints that stop the GP (its engine paces its frames on them) and the CP's overflow behind them, the PI's FIFO in 32-byte units, EFB copies written into RAM where the game reads them on the CPU (its dyed equipment, compressed to CMPR), WPADGetInfo
pc-finalfantasycrystalbearers Final Fantasy Crystal Chronicles: The Crystal Bearers (2009) the Nunchuk (its stick and C and Z on keys), a third KPADStatus size (0xB0), shader programs shared by their code and kept on disk (a game of hundreds of TEV set-ups, its cut-scenes no longer stalled by links seen before)

Each port says where it stands in a .recomp.json at its root (spec): the game, its system, "toolchain": "wiikit" and a status, which trackers such as recomp.board read. A new port adds one, and changes its status in the same commit as its README's.

Some pieces were first written for two earlier Wii studies (The Last Story and Final Fantasy Crystal Chronicles: The Crystal Bearers): the disc extractor, the GX texture decoder, TPL, U8 and DSP-ADPCM.

Using it

A port takes wiikit as a git submodule at wiikit/, so that python -m wiikit.… works from the port's root and the recompiler finds wiikit/runtime next to itself:

git submodule add https://github.com/vs-sr-dev/wiikit.git wiikit
git clone --recursive <port>          # or: git submodule update --init

Each port pins a wiikit commit and moves it forward deliberately.

python -m wiikit.disc GAME.wbfs --info           # .iso, .wbfs, .rvz, .wia; Wii or GameCube
python -m wiikit.rvz GAME.rvz                   # its tables and compression
python -m wiikit.disc GAME.wbfs --extract build/extract
python -m wiikit.dol build/extract/sys/main.dol --info
python -m wiikit.ppc build/extract/sys/main.dol --at 80006124
python -m wiikit.cw 'process__19CSongMoveBlockActorFf'
python -m wiikit.u8 ARCHIVE.arc
python -m wiikit.rso MODULE.rso [--exports|--imports|--relocs]   # an RSO module or a .sel
python -m wiikit.tpl TEXTURE.tpl build/out
python -m wiikit.recomp GAME.elf --out build/recomp [--hooks game-hooks.txt]
python -m wiikit.recomp main.dol --out build/recomp --symbols names.tsv   # stripped
python -m wiikit.recomp main.dol --out build/recomp --symbols names.tsv \
    --sel files/GAME.sel --rso files/MODULE.rso ...                      # with its RSO modules
python -m wiikit.profile build/recomp-build/wiiboot.exe run.err [build/symbols.tsv]

Building recompiled code needs CMake, Ninja, a C++20 compiler (clang from MSYS2 so far) and SDL3; running it needs OpenGL 4.5. The generated project includes runtime/runtime.cmake; a port adds its own targets and its own layer (RtGameLayer) with -DWIIKIT_EXTRA=file.cmake.

Layers

Layer Question it answers Now Next
1. Recognise What is on this disc? disc --info: game id, Wii or GameCube, partitions, WBFS usage; rvz: RVZ/WIA tables a fingerprint: magics, SDK library dates, middleware found by symbol or string (Scaleform, Wwise, Bink, NW4R, Home Button)
2. Extract Turn standard formats into standard files disc (ISO, WBFS, RVZ and WIA; AES, FST; GameCube discs), u8, tpl, gxtex, dsp palette formats C4/C8/C14X2 in Python (the runtime's C++ gxtex has them), BRSTM/BRSAR, THP, BNR
3. Map code What does the code do, where? dol (DOL and ELF, one address map, symbols, --same-as, --libs), rso (the SDK's RSO modules and the static module's .sel: sections, relocations, exports and imports by name), cw (CodeWarrior demangler), ppc (Gekko decoder with paired singles, disassembly, callers, lis/addi and SDA xrefs, instruction census) ppc --mix without symbols; sig: library functions named by signature in stripped executables (Dolphin's .dsy, symbolised ELFs)
4. Translate Turn Gekko code into C++ recomp: units and entry points to a fixed point, switch tables, one C++ function per entry, dispatch table, CMake project; stripped executables: function discovery (recomp/discover.py), switch tables sized from the code, names and hooks from a symbols.tsv, a symbols.tsv of every unit written for the runtime's logs; RSO modules linked at virtual addresses into the executable's image and recompiled with it, their relocated immediates read from the module the game loaded faithful single-precision rounding
5. Runtime Replace the hardware ppc.h (the CPU model), core/mem (guest space, dispatch, hooks, the CPU's reads of the EFB), rso (where the game linked each RSO module: its code's run-time addresses made virtual), os (guest threads on host threads, interrupts, time), hw (PI, VI, DSP micro-codes, AI, EXI, SI, Hollywood; for a GameCube game the disc drive at its registers, 16 MB of ARAM, the controllers on SI), gx (FIFO parsing, the CP's FIFO breakpoints stopping the GP and its overflow behind them, EFB copies written into RAM on a port's request, vertex and texture decoding, the record), gxtex, gxshader (TEV and XF to GLSL), video (SDL3 window, OpenGL 4.5 renderer), ios + disc (IOS HLE at the IPC registers), sysconf, boot, wpad (the Wii Remote on the mouse and keys; the Nunchuk on keys; the Classic Controller on keys and SDL gamepads, up to four, in KPAD's status and WPAD's own samples; the connect and extension callbacks), ax (the AX micro-code, the Wii's and the GameCube's: its 5 ms frames, its per-millisecond updates, samples in ARAM) and audio (SDL3 output), a port's own layer (RtGameLayer), a sampling profiler, and wiiboot (the console from the disc: a GameCube game gets its 162 MHz bus and the IPL's globals, no IOS) memory cards on EXI, the GameCube's second DSP task (the memory card's unlock), the early (2006–07) AX micro-code, locked-cache DMA, synthetic Remote motion (swing, thrust, shake) from mouse gestures, fog and Z textures, Dolphin as the oracle

How the recompiler, the runtime, the renderer and the audio work is written up, with Victorious as the case, in pc-victorious's 09-recompiler, 11-runtime, 12-renderer and 13-audio.

Principles

  • Pure Python, no dependencies, for layers 1–4; the runtime (layer 5) is C++20, and its one dependency is SDL3, for the window and the sound. The one exception is speed, not function: aes uses pycryptodome when it is installed (1.3 MB/s in pure Python, a whole disc in about 18 minutes, against seconds), and gives the same bytes either way.
  • Every claim is checked on a real disc before it goes in.
  • Game knowledge stays out.
  • Every change is checked on every port before it goes in: each still gets as far as it did (Victorious boots, plays and sounds; the others as far as they have come).

Checks behind each module

Module Checked by
aes the FIPS-197 C.1 vector; equal to pycryptodome on a random cluster and on disc data (python -m wiikit.aes)
disc Victorious (WBFS): re-extraction equal to the previous extractor for all 46 files. A PAL disc (ISO): 3 751 files extracted. A GameCube disc (RVZ): 4 246 files, 1.35 GB, in 16 s, the offsets where its FST puts them
rvz a PAL disc as RVZ (zstd 19, 128 KiB chunks): every raw region, junk filler included, equal to DolphinTool's ISO; the whole DATA partition extracted from the RVZ equal to the ISO's, 3 759 of 3 759 files, in 57 s
dol --same-as: all ten DOL sections equal in Victorious's ELF
cw 20 619 of 20 619 function names demangled, including templates, conversion operators and anonymous namespaces
ppc Victorious: 1 658 815 instructions, none undecoded; equal to capstone on every non-paired-single instruction up to standard aliases; paired-single fields checked by prologue/epilogue symmetry in 1 393 functions. A stripped 2007 DOL: 744 768 words, 14 non-zero words undecoded (data in text)
gxtex The Last Story: byte-identical to textures Dolphin dumped from the running game
rso Monster Hunter Tri's 21 modules and its .sel: every relocation one of four kinds (ADDR32 in data, ADDR16_HA/LO on instructions' immediates, REL24 on branches), every import resolved by the .sel; 919 exports of .text each at a function start of the DOL
tpl, u8 the Home Button archives (105 files; the icon decodes correctly)
dsp Crystal Bearers' audio, by ear and spectrogram
recomp/discover Victorious's DOL against its ELF: 20 612 of 20 619 function starts found; 0 units cut through a function (no branch of any function crosses a unit start); 353 units of dead code after tail calls, harmless. A stripped 2007 PAL DOL: 8 738 units, 332 switch tables sized from the code (3 unresolved), no branch to an unknown target. Victorious's generated C++ unchanged, 201 of 201 files
recomp + runtime Victorious: all 20 653 functions compile and link; the game's own sprintf, strtod, 64-bit division, sin/cos, qsort with game comparators, PSMTX* paired-single matrices and memcpy/memset, run natively, match the host in 15 of 15 tests
runtime (hardware) Victorious boots from __start to its main loop: the SDK's own OSInit report, its anti-modchip device check, the Bink logos, Wwise on AX, frames of GX commands
runtime (renderer) Victorious, by eye: the Wii Strap screen, the Bink logos (indirect textures), the Scaleform title and menus, the episodes in 3D, at 30 frames a second. Draws in a row merged into one call: a stripped 2009 game's battles, 23 000 draws a frame (skinned models in strips of 4 to 10 vertices) in about 1 000 calls, 20 frames a second to 27-30; the other ports' boots unchanged
runtime (audio) Victorious, by ear: music, voices, effects, the Bink movies' sound, the rhythm games
runtime (Remote) Victorious, by hand: its first episode played with the mouse, the pointer under the mouse, the rhythm game's presses, holds and shakes
runtime (GameCube) a stripped 2004 GameCube game, from __start to its opening scenes: the SDK's own OSInit report ("Console Type : Retail 3", 24 MB), its disc read through the drive's registers, ARAM sized by ARInit's own check, its controller found, polled and played with scripted presses, its logos, title, menu and real-time scenes at 50 frames a second
runtime (GameCube audio) the same game, by ear: its music streamed through ARAM, its effects and voices, its opening movie's sound; a movie that waits for its sound now plays
recomp + runtime (RSO) Monster Hunter Tri: its executable and 21 modules, 1.59 M instructions, recompiled and linked at the first try; its modules linked by the game at addresses of its choosing, relinked elsewhere as it moves between the village and a quest, their code, virtual methods and _prologs run from the recompiled copy, by hand into a quest
runtime (FIFO breakpoints) The Last Story, which paces its frames on the GP (a breakpoint at each frame's end, a draw-sync token after it, the XFB flipped when the token is one past the last): from three frames to its first chapter at 30 frames a second; a busy frame larger than the FIFO, held behind a breakpoint, suspends the writer at the overflow and resumes it at the underflow
runtime (EFB copies in RAM) The Last Story, which composes its characters' dyed equipment with the GP and compresses the copy to CMPR on the CPU: the bodies' textures as on the console (black without them)
runtime (Classic Controller) a stripped 2010 game that learns of controllers only from the connect callback, by hand: played with an Xbox One pad, and with the keys, the mouse and the pad at once. A stripped 2009 game that reads the Classic from WPAD's raw samples (the 2007 KPAD's copy of them, WPADGetLatestIndexInBuf): played with the same pad into its first battle
runtime (Nunchuk) a stripped 2009 game that stops at its title without one: past it, its stick walking from the keys
runtime (shader cache) the same game's title and opening: 1 160 links for 396 programs in its first minute before, one per program after; a second run 337 of 354 programs from the disk, its stalls on the sound driver's clock from about 5 a second to under 1

Known gaps

  • Monster Hunter Tri's sound has short gaps: AX skips a frame when its DSP is not yet ready at the AI interrupt (the task manager's resume is a lead).
  • ppc.text renders standard simplified mnemonics (sub, clrrwi, mr., crclr…). Tools that need the canonical operation use decode()'s op and fields, never the text.
  • Constant tracking in ppc.Tracker is linear through a function and ignores control flow. That is enough for CodeWarrior's lis/addi pairs.
  • ppc --mix needs function symbols: on a stripped DOL it counts nothing.
  • Discovery misses assembly reached only by an address built in code (exception handlers) and tiny functions reached only by pointer: the runtime reports them as unknown targets, to be given as seeds.

History

wiikit grew inside pc-victorious from its first session to its seventh and was split out with its history (git subtree split) when a second port began. The commit messages of that period describe Victorious's sessions; the changes in them are wiikit's.

Licence

MIT — see LICENSE. wiikit contains no game data and no Nintendo code; it reads and replaces, it does not include.

About

A game-agnostic toolkit for Wii reverse engineering and native PC ports: disc images, executables, a Gekko decoder, a static recompiler to C++, and a runtime that replaces the Wii's hardware. No game data.

Topics

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages