Compare commits
5 Commits
f049db6921
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
dabdf51383 | ||
|
|
ec1295e39c | ||
|
|
f1dbc06425 | ||
|
|
43cc032e02 | ||
|
|
e390461a53 |
10
CLAUDE.md
10
CLAUDE.md
@@ -25,6 +25,8 @@ The four data sources, and which tools own them:
|
|||||||
- **Don't hammer the live server.** It is a real playtest backend. Warn the operator *before* any
|
- **Don't hammer the live server.** It is a real playtest backend. Warn the operator *before* any
|
||||||
action that makes repeated/abnormal connections. BattlEye is active in the game — all scraping is
|
action that makes repeated/abnormal connections. BattlEye is active in the game — all scraping is
|
||||||
done **outside** the game process (replayed protocol / captures / REST), never via injection.
|
done **outside** the game process (replayed protocol / captures / REST), never via injection.
|
||||||
|
- **A `/connect` scrape kicks the live player** (single session per account, newest wins — verified
|
||||||
|
2026-06-16, see `docs/MASTER_SERVER.md`). Don't open `/connect` while the operator is in-game.
|
||||||
|
|
||||||
## Environment & how to run
|
## Environment & how to run
|
||||||
|
|
||||||
@@ -56,6 +58,13 @@ load: gunzip -> XOR decrypt -> Newtonsoft-BSON parse
|
|||||||
If a game update changes the key, recover it with no RE via `walker/recover_key.py`.
|
If a game update changes the key, recover it with no RE via `walker/recover_key.py`.
|
||||||
- `pymongo`'s `bson.encode` reproduces Newtonsoft.Bson byte-for-byte, so decode→encode is identity.
|
- `pymongo`'s `bson.encode` reproduces Newtonsoft.Bson byte-for-byte, so decode→encode is identity.
|
||||||
|
|
||||||
|
### Walker naming convention
|
||||||
|
A walker's display name is **two indices**, not a string: top-level `firstNameIndex` + `secondNameIndex`
|
||||||
|
(BSON int32, **0–31 each**). Resolve them via `name_index.json` → `first_name` / `second_name` tables
|
||||||
|
(e.g. `(16,5)` = "Veteran Veteran", second `6` = "…Beast"). The top-level `name` field is **null/unused**
|
||||||
|
— the shown name comes from the indices. "name1" = `firstNameIndex`, "name2" = `secondNameIndex`. Set
|
||||||
|
both with `build_wbt.py rename <wbt> <first> <second> -o out`, or edit the index directly when copying.
|
||||||
|
|
||||||
### The 5 hashes (a `.wbt` is a serialized `WalkerBlueprintDto`)
|
### The 5 hashes (a `.wbt` is a serialized `WalkerBlueprintDto`)
|
||||||
|
|
||||||
All = `MD5(UTF8(JsonConvert.SerializeObject(obj))).hexUPPER` — Newtonsoft compact JSON: no whitespace,
|
All = `MD5(UTF8(JsonConvert.SerializeObject(obj))).hexUPPER` — Newtonsoft compact JSON: no whitespace,
|
||||||
@@ -155,6 +164,7 @@ All use UnityPy with an IL2CPP TypeTreeGenerator (`GameAssembly.dll` + `global-m
|
|||||||
- **`TASK.md`** — `.wbt` format cracked (BSON-verified) summary.
|
- **`TASK.md`** — `.wbt` format cracked (BSON-verified) summary.
|
||||||
- **`PRODUCTION_LINES.md`**, **`SALES_VALUE.md`**, **`WEAPON_DAMAGE.md`** — static-data location maps (track across updates).
|
- **`PRODUCTION_LINES.md`**, **`SALES_VALUE.md`**, **`WEAPON_DAMAGE.md`** — static-data location maps (track across updates).
|
||||||
- **`SCRAPE_RUNBOOK.md`** — read-only live-scrape steps for when a playtest is online.
|
- **`SCRAPE_RUNBOOK.md`** — read-only live-scrape steps for when a playtest is online.
|
||||||
|
- **`GHIDRA.md`** — headless Ghidra on `GameAssembly.dll`: **inject Il2CppDumper symbols, don't full-analyze** (`ghidra/scripts/apply_il2cpp_symbols.py`); targeted decompile/disasm; the `_JAVA_OPTIONS` heap gotcha. **The named DB is already built at `ghidra/project/SAND`** (564k methods, git-ignored/local) — decompile any function on demand via `-process … -postScript decomp_targets.py`.
|
||||||
- **`BUNDLES.md`** (repo root) — inventory of the 35 asset bundles.
|
- **`BUNDLES.md`** (repo root) — inventory of the 35 asset bundles.
|
||||||
|
|
||||||
Operator memory lives in `~/.claude/projects/-home-downloadpizza-sand-tools/memory/` (loaded each session).
|
Operator memory lives in `~/.claude/projects/-home-downloadpizza-sand-tools/memory/` (loaded each session).
|
||||||
|
|||||||
96
docs/GHIDRA.md
Normal file
96
docs/GHIDRA.md
Normal file
@@ -0,0 +1,96 @@
|
|||||||
|
# Ghidra headless on SAND's `GameAssembly.dll` (IL2CPP)
|
||||||
|
|
||||||
|
How to get a workable Ghidra database for the client, and the **big lesson**: for an IL2CPP binary
|
||||||
|
you **inject the symbol table from Il2CppDumper** — you do *not* sit through full auto-analysis.
|
||||||
|
|
||||||
|
> **CURRENT STATE (2026-06-16): the DB is already built and ready** at `ghidra/project/SAND`
|
||||||
|
> (~945 MB, **564,713 methods named**, 294,174 function boundaries, 32,958 string labels; symbol-inject,
|
||||||
|
> no auto-analysis). `ghidra/` is git-ignored so it's local/machine-specific — if it's missing, rebuild
|
||||||
|
> with the import command below (~17 min). Decompile any target right now via the `-process` command in
|
||||||
|
> "After the DB exists". (First real use: confirmed the master-server WS has **no cert pinning** — see
|
||||||
|
> `docs/MASTER_SERVER.md` / TLS notes.)
|
||||||
|
|
||||||
|
## Inputs (all already on disk)
|
||||||
|
- Binary: `/mnt/d/SteamLibrary/steamapps/common/Sand Playtest/GameAssembly.dll` (~137 MB).
|
||||||
|
- **Il2CppDumper ("yoten")**: `/mnt/c/Users/downloadpizza/Downloads/yoten/` — produces, for the
|
||||||
|
current build:
|
||||||
|
- `script.json` (~254 MB) — the **mapping**: `ScriptMethod[]` (Address+Name+Signature),
|
||||||
|
`ScriptString[]`, `ScriptMetadata[]`, `Addresses[]` (all function starts).
|
||||||
|
- `il2cpp.h` (~124 MB) — every struct/type.
|
||||||
|
- `dump.cs` (~79 MB) — human-readable signatures/RVAs (mirrored to `il2cpp/dump.cs`).
|
||||||
|
- Ready-made apply scripts: `ghidra.py` (names only), `ghidra_with_struct.py` (names+types+sigs),
|
||||||
|
plus IDA variants. **These use an interactive `askFile()` dialog → not headless-safe as shipped.**
|
||||||
|
- Ghidra 11.1.2 install: `ghidra/ghidra_install/` (`support/analyzeHeadless`). Java 17.
|
||||||
|
|
||||||
|
## The lesson: symbol-inject, not full analysis
|
||||||
|
Full auto-analysis of a 137 MB IL2CPP binary is the **wrong default**:
|
||||||
|
- The **Decompiler Parameter ID** analyzer is essentially single-threaded and runs over hundreds of
|
||||||
|
thousands of functions. Observed: **5h21m wall / ~5h40m CPU, pegged at ~105% (one core), with the
|
||||||
|
log silent for 4.5h and no checkpoint** — headless saves the project **only at the very end**, so a
|
||||||
|
crash/OOM mid-run loses everything. No progress %/ETA is emitted.
|
||||||
|
- It largely **rediscovers** what Il2CppDumper already knows exactly (function boundaries, names,
|
||||||
|
signatures). For our targeted-decompile workflow that's wasted time.
|
||||||
|
|
||||||
|
Instead: import with `-noanalysis` and run the dumper's symbol table in. You get a named,
|
||||||
|
function-bounded DB in well under an hour. On-demand decompilation (`decomp_targets.py`) does its own
|
||||||
|
per-function local analysis, so the global analyzers aren't needed for reading code.
|
||||||
|
|
||||||
|
### Headless-adapted applier — `ghidra/scripts/apply_il2cpp_symbols.py`
|
||||||
|
Adapted from `yoten/ghidra.py`: replaced `askFile()` with a script-arg / default path. Light path —
|
||||||
|
creates functions from `Addresses[]`, names them from `ScriptMethod[]`, labels string literals and
|
||||||
|
metadata. **No `il2cpp.h` import, no signatures** (those need the type archive; see below).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /home/downloadpizza/sand_tools
|
||||||
|
# fresh project, import without analysis, inject symbols, save (background + log):
|
||||||
|
rm -rf ghidra/project; mkdir -p ghidra/project
|
||||||
|
_JAVA_OPTIONS= nohup ghidra/ghidra_install/support/analyzeHeadless ghidra/project SAND \
|
||||||
|
-import "/mnt/d/SteamLibrary/steamapps/common/Sand Playtest/GameAssembly.dll" \
|
||||||
|
-noanalysis -overwrite \
|
||||||
|
-scriptPath ghidra/scripts -postScript apply_il2cpp_symbols.py \
|
||||||
|
> ghidra/headless_symbols.log 2>&1 &
|
||||||
|
# optional: pass a different script.json path as the postScript arg.
|
||||||
|
```
|
||||||
|
|
||||||
|
### After the DB exists: targeted decompile / disasm (instant, no re-analysis)
|
||||||
|
Put `rva<TAB>name` lines in `ghidra/targets.txt`, then `-process` the saved program:
|
||||||
|
```bash
|
||||||
|
_JAVA_OPTIONS= ghidra/ghidra_install/support/analyzeHeadless ghidra/project SAND \
|
||||||
|
-process GameAssembly.dll -noanalysis \
|
||||||
|
-scriptPath ghidra/scripts -postScript decomp_targets.py \
|
||||||
|
> ghidra/headless.log 2>&1
|
||||||
|
# -> ghidra/decomp.c (or disasm_targets.py -> ghidra/disasm.txt)
|
||||||
|
```
|
||||||
|
`decomp_targets.py`/`disasm_targets.py` already `disassemble()`+`createFunction()` per target, so they
|
||||||
|
work even on a bare `-noanalysis` import; with symbols injected they also resolve names/xrefs.
|
||||||
|
|
||||||
|
## Typed decompiles (optional, heavy)
|
||||||
|
For params shown as real types (`WalkerBlueprintDto *` …) use the `ghidra_with_struct.py` path: it
|
||||||
|
imports `il2cpp.h` (124 MB) into Ghidra's `DataTypeManager` via the C parser **first**, then applies
|
||||||
|
`ScriptMethod` signatures. The header parse is the slow / memory-hungry step (the usual OOM culprit).
|
||||||
|
Usually unnecessary — `il2cpp/dump.cs` already has every signature for reference. Only do it if you
|
||||||
|
specifically need typed struct fields in the decompiler.
|
||||||
|
|
||||||
|
## Address convention (verified)
|
||||||
|
Il2CppDumper `script.json` `Address` = the Ghidra **offset from image base** directly:
|
||||||
|
`baseAddress.add(Address)` (image base `0x180000000`). **No `-0x1000`.** (Note: the local
|
||||||
|
`ghidra/methods.tsv` index used by `reverse/resolve_decomp.py` stores `rva = scriptAddress - 0x1000`
|
||||||
|
for its own bookkeeping — different thing; don't conflate.)
|
||||||
|
|
||||||
|
## Memory / gotchas
|
||||||
|
- `analyzeHeadless` has `MAXMEM=8G` (already bumped). **But the shell exports `_JAVA_OPTIONS=-Xmx4g`**,
|
||||||
|
which silently caps the heap at 4 GB and causes swap thrash — always prefix runs with
|
||||||
|
`_JAVA_OPTIONS=` to clear it. Machine has ~11 GiB RAM.
|
||||||
|
- The run is detached via `nohup` (survives the session); it is **not** in tmux/screen. Watch with
|
||||||
|
`tail -f ghidra/headless_symbols.log`. `REPORT: Save succeeded` = done.
|
||||||
|
- `ghidra/` is git-ignored (install + project + dumps, all large/regenerable).
|
||||||
|
|
||||||
|
## Tooling map (`reverse/`, `ghidra/scripts/`)
|
||||||
|
> `ghidra/` is git-ignored, so the **tracked masters** live in `reverse/` (`ghidra_*.py`); copy them
|
||||||
|
> into `ghidra/scripts/` (where `-scriptPath` points) to run. e.g.
|
||||||
|
> `cp reverse/ghidra_apply_il2cpp_symbols.py ghidra/scripts/apply_il2cpp_symbols.py`.
|
||||||
|
- `reverse/ghidra_apply_il2cpp_symbols.py` → `ghidra/scripts/apply_il2cpp_symbols.py` — headless symbol injector (this doc).
|
||||||
|
- `ghidra/scripts/decomp_targets.py` — decompile `targets.txt` → `ghidra/decomp.c`.
|
||||||
|
- `ghidra/scripts/disasm_targets.py` — disassemble `targets.txt` → `ghidra/disasm.txt` (fast, no analysis).
|
||||||
|
- `reverse/il2cpp_re.py` — VA↔file-offset, method index from `dump.cs`, xrefs, body disasm + float consts.
|
||||||
|
- `reverse/resolve_decomp.py` — annotate `ghidra/decomp.c` with symbol names + string literals.
|
||||||
@@ -12,6 +12,25 @@ and the formula function (RVAs below); the literal constants need a different me
|
|||||||
end). This corrects an earlier draft that wrongly concluded "no value exists" — the values
|
end). This corrects an earlier draft that wrongly concluded "no value exists" — the values
|
||||||
**are** live at runtime; they just aren't statically anchorable constants.
|
**are** live at runtime; they just aren't statically anchorable constants.
|
||||||
|
|
||||||
|
**Confirmed 2026-06-16 (offline, `bundle/dump_blueprint.py`): the `Damage*DataComponent`s are
|
||||||
|
NOT authored on the item/ammo EntityBlueprints in the bundles.** Decoded `item_grenadeContact`,
|
||||||
|
`item_shotgunAmmo`, `item_pistolAmmo`, `item_shotgun`, `item_revolverSmall_dusters` — every one
|
||||||
|
has only generic components (InteractActions, Count, ItemName, ItemType, NiceName, View,
|
||||||
|
ViewSize, colliders, Physics); **zero** `Damage{Physical,…}DataComponent` / `MeleeDataComponent`
|
||||||
|
/ AoE. The only `.value` floats present are `ViewSizeDataComponent` (~0.3–0.97), not damage. So
|
||||||
|
`GetDamage`'s `DamageXxxDataComponent.value` reads are populated **at runtime (server-authoritative)**
|
||||||
|
— there is no per-weapon damage constant in the bundles *or* as a static anchor in the DLL.
|
||||||
|
|
||||||
|
**Ways to get actual base numbers, ranked:**
|
||||||
|
- ✅ **In-game empirical measurement** (controlled damage tests) — the only clean route.
|
||||||
|
- ⚠️ Live-client runtime memory (the components hold real values once spawned) — but that's
|
||||||
|
process inspection → **BattlEye / no injection** → off-limits.
|
||||||
|
- ❌ Static extraction (bundles) — values absent (proven above).
|
||||||
|
- ❌ Static decompile constant — none exists (generic Entitas dispatch).
|
||||||
|
- ❌ Master-server query — no damage field / no stats endpoint (see [MASTER_SERVER.md](MASTER_SERVER.md)).
|
||||||
|
- ❓ Game-server (not master) entity-snapshot capture *might* carry component values, but unverified
|
||||||
|
and the server may only transmit results, not per-weapon stats.
|
||||||
|
|
||||||
## Damage model (all static, all in `il2cpp/dump.cs` + verified by disasm)
|
## Damage model (all static, all in `il2cpp/dump.cs` + verified by disasm)
|
||||||
|
|
||||||
Per-type damage lives as a `float value` (object offset **+0x10**) on 8 components on the
|
Per-type damage lives as a `float value` (object offset **+0x10**) on 8 components on the
|
||||||
|
|||||||
87
reverse/ghidra_apply_il2cpp_symbols.py
Normal file
87
reverse/ghidra_apply_il2cpp_symbols.py
Normal file
@@ -0,0 +1,87 @@
|
|||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
# Headless-adapted Il2CppDumper -> Ghidra symbol applier ("fake PDB" from script.json).
|
||||||
|
# Names every method, string literal and metadata entry, and creates function boundaries
|
||||||
|
# from the dumper's address table -- WITHOUT Ghidra having to rediscover them via analysis.
|
||||||
|
# Light path: names + function starts only (no il2cpp.h struct import, no signatures).
|
||||||
|
#
|
||||||
|
# Adapted from yoten/ghidra.py: replaced the interactive askFile() with a script arg / default
|
||||||
|
# path so it runs under analyzeHeadless. Jython 2.7.
|
||||||
|
#
|
||||||
|
# Run (after the program is imported into the project):
|
||||||
|
# analyzeHeadless ghidra/project SAND -process GameAssembly.dll -noanalysis \
|
||||||
|
# -scriptPath ghidra/scripts -postScript apply_il2cpp_symbols.py [path/to/script.json]
|
||||||
|
# Default script.json: the live yoten dump.
|
||||||
|
# @category il2cpp
|
||||||
|
import json
|
||||||
|
|
||||||
|
DEFAULT_SCRIPT_JSON = "/mnt/c/Users/downloadpizza/Downloads/yoten/script.json"
|
||||||
|
|
||||||
|
processFields = ["ScriptMethod", "ScriptString", "ScriptMetadata", "ScriptMetadataMethod", "Addresses"]
|
||||||
|
|
||||||
|
baseAddress = currentProgram.getImageBase()
|
||||||
|
USER_DEFINED = ghidra.program.model.symbol.SourceType.USER_DEFINED
|
||||||
|
|
||||||
|
|
||||||
|
def get_addr(addr):
|
||||||
|
return baseAddress.add(addr)
|
||||||
|
|
||||||
|
|
||||||
|
def set_name(addr, name):
|
||||||
|
try:
|
||||||
|
createLabel(addr, name.replace(' ', '-'), True, USER_DEFINED)
|
||||||
|
except:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def make_function(start):
|
||||||
|
if getFunctionAt(start) is None:
|
||||||
|
try:
|
||||||
|
createFunction(start, None)
|
||||||
|
except:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
args = getScriptArgs()
|
||||||
|
path = args[0] if args else DEFAULT_SCRIPT_JSON
|
||||||
|
print("apply_il2cpp_symbols: loading " + path)
|
||||||
|
data = json.loads(open(path, 'rb').read().decode('utf-8'))
|
||||||
|
|
||||||
|
if "Addresses" in data and "Addresses" in processFields:
|
||||||
|
addresses = data["Addresses"]
|
||||||
|
monitor.initialize(len(addresses))
|
||||||
|
monitor.setMessage("Function boundaries")
|
||||||
|
for index in range(len(addresses) - 1):
|
||||||
|
make_function(get_addr(addresses[index]))
|
||||||
|
monitor.incrementProgress(1)
|
||||||
|
|
||||||
|
if "ScriptMethod" in data and "ScriptMethod" in processFields:
|
||||||
|
scriptMethods = data["ScriptMethod"]
|
||||||
|
monitor.initialize(len(scriptMethods))
|
||||||
|
monitor.setMessage("Methods")
|
||||||
|
for sm in scriptMethods:
|
||||||
|
set_name(get_addr(sm["Address"]), sm["Name"].encode("utf-8"))
|
||||||
|
monitor.incrementProgress(1)
|
||||||
|
|
||||||
|
if "ScriptString" in data and "ScriptString" in processFields:
|
||||||
|
scriptStrings = data["ScriptString"]
|
||||||
|
monitor.initialize(len(scriptStrings))
|
||||||
|
monitor.setMessage("Strings")
|
||||||
|
for i, ss in enumerate(scriptStrings, 1):
|
||||||
|
addr = get_addr(ss["Address"])
|
||||||
|
createLabel(addr, "StringLiteral_" + str(i), True, USER_DEFINED)
|
||||||
|
setEOLComment(addr, ss["Value"].encode("utf-8"))
|
||||||
|
monitor.incrementProgress(1)
|
||||||
|
|
||||||
|
if "ScriptMetadata" in data and "ScriptMetadata" in processFields:
|
||||||
|
for md in data["ScriptMetadata"]:
|
||||||
|
addr = get_addr(md["Address"])
|
||||||
|
set_name(addr, md["Name"].encode("utf-8"))
|
||||||
|
setEOLComment(addr, md["Name"].encode("utf-8"))
|
||||||
|
|
||||||
|
if "ScriptMetadataMethod" in data and "ScriptMetadataMethod" in processFields:
|
||||||
|
for mdm in data["ScriptMetadataMethod"]:
|
||||||
|
addr = get_addr(mdm["Address"])
|
||||||
|
set_name(addr, mdm["Name"].encode("utf-8"))
|
||||||
|
setEOLComment(addr, mdm["Name"].encode("utf-8"))
|
||||||
|
|
||||||
|
print("apply_il2cpp_symbols: done")
|
||||||
Reference in New Issue
Block a user